AmberYifan/capsd-medicine-dedup-marin-8b-base-medicine_random_b2000_s0

Model Overview

This model, AmberYifan/capsd-medicine-dedup-marin-8b-base-medicine_random_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for the medical domain. The fine-tuning process utilized the capsd_marin-8b-base-n80000-medicine-dedup80k__mix_medicine_random_b2000_s0 dataset, indicating a focus on medical text.

Training Details

The model was trained with a learning rate of 1e-05, a train_batch_size of 2, and gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. It used the AdamW optimizer and a cosine learning rate scheduler with 0.03 warmup steps over 1 epoch. The training was conducted on a multi-GPU setup with 4 devices.

Potential Use Cases

Given its fine-tuning on a medical dataset, this model is likely suitable for applications requiring specialized understanding of medical terminology and concepts. This could include tasks such as medical text analysis, information extraction from clinical notes, or supporting medical research.