AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b1000_s0
AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b1000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model is specifically adapted for medical applications, having been trained on the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b1000_s0 dataset. It leverages a context length of 8192 tokens and is optimized for tasks within the medical domain.
Loading preview...
Model Overview
AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b1000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone specific fine-tuning to specialize in medical applications, utilizing the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b1000_s0 dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Domain Specialization: Explicitly fine-tuned on a medical dataset, indicating a focus on medical language understanding and generation.
Training Details
The model was trained with a learning rate of 1e-05, a train_batch_size of 2, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. It utilized the AdamW_TORCH optimizer with a cosine learning rate scheduler over 1 epoch. The training was conducted on 4 devices using a multi-GPU distributed setup.
Potential Use Cases
Given its specialized training, this model is likely suitable for tasks requiring an understanding of medical terminology and concepts, such as medical text analysis, information extraction from clinical notes, or generating medically relevant content.