AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b8000_s0
The AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b8000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for medical applications, having been trained on the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b8000_s0 dataset. It is designed to enhance performance in medical contexts, leveraging its 8192 token context length for processing longer medical texts.
Loading preview...
Model Overview
This model, AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b8000_s0, is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. It has been specifically fine-tuned on a medical dataset, capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b8000_s0, to specialize in medical language understanding and generation.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Specialization: Optimized for medical domain tasks through targeted fine-tuning.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05. Key hyperparameters included a train_batch_size of 2, eval_batch_size of 8, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. The optimizer used was ADAMW_TORCH with a cosine learning rate scheduler.
Intended Use Cases
Given its fine-tuning on a medical dataset, this model is intended for applications requiring robust understanding and generation of medical text. While specific use cases are not detailed, its specialization suggests utility in areas such as medical information extraction, clinical note summarization, or supporting medical research.