AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b2000_s0
AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically fine-tuned on a medical dataset, indicating an optimization for medical domain-specific language understanding and generation. Its primary application is likely in tasks requiring knowledge of medical terminology and concepts.
Loading preview...
Model Overview
This model, AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned version of the marin-community/marin-8b-base model, specifically adapted using the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b2000_s0 dataset.
Training Details
The model underwent a single epoch of training with a learning rate of 1e-05. Key training hyperparameters include a train_batch_size of 2, an eval_batch_size of 8, and a gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. The optimizer used was ADAMW_TORCH with a cosine learning rate scheduler and 0.03 warmup steps. The training was conducted on a multi-GPU setup with 4 devices.
Potential Use Cases
Given its fine-tuning on a medical dataset, this model is likely optimized for applications within the medical domain. Developers might consider using it for tasks such as:
- Medical text analysis
- Generating medical summaries
- Answering medical-related questions
- Processing clinical notes
Limitations
The model card indicates that more information is needed regarding its specific capabilities, intended uses, and limitations. Users should perform thorough evaluations for their specific medical applications.