AmberYifan/capsd-medicine-dedup-marin-8b-base-medicine_ppl_b2000_s0
The AmberYifan/capsd-medicine-dedup-marin-8b-base-medicine_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for medical applications, leveraging a dedicated dataset for improved performance in the medicine domain. It is designed for tasks requiring specialized knowledge within the medical field.
Loading preview...
Overview
This model, AmberYifan/capsd-medicine-dedup-marin-8b-base-medicine_ppl_b2000_s0, is an 8 billion parameter language model. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically adapted for the medical domain.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports a context length of 8192 tokens.
- Specialization: Optimized for medical applications through fine-tuning on the
capsd_marin-8b-base-n80000-medicine-dedup80k__mix_medicine_ppl_b2000_s0dataset.
Training Details
The model was trained with a learning rate of 1e-05, a batch size of 2 (total batch size of 64 with gradient accumulation), and utilized a cosine learning rate scheduler with 0.03 warmup steps over 1 epoch. The training was distributed across 4 GPUs.
Intended Use Cases
Given its specialized fine-tuning on medical data, this model is intended for use cases within the healthcare and medical research sectors where domain-specific language understanding and generation are critical. Potential applications include medical text analysis, information extraction from clinical notes, or supporting medical question-answering systems.