AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b4000_s0
AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b4000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for medical applications, having been trained on the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b4000_s0 dataset. Its primary differentiation lies in its specialized domain adaptation for medicine, making it suitable for tasks requiring medical knowledge. The model was trained with a learning rate of 1e-05 and a total batch size of 64 over one epoch.
Loading preview...
Model Overview
AmberYifan/capsd-medcase-marin-8b-base-medicine_ppl_b4000_s0 is an 8 billion parameter language model that has been fine-tuned from the marin-community/marin-8b-base architecture. This model is specifically designed for applications within the medical domain, leveraging a specialized dataset for its training.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Domain Specialization: Adapted for medical use cases through training on the
capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_ppl_b4000_s0dataset.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A total training batch size of 64 (train_batch_size: 2, gradient_accumulation_steps: 8).
- Optimizer: ADAMW_TORCH with betas=(0.9, 0.999) and epsilon=1e-08.
- LR Scheduler: Cosine type with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
- Frameworks: Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, Tokenizers 0.22.2.
Intended Use
This model is primarily intended for applications requiring specialized knowledge in the medical field due to its targeted fine-tuning. Users should consider its domain-specific training when evaluating its suitability for general-purpose tasks.