AmberYifan/capsd-medmcqa-marin-8b-base-medicine_random_b2000_s0

Model Overview

AmberYifan/capsd-medmcqa-marin-8b-base-medicine_random_b2000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone specialized fine-tuning to enhance its capabilities within the medical domain.

Key Specialization

• Medical Domain Adaptation: The model was fine-tuned on the capsd_marin-8b-base-n80000-medicine-medmcqa__mix_medicine_random_b2000_s0 dataset, indicating a focus on medical question-answering or related tasks.

Training Details

The fine-tuning process utilized specific hyperparameters:

• Learning Rate: 1e-05
• Batch Sizes: train_batch_size of 2, eval_batch_size of 8
• Gradient Accumulation: 8 steps, leading to a total_train_batch_size of 64
• Optimizer: ADAMW_TORCH with default betas and epsilon
• Scheduler: cosine learning rate scheduler with 0.03 warmup steps
• Epochs: Trained for 1 epoch

Framework Versions

The training environment included:

• Transformers 5.7.0
• Pytorch 2.13.0+cu130
• Datasets 4.0.0
• Tokenizers 0.22.2

Intended Use

This model is best suited for applications requiring specialized understanding and generation within the medical field, particularly for tasks aligned with its training on medical question-answering datasets.