AmberYifan/capsd-medcase-marin-8b-base-medicine_cap_b8000_s0

Model Overview

AmberYifan/capsd-medcase-marin-8b-base-medicine_cap_b8000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specialized for medical applications through training on the capsd_marin-8b-base-n13092-medicine-medcase__mix_medicine_cap_b8000_s0 dataset.

Key Characteristics

• Base Model: Fine-tuned from marin-community/marin-8b-base.
• Parameter Count: 8 billion parameters.
• Context Length: Supports an 8192-token context window.
• Specialization: Optimized for tasks within the medical domain.

Training Details

The model was trained using the following hyperparameters:

• Learning Rate: 1e-05
• Batch Size: A total training batch size of 64 (with train_batch_size: 2 and gradient_accumulation_steps: 8).
• Optimizer: ADAMW_TORCH with default betas and epsilon.
• Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
• Epochs: Trained for 1 epoch.
• Frameworks: Utilized Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, and Tokenizers 0.22.2.

Intended Use

This model is primarily intended for applications requiring specialized language processing in the medical field. Its fine-tuning on a dedicated medical dataset suggests suitability for tasks such as medical text analysis, information extraction from clinical notes, or generating medically relevant content. Developers should consider its domain-specific training when evaluating its fit for their particular use cases.