AmberYifan/capsd-medmcqa-marin-8b-base-medicine_cap_b10000_s0

Model Overview

This model, AmberYifan/capsd-medmcqa-marin-8b-base-medicine_cap_b10000_s0, is an 8 billion parameter language model built upon the marin-community/marin-8b-base architecture. It has been specifically fine-tuned on a specialized medical dataset, capsd_marin-8b-base-n80000-medicine-medmcqa__mix_medicine_cap_b10000_s0, to enhance its capabilities in the medical domain.

Key Characteristics

• Base Model: Fine-tuned from marin-community/marin-8b-base.
• Parameter Count: 8 billion parameters.
• Context Length: Supports an 8192 token context window.
• Specialization: Optimized for medical question answering and understanding through targeted fine-tuning.

Training Details

The model underwent a single epoch of training with a learning rate of 1e-05, a train_batch_size of 2, and gradient_accumulation_steps of 8, resulting in a total_train_batch_size of 64. It utilized the AdamW optimizer with a cosine learning rate scheduler. The training was performed using Transformers 5.7.0 and Pytorch 2.13.0+cu130.

Intended Use Cases

This model is primarily intended for applications requiring robust performance in medical contexts, such as:

• Medical question answering systems.
• Information retrieval from medical texts.
• Assisting with medical knowledge-based tasks.