AmberYifan/capsd-marin-8b-base-math_qurating_b4000_s0
The AmberYifan/capsd-marin-8b-base-math_qurating_b4000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for mathematical reasoning and tasks, leveraging a specialized dataset for its training. It is designed to excel in applications requiring robust mathematical understanding and problem-solving capabilities. With an 8192 token context length, it can handle complex mathematical prompts.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-math_qurating_b4000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically trained on the capsd_marin-8b-base-n80000-numina__mix_math_qurating_b4000_s0 dataset, indicating a strong focus on mathematical reasoning and problem-solving.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192 token context window.
- Specialization: Optimized for mathematical tasks through targeted fine-tuning.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A total training batch size of 64 (2 per device with 8 gradient accumulation steps across 4 GPUs).
- Optimizer: ADAMW_TORCH with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
Intended Use Cases
This model is particularly well-suited for applications requiring strong mathematical capabilities, such as:
- Solving mathematical problems.
- Generating mathematical explanations.
- Assisting in quantitative analysis tasks.
Limitations
As the model card indicates, further information regarding specific intended uses and limitations is needed for a comprehensive understanding.