AmberYifan/capsd-marin-8b-base-math_less_b1000_s0
AmberYifan/capsd-marin-8b-base-math_less_b1000_s0 is an 8 billion parameter language model fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for mathematical tasks, leveraging the capsd_marin-8b-base-n80000-numina__mix_math_less_b1000_s0 dataset. It is designed to enhance performance in mathematical reasoning and problem-solving within an 8192 token context length. Developers can utilize this model for applications requiring specialized mathematical capabilities.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-math_less_b1000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone specific fine-tuning to improve its proficiency in mathematical tasks.
Key Capabilities
- Mathematical Optimization: Fine-tuned on the
capsd_marin-8b-base-n80000-numina__mix_math_less_b1000_s0dataset, indicating a focus on mathematical reasoning and problem-solving. - Base Model: Built upon
marin-community/marin-8b-base, suggesting a foundation in general language understanding prior to specialized tuning. - Context Length: Supports an 8192 token context window, allowing for processing of moderately long inputs.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A total training batch size of 64 (2 per device across 4 GPUs with 8 gradient accumulation steps).
- Optimizer: ADAMW_TORCH with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
Intended Use Cases
This model is particularly suitable for applications that require enhanced mathematical understanding and generation. Its fine-tuning on a math-specific dataset suggests improved performance in areas such as:
- Solving mathematical problems.
- Generating mathematical explanations.
- Assisting with quantitative analysis tasks.