AmberYifan/capsdnum-marin-8b-base-math_ppl_b8000_s0
The AmberYifan/capsdnum-marin-8b-base-math_ppl_b8000_s0 model is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically optimized using the capsd_marin-8b-base-n80000-numina__mix_math_ppl_b8000_s0 dataset, suggesting a focus on mathematical reasoning and problem-solving tasks. With a context length of 8192 tokens, it is designed for applications requiring robust numerical and logical processing.
Loading preview...
Model Overview
The AmberYifan/capsdnum-marin-8b-base-math_ppl_b8000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base base model. This fine-tuning process utilized the capsd_marin-8b-base-n80000-numina__mix_math_ppl_b8000_s0 dataset, indicating a specialized focus on mathematical and numerical processing tasks.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Specialization: The training dataset suggests an optimization for mathematical reasoning and problem-solving.
Training Details
The model was trained with the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A total training batch size of 64 (2 per device with 8 gradient accumulation steps across 4 GPUs).
- Optimizer: AdamW with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
Intended Use Cases
Given its specialized training, this model is likely suitable for applications requiring:
- Mathematical problem-solving.
- Numerical analysis.
- Tasks involving logical reasoning with quantitative data.
Further details on specific intended uses, limitations, and comprehensive evaluation data are not provided in the current model card.