AmberYifan/capsdnum-marin-8b-base-math_ppl_b2000_s0
AmberYifan/capsdnum-marin-8b-base-math_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model was specifically trained on the capsd_marin-8b-base-n80000-numina__mix_math_ppl_b2000_s0 dataset, indicating a specialization towards mathematical reasoning and problem-solving. It is designed for tasks requiring numerical understanding and logical inference, leveraging its 8192-token context length.
Loading preview...
Model Overview
AmberYifan/capsdnum-marin-8b-base-math_ppl_b2000_s0 is an 8 billion parameter language model, fine-tuned from the marin-community/marin-8b-base architecture. This model has been specifically adapted through further training on the capsd_marin-8b-base-n80000-numina__mix_math_ppl_b2000_s0 dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Training Focus: The specific dataset used for fine-tuning suggests an emphasis on mathematical reasoning and numerical processing capabilities.
Training Details
The model was trained with a learning rate of 1e-05, a total batch size of 64 (achieved with a train batch size of 2 and 8 gradient accumulation steps across 4 devices), and a cosine learning rate scheduler. The training consisted of 1 epoch, utilizing Transformers 5.7.0 and PyTorch 2.13.0+cu130.
Potential Use Cases
Given its specialized training data, this model is likely suitable for applications requiring:
- Mathematical problem-solving.
- Numerical analysis and data interpretation.
- Tasks involving logical inference with quantitative information.