AmberYifan/capsdnum-marin-8b-base-math_cap_b2000_s0
The AmberYifan/capsdnum-marin-8b-base-math_cap_b2000_s0 model is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically optimized for mathematical tasks, leveraging a specialized dataset for its training. It is designed to enhance performance in numerical reasoning and mathematical problem-solving contexts. The model has a context length of 8192 tokens.
Loading preview...
Model Overview
This model, AmberYifan/capsdnum-marin-8b-base-math_cap_b2000_s0, is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. It has been fine-tuned with a focus on mathematical capabilities, utilizing the capsd_marin-8b-base-n80000-numina__mix_math_cap_b2000_s0 dataset. The training involved a single epoch with a learning rate of 1e-05 and a total batch size of 64, using a cosine learning rate scheduler.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Optimization: Specifically fine-tuned for mathematical tasks and numerical reasoning.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- Batch Size: A total training batch size of 64 (2 per device with 8 gradient accumulation steps on 4 GPUs).
- Epochs: 1
- Frameworks: Transformers 5.7.0, Pytorch 2.13.0+cu130, Datasets 4.0.0, Tokenizers 0.22.2.
Intended Use Cases
This model is suitable for applications requiring strong mathematical understanding and problem-solving abilities. Its specialized fine-tuning suggests improved performance in tasks involving numerical operations, mathematical reasoning, and quantitative analysis compared to general-purpose models of similar size.