AmberYifan/capsd-marin-8b-base-math_ifd_b2000_s0
AmberYifan/capsd-marin-8b-base-math_ifd_b2000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically adapted for mathematical tasks, leveraging the capsd_marin-8b-base-n80000-numina__mix_math_ifd_b2000_s0 dataset. It is designed to enhance performance in mathematical reasoning and problem-solving contexts.
Loading preview...
Model Overview
This model, capsd-marin-8b-base-math_ifd_b2000_s0, is an 8 billion parameter language model developed by AmberYifan. It is a fine-tuned variant of the marin-community/marin-8b-base architecture, specifically optimized for mathematical applications.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: Supports an 8192-token context window.
- Specialization: Enhanced for mathematical tasks through fine-tuning on the
capsd_marin-8b-base-n80000-numina__mix_math_ifd_b2000_s0dataset.
Training Details
The model was trained using the following hyperparameters:
- Learning Rate: 1e-05
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
- Batch Size: A total train batch size of 64 (2 per device with 8 gradient accumulation steps across 4 GPUs).
Intended Use Cases
This model is suitable for applications requiring strong mathematical reasoning and problem-solving capabilities, particularly within the domain it was fine-tuned for.