AmberYifan/capsd-marin-8b-base-math_ifd_b8000_s0
AmberYifan/capsd-marin-8b-base-math_ifd_b8000_s0 is an 8 billion parameter language model, fine-tuned from marin-community/marin-8b-base. This model is specifically fine-tuned on a mathematical instruction-following dataset, suggesting an optimization for mathematical reasoning and problem-solving tasks. It is designed for use cases requiring robust performance in numerical and logical operations.
Loading preview...
Model Overview
AmberYifan/capsd-marin-8b-base-math_ifd_b8000_s0 is an 8 billion parameter language model derived from the marin-community/marin-8b-base architecture. This model has undergone a specific fine-tuning process using the capsd_marin-8b-base-n80000-numina__mix_math_ifd_b8000_s0 dataset.
Key Characteristics
- Base Model: Fine-tuned from
marin-community/marin-8b-base. - Parameter Count: 8 billion parameters.
- Context Length: 8192 tokens.
- Fine-tuning Focus: The model's training on a mathematically-oriented instruction-following dataset indicates a specialization in mathematical reasoning and problem-solving.
Training Details
The fine-tuning process utilized the following hyperparameters:
- Learning Rate: 1e-05
- Batch Size: A
train_batch_sizeof 2 andeval_batch_sizeof 8, with atotal_train_batch_sizeof 64 across 4 GPUs. - Optimizer: ADAMW_TORCH with default betas and epsilon.
- Scheduler: Cosine learning rate scheduler with 0.03 warmup steps.
- Epochs: Trained for 1 epoch.
Potential Use Cases
This model is likely suitable for applications requiring strong mathematical capabilities, such as:
- Solving mathematical problems.
- Generating mathematical explanations or proofs.
- Assisting with data analysis tasks involving numerical reasoning.