mlfoundations-dev/multiple_samples_majority_consensus_pick_one_numina_aime_math_verify
This model is a 7.6 billion parameter language model, fine-tuned from Qwen/Qwen2.5-7B-Instruct. It was trained on the mlfoundations-dev/multiple_samples_majority_consensus_pick_one_numina_aime_math_verify dataset, suggesting a specialization in tasks related to mathematical verification or consensus-based problem-solving. With a context length of 131072 tokens, it is designed for processing extensive inputs, likely for complex analytical or reasoning tasks.
Loading preview...
Model Overview
This model is a fine-tuned version of the Qwen/Qwen2.5-7B-Instruct base model, featuring 7.6 billion parameters and a substantial context length of 131072 tokens. It has been specifically trained on the mlfoundations-dev/multiple_samples_majority_consensus_pick_one_numina_aime_math_verify dataset.
Key Characteristics
- Base Model: Qwen/Qwen2.5-7B-Instruct
- Parameter Count: 7.6 billion
- Context Length: 131072 tokens, enabling the processing of very long inputs.
- Training Focus: Fine-tuned on a dataset related to "multiple samples majority consensus pick one numina aime math verify," indicating a potential specialization in mathematical reasoning, verification, or tasks requiring consensus from multiple outputs.
Training Details
The model was trained using a learning rate of 1e-05, a batch size of 1 per device across 32 GPUs (totaling 96 effective batch size with gradient accumulation), and a cosine learning rate scheduler with a 0.1 warmup ratio over 3 epochs. The training utilized Transformers 4.46.1 and Pytorch 2.3.0.