MAmmoTH2-7B-Plus: Enhanced Reasoning through Web-Scale Instruction Tuning

MAmmoTH2-7B-Plus, developed by TIGER-Lab, is a 7 billion parameter model based on the Mistral architecture, designed to significantly improve the reasoning capabilities of large language models. This model is a "Plus" variant, indicating further training on public instruction tuning datasets beyond its initial MAmmoTH2 version.

Key Capabilities & Differentiators

• Innovative Instruction Tuning: MAmmoTH2 models are distinguished by their unique training methodology, which involves efficiently harvesting 10 million instruction-response pairs from pre-training web corpora. This cost-effective approach provides large-scale, high-quality instruction data.
• Enhanced Reasoning Performance: The model shows substantial gains in reasoning benchmarks. For instance, the MAmmoTH2-7B base model's performance on MATH soared from 11% to 36.7% and on GSM8K from 36% to 68.4% without domain-specific data. The MAmmoTH2-7B-Plus further improves these scores, achieving 46.0% on MATH and 84.6% on GSM8K.
• Broad Benchmark Improvement: Evaluation across various datasets including TheoremQA, MATH, GSM8K, GPQA, MMLU-ST, BBH, and ARC-C demonstrates its strong general reasoning abilities.

Ideal Use Cases

• Mathematical Problem Solving: Excels in open-ended and multiple-choice math problems, making it suitable for educational tools, research, or applications requiring strong quantitative reasoning.
• Complex Reasoning Tasks: Its enhanced instruction following and reasoning capabilities make it effective for tasks requiring logical deduction and problem-solving beyond simple question-answering.
• Cost-Effective High Performance: Offers a powerful solution for reasoning tasks, developed through a cost-efficient data acquisition method, providing high performance without relying on expensive, domain-specific datasets.