SuperCorrect-7B: Enhanced Mathematical Reasoning

SuperCorrect-7B is a 7.6 billion parameter model developed by Ling Yang, Zhaochen Yu, and their collaborators, focusing on improving mathematical reasoning and self-correction in LLMs. It introduces a novel two-stage fine-tuning method called SuperCorrect.

Key Capabilities & Differentiators

• Superior Mathematical Performance: Achieves state-of-the-art results among 7B models on the MATH and GSM8K benchmarks, surpassing DeepSeekMath-7B by 7.8%/5.3% and Qwen2.5-Math-7B by 15.1%/6.3% respectively.
• Hierarchical Thought Template: Integrates a pre-defined hierarchical thought template, Buffer of Thought (BoT), for more deliberate and structured reasoning, distinct from conventional Chain-of-Thought (CoT).
• Error-Driven Insights: The SuperCorrect method leverages error-driven insights during its two-stage fine-tuning process to enhance accuracy and self-correction.
• Pure Mathematical Reasoning: Evaluation methods focus on the model's intrinsic mathematical reasoning abilities, without relying on external programming methods like Program-of-Thought (PoT) or Tree-of-Thought (ToRA).

Good For

• Complex Mathematical Problem Solving: Excels in tasks requiring detailed, step-by-step mathematical reasoning.
• Educational Applications: Can be used to generate detailed explanations and generalized solutions for math problems, aiding learning.
• Research in LLM Reasoning: Provides a strong baseline and methodology for further research into improving LLM reasoning and self-correction mechanisms.