BitStarWalkin/SuperCorrect-7B
BitStarWalkin/SuperCorrect-7B is a 7.6 billion parameter language model developed by Ling Yang, Zhaochen Yu, and their collaborators from Peking University, Skywork AI, UC Berkeley, and Stanford University. It utilizes a novel two-stage fine-tuning method called SuperCorrect to enhance reasoning accuracy and self-correction abilities. This model significantly outperforms other 7B models like DeepSeekMath-7B and Qwen2.5-Math-7B on mathematical benchmarks (MATH/GSM8K) by incorporating a hierarchical thought template (Buffer of Thought) for deliberate reasoning.
Loading preview...
SuperCorrect-7B: Enhanced Mathematical Reasoning
SuperCorrect-7B is a 7.6 billion parameter model developed by Ling Yang, Zhaochen Yu, and their collaborators, focusing on improving mathematical reasoning and self-correction in LLMs. It introduces a novel two-stage fine-tuning method called SuperCorrect.
Key Capabilities & Differentiators
- Superior Mathematical Performance: Achieves state-of-the-art results among 7B models on the MATH and GSM8K benchmarks, surpassing DeepSeekMath-7B by 7.8%/5.3% and Qwen2.5-Math-7B by 15.1%/6.3% respectively.
- Hierarchical Thought Template: Integrates a pre-defined hierarchical thought template, Buffer of Thought (BoT), for more deliberate and structured reasoning, distinct from conventional Chain-of-Thought (CoT).
- Error-Driven Insights: The SuperCorrect method leverages error-driven insights during its two-stage fine-tuning process to enhance accuracy and self-correction.
- Pure Mathematical Reasoning: Evaluation methods focus on the model's intrinsic mathematical reasoning abilities, without relying on external programming methods like Program-of-Thought (PoT) or Tree-of-Thought (ToRA).
Good For
- Complex Mathematical Problem Solving: Excels in tasks requiring detailed, step-by-step mathematical reasoning.
- Educational Applications: Can be used to generate detailed explanations and generalized solutions for math problems, aiding learning.
- Research in LLM Reasoning: Provides a strong baseline and methodology for further research into improving LLM reasoning and self-correction mechanisms.