kevinpro/MetaMathOctopus-MAPO-DPO-13B
The kevinpro/MetaMathOctopus-MAPO-DPO-13B is a 13 billion parameter language model developed by Shuaijie She et al. (MAPO project) with a 4096-token context length. It is specifically fine-tuned using Multilingual Alignment-as-Preference Optimization (MAPO-DPO) to enhance multilingual reasoning capabilities. This model excels in mathematical and numerical reasoning tasks across various languages, demonstrating strong performance on benchmarks like MSVAMP, MGSM, and MNumGLUESub.
Loading preview...
MetaMathOctopus-MAPO-DPO-13B: Enhanced Multilingual Reasoning
This model, developed by Shuaijie She et al. as part of the MAPO project, is a 13 billion parameter language model specifically designed to advance multilingual reasoning. It leverages a novel training approach called Multilingual Alignment-as-Preference Optimization (MAPO-DPO).
Key Capabilities & Differentiators
- Multilingual Reasoning: The core strength of this model lies in its ability to perform complex reasoning tasks across multiple languages, particularly in mathematical and numerical domains.
- MAPO-DPO Training: It utilizes a specialized preference optimization technique to align the model's outputs with desired multilingual reasoning patterns.
- Strong Benchmark Performance: The 13B variant, specifically the MetaMathOctopus base with MAPO-DPO, shows significant improvements over its base model and other comparable models like GPT-3.5-Turbo, MAmmoTH, WizardMath, and MetaMath on multilingual reasoning benchmarks. For instance, it achieves 67.0 on MSVAMP, 58.0 on MGSM, and 59.8 on MNumGLUESub, outperforming GPT-3.5-Turbo in these specific metrics.
Use Cases
This model is particularly well-suited for applications requiring:
- Multilingual Mathematical Problem Solving: Solving word problems, equations, and numerical tasks in various languages.
- Cross-lingual Reasoning: Tasks that involve understanding and generating logical inferences across different linguistic contexts.
- Educational Tools: Developing AI tutors or assessment systems for mathematics in diverse language settings.
For more technical details, refer to the associated research paper: MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization.