THU-KEG/Mistral-Crab-DPO
THU-KEG/Mistral-Crab-DPO is a 7 billion parameter Mistral-based language model developed by Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, and Juanzi Li. It is specifically fine-tuned using the CRAB dataset and Direct Preference Optimization (DPO) to significantly enhance its ability to follow complex instructions with implicit constraints. This model excels at tasks requiring precise adherence to format, length, and other nuanced instructions, making it suitable for applications demanding high instruction-following accuracy.
Loading preview...
Overview
THU-KEG/Mistral-Crab-DPO is a 7 billion parameter language model, fine-tuned from Mistral-7B-v0.3, developed by Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, and Juanzi Li. This model addresses the challenge of large language models struggling with complex instruction following, particularly those with implicit constraints like specific formats or lengths. The developers introduced a novel data generation technique called constraint back-translation.
Key Capabilities
- Enhanced Complex Instruction Following: The model is specifically trained on the CRAB dataset, which is generated by back-translating constraints into existing high-quality instruction-response pairs using advanced LLMs like Llama3-70B-Instruct. This process ensures that the added constraints are naturally met by the responses, reducing data noise.
- Improved Performance on Instruction-Following Benchmarks: Post-training on CRAB, combined with Direct Preference Optimization (DPO), significantly improves the model's scores on extensive instruction-following benchmarks such as IFEval and FollowBench (HSR).
- Robustness to Implicit Constraints: By focusing on constraints already present in high-quality data, the model learns to better interpret and adhere to nuanced instructions that might not be explicitly stated.
Performance Highlights
Compared to the base Mistral 7B model, Mistral-Crab-DPO shows substantial improvements across instruction-following metrics:
- IFEval: Improved from 25.2 to 59.4.
- FollowBench (HSR) AVG: Improved from 10.1 to 49.4.
Good For
- Applications requiring precise adherence to complex, multi-faceted instructions.
- Tasks where output format, length, or specific structural constraints are critical.
- Developers looking for a 7B model optimized for high-fidelity instruction following.