FuseChat-Qwen-2.5-7B-Instruct-Heretic Overview

This model is a decensored variant of the FuseAI/FuseChat-Qwen-2.5-7B-Instruct, created using the Heretic v1.0.1 tool. Its primary distinction lies in its significantly reduced refusal rate, dropping from 93/100 in the original model to 3/100, while maintaining a low KL divergence of 0.07.

Key Capabilities & Training

The base FuseChat-Qwen-2.5-7B-Instruct model is part of the FuseChat-3.0 series, which employs an implicit model fusion (IMF) process. This involves a two-stage training pipeline:

• Supervised Fine-Tuning (SFT): Utilizes best responses from powerful source LLMs (Gemma-2-27B-It, Mistral-Large-Instruct-2407, Qwen-2.5-72B-Instruct, Llama-3.1-70B-Instruct) to align the target model and mitigate distribution discrepancies.
• Direct Preference Optimization (DPO): Learns preferences by using best and worst response pairs from source models, further enhancing performance.

The training dataset comprises over 158,000 entries, covering instruction following, general conversation, mathematics, coding, and Chinese language tasks, with specific sampling strategies for each domain.

Performance Highlights

Evaluations across 14 benchmarks show that FuseChat-Qwen-2.5-7B-Instruct achieves an average score of 52.9, outperforming the original Qwen-2.5-7B-Instruct (50.0). Notably, it demonstrates substantial improvements in:

• AlpacaEval-2: 63.6% (vs. 33.2% for original)
• Arena-Hard: 61.4% (vs. 50.7% for original)
• MT-Bench: 9.0 (vs. 8.4 for original)
• AMC 23: 57.5% (vs. 52.5% for original)
• LiveCodeBench: 18.9% (vs. 15.8% for original)

Use Cases

This model is well-suited for applications requiring:

• Reduced content refusal: Ideal for scenarios where broader response generation is desired.
• General instruction following and conversation.
• Mathematical problem-solving.
• Code generation and understanding.
• Multilingual applications, particularly with strong Chinese language capabilities.