FuseAI/FuseChat-Gemma-2-9B-Instruct
FuseChat-Gemma-2-9B-Instruct is a 9 billion parameter instruction-tuned model from the FuseChat-3.0 series, developed by FuseAI. It leverages implicit model fusion (IMF) to integrate the strengths of larger source LLMs like Gemma-2-27B-It and Llama-3.1-70B-Instruct into a more compact Gemma-2-9B-It target model. This model excels in general conversation, instruction following, mathematics, and coding tasks, demonstrating significant improvements on benchmarks like AlpacaEval-2 and Arena-Hard through a two-stage Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) pipeline.
Loading preview...
FuseChat-Gemma-2-9B-Instruct: Implicit Model Fusion
FuseChat-Gemma-2-9B-Instruct is a 9 billion parameter model from the FuseChat-3.0 series, developed by FuseAI. It is built upon the Gemma-2-9B-It architecture and enhanced through a novel implicit model fusion (IMF) process. This method integrates the capabilities of powerful source LLMs such as Gemma-2-27B-It, Mistral-Large-Instruct-2407, Qwen-2.5-72B-Instruct, and Llama-3.1-70B-Instruct into a smaller target model.
Key Capabilities & Training:
- Implicit Model Fusion (IMF): Unlike previous explicit fusion methods, FuseChat-3.0 uses a three-stage IMF process:
- Dataset Construction: Responses are sampled from multiple source LLMs and annotated using an external reward model (ArmoRM).
- Supervised Fine-Tuning (SFT): The target model is fine-tuned on the best responses to enhance capabilities and reduce distributional discrepancies.
- Direct Preference Optimization (DPO): The model is further optimized using best and worst response pairs from source models as preference data.
- Comprehensive Training Data: Datasets were curated from sources like UltraFeedback, OpenMathInstruct-2, and LeetCode, covering instruction following, general conversation, mathematics, coding, and Chinese language tasks.
- Performance Improvements: The FuseChat-Gemma-2-9B-Instruct model shows notable gains, particularly on instruction-following benchmarks. It achieved 70.2% on AlpacaEval-2 and 63.4% on Arena-Hard, significantly outperforming its base Gemma-2-9B-It model.
Good for:
- Applications requiring strong instruction following and general conversational abilities.
- Tasks involving mathematics and coding, where it demonstrates improved accuracy.
- Developers seeking a compact yet capable model that benefits from the strengths of larger, more robust LLMs without their computational overhead.