SauerkrautLM-gemma-2-9b-it: Resource-Efficient Bilingual Fine-Tuning

SauerkrautLM-gemma-2-9b-it is a 9 billion parameter instruction-tuned model from VAGO solutions, built upon Google's Gemma-2-9b-it. This model showcases the effectiveness of Spectrum Fine-Tuning for enhancing large language models with reduced computational resources.

Key Capabilities & Training:

• Bilingual Proficiency: Fine-tuned on a proprietary "German-English Sauerkraut Mix v2" dataset, focusing on high-quality German and English data.
• Resource-Efficient Enhancement: Utilizes Spectrum Fine-Tuning, targeting only 25% of the model's layers, to efficiently improve capabilities.
• Improved Performance: Demonstrates significant enhancements in instruction-following, common-sense reasoning, and mathematical tasks, as indicated by internal evaluations (AGIEVAL, GPT4ALL, TRUTHFULQA, OPENLEADERBOARD 2, MMLU 5-shot).
• Preserves Knowledge: The fine-tuning approach is designed to enhance specific skills while largely preserving the model's existing knowledge base.

Ideal Use Cases:

• Applications requiring strong performance in both German and English.
• Scenarios where resource-efficient model enhancement is critical.
• Tasks benefiting from improved instruction-following and reasoning abilities.