Tito-7B-slerp: A Serbian-Optimized Merged Language Model

Tito-7B-slerp is a 7 billion parameter model developed by Stopwolf, created through a spherical linear interpolation (slerp) merge of two base models: gordicaleksa/YugoGPT and mlabonne/AlphaMonarch-7B. This merging strategy aims to combine the strengths of its constituent models, with specific parameter weighting applied to self-attention and MLP layers.

Key Capabilities & Performance

This model is primarily distinguished by its strong performance on Serbian language evaluation suites. Benchmarks show Tito-7B-slerp achieving competitive and often leading results in tasks such as ARC-C, Hellaswag, and Winogrande when evaluated on Serbian datasets. For instance, it scored 47.82% average on a Serbian LLM eval suite, outperforming other models like Zamfir-7B and Mustra-7B. On the Open LLM Leaderboard, Tito-7B-slerp demonstrates an average score of 70.13, significantly higher than its YugoGPT base, with notable scores in HellaSwag (86.38%) and MMLU (64.01%).

Good for

• Serbian Language Applications: Ideal for tasks requiring high proficiency in the Serbian language, including content generation, translation, and understanding.
• Research into Model Merging: Provides a practical example of slerp merging for performance optimization in specific linguistic contexts.