Monarch-7B: A Merged 7B Language Model

Monarch-7B, developed by mlabonne, is a 7 billion parameter language model constructed through a merge of mlabonne/OmniTruthyBeagle-7B-v0, mlabonne/NeuBeagle-7B, and mlabonne/NeuralOmniBeagle-7B using the LazyMergekit tool. This model has achieved recognition for its strong performance on various benchmarks.

Key Capabilities and Performance

• Leaderboard Performance: Monarch-7B was noted as the best-performing model on the YALL leaderboard, indicating its competitive capabilities among similar-sized models.
• Benchmark Scores: It achieves an average score of 62.68 on the Nous suite evaluation, with specific scores including 45.48 on AGIEval, 77.07 on GPT4All, 78.04 on TruthfulQA, and 50.14 on Bigbench.
• Open LLM Leaderboard: On the Hugging Face Open LLM Leaderboard, Monarch-7B shows an average of 76.25, with notable scores like 73.04 on AI2 Reasoning Challenge, 89.03 on HellaSwag, 64.41 on MMLU, and 69.07 on GSM8k.

Configuration and Usage

The model was configured using a dare_ties merge method, incorporating specific density and weight parameters for its constituent models. It supports standard transformers library usage for text generation tasks, with a context length of 4096 tokens.