OmniBeagle-7B: A Merged 7B Language Model

OmniBeagle-7B is a 7 billion parameter language model developed by mlabonne, created through a merge of three distinct BeagleSempra-7B variants: BeagleSempra-7B, BeagSake-7B, and WestBeagle-7B. This merge was performed using the DARE TIES method, with mistralai/Mistral-7B-v0.1 serving as the base model.

Key Capabilities & Performance

OmniBeagle-7B demonstrates competitive performance on the Open LLM Leaderboard, achieving an average score of 75.66. Specific benchmark results include:

• AI2 Reasoning Challenge (25-Shot): 72.61
• HellaSwag (10-Shot): 88.93
• MMLU (5-Shot): 64.80
• TruthfulQA (0-shot): 74.45
• Winogrande (5-shot): 83.11
• GSM8k (5-shot): 70.05

These scores indicate its proficiency in reasoning, common sense inference, language understanding, and mathematical problem-solving. The model operates with a context length of 4096 tokens.

Usage

This model can be easily integrated into projects using the Hugging Face transformers library, with provided Python code examples for text generation. Its architecture and merging strategy aim to combine the strengths of its constituent models, making it suitable for a range of general-purpose natural language processing tasks.