Kukedlc/NeuTrixOmniBe-7B-model-remix
Kukedlc/NeuTrixOmniBe-7B-model-remix is a 7 billion parameter merged language model created by Kukedlc, combining CultriX/NeuralTrix-7B-dpo and paulml/OmniBeagleSquaredMBX-v3-7B-v2. This model leverages a slerp merge method to achieve a balanced performance across various benchmarks, including an average score of 76.30 on the Open LLM Leaderboard. With a 4096 token context length, it is designed for general-purpose language generation and understanding tasks, demonstrating strong capabilities in reasoning, common sense, and question answering.
Loading preview...
NeuTrixOmniBe-7B-model-remix Overview
NeuTrixOmniBe-7B-model-remix is a 7 billion parameter language model developed by Kukedlc, created through a strategic merge of two distinct base models: CultriX/NeuralTrix-7B-dpo and paulml/OmniBeagleSquaredMBX-v3-7B-v2. This merge was performed using LazyMergekit, employing a slerp method to combine their strengths.
Key Capabilities & Performance
This model demonstrates robust performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. It achieved an average score of 76.30, indicating strong general-purpose language understanding and generation. Notable scores include:
- AI2 Reasoning Challenge (25-Shot): 72.70
- HellaSwag (10-Shot): 89.03
- MMLU (5-Shot): 64.57
- TruthfulQA (0-shot): 76.90
- Winogrande (5-shot): 85.08
- GSM8k (5-shot): 69.52
These results highlight its proficiency in reasoning, common sense, factual recall, and mathematical problem-solving. The model operates with a context length of 4096 tokens.
What Makes This Model Different?
Unlike single-source models, NeuTrixOmniBe-7B-model-remix benefits from the slerp merging technique, which aims to combine the distinct strengths of its constituent models. This approach allows it to potentially inherit and balance the specialized capabilities of both NeuralTrix-7B-dpo and OmniBeagleSquaredMBX-v3-7B-v2, leading to a more versatile and well-rounded performance profile across diverse tasks compared to its individual components. Its balanced benchmark scores reflect this merged capability.