INEX8-7B: A Merged 7B Language Model

INEX8-7B is a 7 billion parameter model developed by liminerity, constructed using a multi-stage slerp merging process via mergekit. This model integrates capabilities from several base models, including liminerity/merge3 and yam-peleg/Experiment26-7B, to create a versatile language understanding and generation system.

Key Capabilities & Performance

INEX8-7B demonstrates competitive performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard:

• Average Score: 76.44
• AI2 Reasoning Challenge (25-Shot): 73.29
• HellaSwag (10-Shot): 89.19
• MMLU (5-Shot): 64.47
• TruthfulQA (0-shot): 77.83
• Winogrande (5-shot): 84.85
• GSM8k (5-shot): 68.99

These scores indicate a balanced proficiency in reasoning, common sense, factual recall, and mathematical problem-solving. The model's architecture, derived from multiple merges, aims to combine the strengths of its constituent models.

Use Cases

Given its general-purpose nature and benchmark performance, INEX8-7B is suitable for a variety of applications requiring a capable 7B language model, such as:

• General text generation and completion
• Question answering
• Reasoning tasks
• Content creation

Its 4096-token context length supports processing moderately sized inputs and generating coherent responses.