dhanushreddy29/BrokenKeyboardMerge
BrokenKeyboardMerge is an 8 billion parameter language model created by dhanushreddy29, formed by merging layers from dhanushreddy29/BrokenKeyboard and udkai/Turdus. This model leverages a passthrough merge method to combine the strengths of its constituent models, offering a balanced performance across various reasoning and common sense benchmarks. With an 8192-token context length, it is suitable for general-purpose text generation and understanding tasks.
Loading preview...
BrokenKeyboardMerge Overview
BrokenKeyboardMerge is an 8 billion parameter language model developed by dhanushreddy29, created through a merge of two distinct models: dhanushreddy29/BrokenKeyboard and udkai/Turdus. This merge was performed using LazyMergekit, specifically employing a passthrough merge method with float16 data type.
Key Capabilities & Performance
The model demonstrates a balanced performance across a range of benchmarks, as evaluated on the Open LLM Leaderboard. Its average score is 59.33, with notable results in:
- HellaSwag (10-Shot): 81.25
- Winogrande (5-shot): 78.69
- AI2 Reasoning Challenge (25-Shot): 59.73
- MMLU (5-Shot): 58.36
While performing well in common sense reasoning and general knowledge, its score on GSM8k (5-shot) is 25.93, indicating potential areas for improvement in complex mathematical reasoning. The model supports an 8192-token context length, making it suitable for tasks requiring moderate context understanding.
Usage
Developers can easily integrate BrokenKeyboardMerge into their projects using the transformers library, as demonstrated in the provided Python usage example. This allows for straightforward text generation and chat-template application.