beberik/Nyxene-11B
beberik/Nyxene-11B is a 10.7 billion parameter language model created by beberik, built using a sophisticated merge of several 7B parameter models including Starling-LM-7B-alpha, NeuralHermes-2.5-Mistral-7B, juanako-7b-UNA, and dolphin-2.1-mistral-7b. This model leverages a slerp merge method with specific parameter weighting to combine the strengths of its constituent models. It achieves an average score of 67.72 on the Open LLM Leaderboard, demonstrating strong performance across various reasoning and language understanding tasks.
Loading preview...
Nyxene-11B: A Merged 10.7B Parameter Model
Nyxene-11B is a 10.7 billion parameter model developed by beberik, constructed by merging four distinct 7B parameter models: Starling-LM-7B-alpha, NeuralHermes-2.5-Mistral-7B, juanako-7b-UNA, and dolphin-2.1-mistral-7b. This model utilizes the mergekit tool, employing a slerp merge method with specific layer and tensor weighting to create a composite model.
Key Capabilities & Performance
Nyxene-11B demonstrates competitive performance on the Open LLM Leaderboard, achieving an average score of 67.72. Notable scores include:
- AI2 Reasoning Challenge (25-Shot): 68.34
- HellaSwag (10-Shot): 84.54
- MMLU (5-Shot): 65.09
- Winogrande (5-Shot): 79.08
Recommended Usage
The model is designed to be used with a specific prompt template for optimal performance:
<|system|>
Below is an instruction that describes a task. Write a response that appropriately completes the request.
<|user|>
{prompt}
<|assistant|>This structure helps guide the model to generate appropriate responses based on the given instructions. The detailed evaluation results can be found here.