Nyxene-11B: A Merged 10.7B Parameter Model

Nyxene-11B is a 10.7 billion parameter model developed by beberik, constructed by merging four distinct 7B parameter models: Starling-LM-7B-alpha, NeuralHermes-2.5-Mistral-7B, juanako-7b-UNA, and dolphin-2.1-mistral-7b. This model utilizes the mergekit tool, employing a slerp merge method with specific layer and tensor weighting to create a composite model.

Key Capabilities & Performance

Nyxene-11B demonstrates competitive performance on the Open LLM Leaderboard, achieving an average score of 67.72. Notable scores include:

• AI2 Reasoning Challenge (25-Shot): 68.34
• HellaSwag (10-Shot): 84.54
• MMLU (5-Shot): 65.09
• Winogrande (5-Shot): 79.08

Recommended Usage

The model is designed to be used with a specific prompt template for optimal performance:

<|system|>
Below is an instruction that describes a task. Write a response that appropriately completes the request.
<|user|>
{prompt}
<|assistant|>

This structure helps guide the model to generate appropriate responses based on the given instructions. The detailed evaluation results can be found here.