Open_Neural_Monarch_Maidv0.1 Overview

This model, created by giraffe176, is a 7 billion parameter language model built upon the mistralai/Mistral-7B-v0.1 base. It was developed using the DARE TIES merge method from mergekit, combining several specialized models to enhance its capabilities.

Key Components and Merge Details

The Open_Neural_Monarch_Maidv0.1 integrates the strengths of four distinct models:

• Intel/neural-chat-7b-v3-1
• NeverSleep/Noromaid-7B-0.4-DPO
• teknium/OpenHermes-2.5-Mistral-7B
• mlabonne/Monarch-7B

Each component was weighted and integrated using specific density parameters, with int8_mask enabled and bfloat16 precision for the merge process.

Performance Highlights

Evaluated on the Open LLM Leaderboard, Open_Neural_Monarch_Maidv0.1 demonstrates competitive performance with an average score of 69.28. Notable scores include:

• AI2 Reasoning Challenge (25-Shot): 67.66
• HellaSwag (10-Shot): 85.94
• MMLU (5-Shot): 65.02
• Winogrande (5-Shot): 79.32
• GSM8k (5-Shot): 61.33

These results indicate a well-rounded model capable of handling a variety of reasoning, common sense, and language understanding tasks. Detailed evaluation results are available on the Hugging Face Open LLM Leaderboard.