Inv/Konstanta-Alpha-V2-7B
Inv/Konstanta-Alpha-V2-7B is a 7 billion parameter language model created by Inv, merged from several pre-trained models including Kunoichi-DPO-v2-7B, PiVoT-0.1-Evil-a, ArchBeagle-7B, and Silicon-Alice-7B. This model leverages a multi-step DARE TIES and SLERP merge method, resulting in a model with a 4096 token context length. It achieves an average score of 72.35 on the Open LLM Leaderboard, demonstrating strong general reasoning and language understanding capabilities.
Loading preview...
Konstanta-Alpha-V2-7B Overview
Konstanta-Alpha-V2-7B is a 7 billion parameter language model developed by Inv, constructed through a sophisticated merging process of multiple pre-trained models. This model integrates components from SanjiWatsuki/Kunoichi-DPO-v2-7B, maywell/PiVoT-0.1-Evil-a, mlabonne/ArchBeagle-7B, and LakoMoor/Silicon-Alice-7B.
Merge Methodology
The model was created using a two-stage merging approach. Initially, DARE TIES was applied to merge Kunoichi with PiVoT Evil, and separately, ArchBeagle with Silicon Alice. These two resulting models were then combined using the gradient SLERP merge method. The configuration indicates a base model of mistralai/Mistral-7B-v0.1 for the initial DARE TIES steps, with ChatML identified as the most effective chat format.
Performance Highlights
Evaluations on the Open LLM Leaderboard show Konstanta-Alpha-V2-7B achieving an average score of 72.35. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 69.62
- HellaSwag (10-Shot): 87.14
- MMLU (5-Shot): 65.11
- TruthfulQA (0-shot): 61.08
- Winogrande (5-shot): 81.22
- GSM8k (5-shot): 69.90
These scores indicate a balanced performance across various reasoning, common sense, and language understanding tasks, making it suitable for general-purpose applications requiring robust language capabilities.