Prima-LelantaclesV5-7b: A DARE TIES Merged Model

ChaoticNeutrals/Prima-LelantaclesV5-7b is a 7 billion parameter language model developed by ChaoticNeutrals, created through a DARE TIES merge. This model combines the strengths of Test157t/Pasta-Lake-7b and Test157t/Prima-LelantaclesV4-7b-16k.

Key Capabilities & Performance

• Extended Context Window: Unexpectedly performs well at a 16384-token context length, which is a notable improvement over typical Mistral-based models that usually operate around 8192 tokens.
• Merged Architecture: Utilizes the DARE TIES merge method, indicating a focus on combining and enhancing capabilities from its constituent models.
• Leaderboard Performance: Achieves a competitive average score of 73.09 on the Open LLM Leaderboard, with specific scores including:
  • AI2 Reasoning Challenge (25-Shot): 70.65
  • HellaSwag (10-Shot): 87.87
  • MMLU (5-Shot): 64.52
  • TruthfulQA (0-shot): 68.26
  • Winogrande (5-shot): 82.40
  • GSM8k (5-shot): 64.82

Use Cases

This model is well-suited for applications requiring:

• General language understanding and generation.
• Tasks benefiting from an extended context window, such as summarizing longer documents or maintaining conversational coherence over more turns.
• Reasoning tasks, as indicated by its performance on benchmarks like AI2 Reasoning Challenge and GSM8k.