diffnamehard/Psyfighter2-Noromaid-ties-13B

TEXT GENERATIONConcurrency Cost:1Model Size:13BQuant:FP8Ctx Length:4kPublished:Dec 28, 2023License:cc-by-nc-4.0Architecture:Transformer0.0K Open Weights Cold

diffnamehard/Psyfighter2-Noromaid-ties-13B is a 13 billion parameter language model created by diffnamehard, formed by merging KoboldAI/LLaMA2-13B-Psyfighter2 and NeverSleep/Noromaid-13b-v0.1.1 using the TIES merging method. This model combines the strengths of its base components, achieving an average benchmark score of 59.47 across various tasks. It is designed for general language generation tasks, leveraging its merged architecture for balanced performance.

Loading preview...

Model Overview

diffnamehard/Psyfighter2-Noromaid-ties-13B is a 13 billion parameter language model resulting from a TIES merge of two distinct models: KoboldAI/LLaMA2-13B-Psyfighter2 and NeverSleep/Noromaid-13b-v0.1.1. This merging strategy aims to combine the capabilities of both base models into a single, cohesive unit.

Merging Details

The model was created using the ties merge method, with specific parameters applied to the Noromaid component, including a density of 0.65 and a weighted merge strategy. The base model for the merge was LLaMA2-13B-Psyfighter2, with normalization and int8 masking enabled during the process. The final model uses float16 data type.

Performance Benchmarks

This merged model demonstrates a balanced performance across several common benchmarks:

  • Avg. Score: 59.47
  • ARC (25-shot): 61.86
  • HellaSwag (10-shot): 84.58
  • MMLU (5-shot): 57.04
  • TruthfulQA (0-shot): 50.66
  • Winogrande (5-shot): 75.37
  • GSM8K (5-shot): 27.29

Potential Use Cases

Given its merged nature and benchmark scores, this model is suitable for general-purpose text generation, conversational AI, and tasks requiring a blend of reasoning and creative capabilities, benefiting from the combined strengths of its constituent models.

Finetunes
3 models