UNA-SOLAR-10.7B-Instruct-v1.0 Overview

UNA-SOLAR-10.7B-Instruct-v1.0 is a 10.7 billion parameter instruction-tuned language model developed by fblgit, leveraging Upstage.AI's SOLAR base model. It incorporates a technique called Uniform Neural Alignment (UNA) for supervised fine-tuning, which involves specific linear and intermediary layer configurations to regularize the training process. The model supports a context length of 4096 tokens.

Key Capabilities & Performance

This model demonstrates competitive performance across a range of benchmarks:

• Instruction Following (MT-Bench): Achieves an average score of 7.521875, placing it favorably among models like Vicuna-33B-v1.3 and WizardLM-30B.
• Reasoning: Scores 0.6954 accuracy on ARC Challenge and 0.671 exact match on GSM8K (5-shot).
• Truthfulness: Attains 0.7297 accuracy on TruthfulQA_MC2.
• General Knowledge (MMLU): Shows an overall accuracy of 0.6513, with strong performance in social sciences (0.7537 acc) and humanities (0.6077 acc).
• Big-Bench Hard (BBH): Achieves an overall exact match score of 0.466, with notable results in boolean expressions (0.816) and sports understanding (0.832).

Good For

• General Instruction Following: Its strong MT-Bench scores indicate proficiency in understanding and executing diverse instructions.
• Reasoning Tasks: Performance on ARC Challenge and GSM8K suggests suitability for tasks requiring logical deduction and mathematical problem-solving.
• Knowledge-intensive Applications: Solid MMLU scores make it a good candidate for applications requiring broad factual knowledge.