fblgit/una-cybertron-7b-v2-bf16
The fblgit/una-cybertron-7b-v2-bf16 is a 7 billion parameter language model developed by Xavier M. at juanako.ai, based on the MistralAI architecture. This model is fine-tuned using SFT, DPO, and a proprietary Uniform Neural Alignment (UNA) technique, achieving a 69.67 score on the Hugging Face Open LLM Leaderboard. It excels in mathematics, logic, and reasoning, demonstrating deep contextual understanding and attention to detail.
Loading preview...
Model Overview
fblgit/una-cybertron-7b-v2-bf16, developed by Xavier M. at juanako.ai, is a 7 billion parameter model built upon the MistralAI architecture. This model distinguishes itself through its training methodology, which incorporates Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a unique proprietary technique called Uniform Neural Alignment (UNA). The UNA technique is described as a formula and method to "tame" models, and is not a merged layers approach like SLERP.
Key Capabilities & Performance
This model demonstrates strong performance across various benchmarks, achieving a notable 69.67 average score on the Hugging Face Open LLM Leaderboard, positioning it highly among models of all sizes. Specific benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 68.26
- HellaSwag (10-Shot): 85.85
- MMLU (5-Shot): 63.23
- TruthfulQA (0-Shot): 64.63
- Winogrande (5-Shot): 80.98
- GSM8k (5-Shot): 55.04
The model is noted for its proficiency in mathematics, logic, and reasoning, and is designed to provide deep reasoning over context and prompts without missing details.
Recommended Usage
While the model performs well with various prompts, optimal results are reported when using ChatML format or Alpaca System prompts. Users have also found good results with the exllamav2_HF loader, specifically with 8bpw-h8 exl2 quant and the simple-1 preset, and setting alpha=2.5 for 16K context.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.