fblgit/una-xaberius-34b-v1beta
The fblgit/una-xaberius-34b-v1beta is an experimental 34 billion parameter LLaMa-Yi-34B based language model developed by juanako.ai, featuring a 32K context length. It is trained using SFT, DPO, and a proprietary Uniform Neural Alignment (UNA) technique. This model achieved a 74.18 average score on the Open LLM Leaderboard, notably scoring 78.15 on MMLU, positioning it as a strong performer for general language understanding and reasoning tasks.
Loading preview...
UNA Xaberius 34B v1-BETA: A High-Performing LLaMa-Yi-34B Model
fblgit/una-xaberius-34b-v1beta is an experimental 34 billion parameter language model developed by juanako.ai, built upon the LLaMa-Yi-34B architecture. This model distinguishes itself through its training methodology, which incorporates Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and a unique proprietary technique called Uniform Neural Alignment (UNA).
Key Capabilities and Performance
- Leaderboard Performance: On December 8, 2023, Xaberius 34B achieved an average score of 74.18 on the Hugging Face Open LLM Leaderboard, outperforming several larger models.
- MMLU Record: It scored 78.15 on the MMLU (Massive Multitask Language Understanding) benchmark, setting a new record for 34B models and other open-source LLMs at the time of its evaluation.
- Training Innovation: The model utilizes UNA, described as a formula and technique to "tame" models, distinct from merging or SLERP methods.
Recommended Use Cases
- General Language Tasks: Excels in tasks requiring strong language understanding and reasoning, as indicated by its high MMLU score.
- Research and Development: Suitable for researchers and developers exploring advanced alignment techniques and high-performance 34B models.
- Instruction Following: Optimized for ChatML and Alpaca System prompt formats, making it effective for instruction-tuned applications.