fblgit/UNA-SimpleSmaug-34b-v1beta
UNA-SimpleSmaug-34b-v1beta by fblgit is a 34 billion parameter language model based on the Smaug architecture, fine-tuned with the SimpleMath dataset. It significantly improves mathematical and reasoning capabilities, scoring 77.41 on average across various benchmarks, including 72.48 on GSM8k and 76.68 on MMLU. This model is optimized for tasks requiring strong numerical and logical processing while preserving general language understanding.
Loading preview...
UNA-SimpleSmaug-34b-v1beta Overview
UNA-SimpleSmaug-34b-v1beta is a 34 billion parameter model developed by fblgit, built upon the Smaug-34B base model. It was fine-tuned using the Axolotl framework, specifically applying the UNA method to the attention layers with the SimpleMath dataset. This targeted training aimed to enhance the model's mathematical and reasoning abilities without degrading its existing language understanding.
Key Capabilities & Performance
The model demonstrates strong performance in reasoning and mathematical tasks, achieving an average score of 77.41 on the Open LLM Leaderboard (as of February 2024). Notable benchmark results include:
- AI2 Reasoning Challenge (ARC): 74.57
- GSM8k: 72.48
- MMLU: 76.68
- HellaSwag: 86.74
- TruthfulQA: 70.17
- Winogrande: 83.82
These scores indicate significant improvements in areas like arithmetic, logical deduction, and general knowledge compared to its base model. The developers confirmed the model's lineage and improvements using their ModelSimilarities tool detector.
Ideal Use Cases
- Mathematical Problem Solving: Excels in tasks requiring numerical reasoning and arithmetic.
- Logical Deduction: Suitable for applications needing strong analytical and reasoning capabilities.
- General Purpose Language Tasks: Maintains robust performance in broader language understanding due to preserving previous training sessions.