TomGrc/FusionNet_linear
TomGrc/FusionNet_linear is a 10.7 billion parameter language model fine-tuned using the linear Fusion method of FusionNet. This model is an experimental fine-tune on English language data, demonstrating the application of the FusionNet technique. It achieves an average score of 74.43 across various benchmarks, including 71.25 on AI2 Reasoning Challenge and 66.35 on MMLU, indicating general language understanding and reasoning capabilities.
Loading preview...
Model Overview
TomGrc/FusionNet_linear is a 10.7 billion parameter language model developed by TomGrc. This model represents an experimental application of the linear Fusion method from FusionNet, fine-tuned specifically on English language data.
Key Capabilities & Performance
This model demonstrates general language understanding and reasoning, as evidenced by its performance on the Open LLM Leaderboard benchmarks. Key scores include:
- Average Score: 74.43
- AI2 Reasoning Challenge (25-Shot): 71.25
- HellaSwag (10-Shot): 88.44
- MMLU (5-Shot): 66.35
- TruthfulQA (0-shot): 71.94
- Winogrande (5-shot): 83.27
- GSM8k (5-shot): 65.35
Use Cases
Given its fine-tuned nature and benchmark performance, FusionNet_linear is suitable for tasks requiring general English language comprehension and reasoning. Its experimental nature with the FusionNet method makes it particularly interesting for researchers and developers exploring novel fine-tuning techniques and their impact on model performance.