Model Overview
RefalMachine/ruadapt_mistral7b_full_vo_1e4 is a language model developed by RefalMachine, fine-tuned from an unspecified base model. While specific details regarding its architecture, training data, and parameters are not provided in the model card, its performance on the Open LLM Leaderboard offers insights into its capabilities.
Key Capabilities
- General Reasoning: Achieves a 55.46 score on the AI2 Reasoning Challenge (25-Shot).
- Common Sense Reasoning: Demonstrates strong performance with 79.55 on HellaSwag (10-Shot) and 74.43 on Winogrande (5-shot).
- Knowledge & Comprehension: Scores 60.34 on MMLU (5-Shot), indicating a reasonable grasp of various subjects.
- Truthfulness: Records 42.53 on TruthfulQA (0-shot).
Performance Metrics
The model's average score on the Open LLM Leaderboard is 56.88. Individual benchmark results include:
- AI2 Reasoning Challenge (25-Shot): 55.46
- HellaSwag (10-Shot): 79.55
- MMLU (5-Shot): 60.34
- TruthfulQA (0-shot): 42.53
- Winogrande (5-shot): 74.43
- GSM8k (5-shot): 28.96
Good For
- General-purpose language tasks requiring strong common sense and contextual understanding.
- Applications where HellaSwag and Winogrande performance are critical.
- Initial exploration for tasks that benefit from a balanced performance across various reasoning and comprehension benchmarks.