itsliupeng/llama2_7b_mmlu
The itsliupeng/llama2_7b_mmlu model is a 7 billion parameter Llama-2-based causal language model continuously trained by itsliupeng. It is specifically optimized to enhance performance on MMLU benchmarks, utilizing the mmlu_recall dataset. This model maintains a 4096-token context length and aims to improve MMLU scores without negatively impacting other metric performances.
Loading preview...
Model Overview
The itsliupeng/llama2_7b_mmlu is a 7 billion parameter language model built upon the Llama-2-7b-hf architecture. Developed by itsliupeng, this model undergoes continuous training using the mmlu_recall dataset with the primary objective of significantly improving its performance on MMLU (Massive Multitask Language Understanding) benchmarks.
Key Capabilities & Performance
This model focuses on specialized improvement in MMLU while striving to preserve performance across other general language understanding tasks. Evaluation results from the Open LLM Leaderboard highlight its specific strengths and current standing:
- MMLU (5-shot): Achieves a score of 60.04.
- HellaSwag (10-shot): Scores 79.13.
- ARC (25-shot): Reaches 56.14.
- TruthfulQA (0-shot): Scores 40.95.
- Winogrande (5-shot): Achieves 74.43.
Use Cases
This model is particularly well-suited for applications requiring strong performance in complex, multi-disciplinary reasoning and knowledge-based tasks, as indicated by its targeted MMLU optimization. Developers can leverage this model for scenarios where robust understanding and accurate responses across a broad range of academic and professional subjects are critical.