dhmeltzer/Llama-2-7b-hf-eli5-cleaned-wiki65k-1024_qlora_merged
dhmeltzer/Llama-2-7b-hf-eli5-cleaned-wiki65k-1024_qlora_merged is a 7 billion parameter language model based on the Llama 2 architecture, fine-tuned using QLoRA. This model is specifically trained on a cleaned subset of Wikipedia data, focusing on 'Explain Like I'm 5' (ELI5) style content to enhance its ability to provide simplified explanations. It is optimized for tasks requiring clear, concise, and easy-to-understand responses, making it suitable for educational content generation or simplifying complex topics.
Loading preview...
Model Overview
dhmeltzer/Llama-2-7b-hf-eli5-cleaned-wiki65k-1024_qlora_merged is a 7 billion parameter language model built upon the Llama 2 architecture. It has been fine-tuned using the QLoRA method, specifically leveraging a cleaned dataset derived from Wikipedia, with a focus on 'Explain Like I'm 5' (ELI5) style content. This specialized training aims to improve the model's capacity for generating simplified and accessible explanations.
Key Capabilities & Performance
This model demonstrates a balanced performance across various benchmarks, with an average score of 43.55 on the Open LLM Leaderboard. Notable scores include:
- ARC (25-shot): 53.67
- HellaSwag (10-shot): 78.09
- MMLU (5-shot): 45.63
- TruthfulQA (0-shot): 41.72
- Winogrande (5-shot): 73.56
While its performance on mathematical reasoning (GSM8K) and reading comprehension (DROP) is lower, its strengths lie in general knowledge and common sense reasoning, likely benefiting from its ELI5-focused training.
Good For
- Generating simplified explanations of complex topics.
- Creating educational content that requires clear and concise language.
- Applications where easy-to-understand responses are prioritized.
- Tasks benefiting from a model fine-tuned on a curated Wikipedia dataset.