StudentLLM/Alpagasus-2-13b-QLoRA-merged
StudentLLM/Alpagasus-2-13b-QLoRA-merged is a 13 billion parameter auto-regressive language model developed by Yunsang Yoo and Hyunwoo Ko. It is an unofficial QLoRA fine-tune of Meta's Llama-2-13b-hf, based on the AlpaGasus methodology for efficient training with less data. This model is instruction-tuned using a GPT-3.5-turbo filtered dataset and is designed for general English language tasks, achieving an average score of 59.34 on the OpenLLM Leaderboard benchmarks.
Loading preview...
Model Overview
StudentLLM/Alpagasus-2-13b-QLoRA-merged is a 13 billion parameter instruction-tuned language model, developed by Yunsang Yoo and Hyunwoo Ko. It is an unofficial implementation of the AlpaGasus methodology, which focuses on training a capable model with fewer data, applied to the Llama-2-13b-hf base model using QLoRA fine-tuning.
Key Capabilities & Training
- Base Model: Built upon the robust
meta-llama/Llama-2-13b-hfarchitecture. - Efficient Fine-tuning: Utilizes QLoRA (Quantized Low-Rank Adaptation) for efficient training on a single A100 80GB GPU.
- Dataset: Fine-tuned on a GPT-3.5-turbo filtered dataset, specifically 'alpaca_t45.json' from gpt4life, following an Alpaca-style prompt template.
- Language: Primarily supports English language tasks.
Performance Benchmarks
The model's performance has been evaluated on the Hugging Face OpenLLM Leaderboard, demonstrating its capabilities across various tasks:
- Average Score: 59.34
- MMLU: 55.27
- ARC: 61.09
- HellaSwag: 82.46
- TruthfulQA: 38.53
Licensing
This model is released under a Non-Commercial Creative Commons license (CC BY-NC-4.0), restricting its use to non-commercial applications.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.