Model Overview

StudentLLM/Alpagasus-2-13b-QLoRA-merged is a 13 billion parameter instruction-tuned language model, developed by Yunsang Yoo and Hyunwoo Ko. It is an unofficial implementation of the AlpaGasus methodology, which focuses on training a capable model with fewer data, applied to the Llama-2-13b-hf base model using QLoRA fine-tuning.

Key Capabilities & Training

• Base Model: Built upon the robust meta-llama/Llama-2-13b-hf architecture.
• Efficient Fine-tuning: Utilizes QLoRA (Quantized Low-Rank Adaptation) for efficient training on a single A100 80GB GPU.
• Dataset: Fine-tuned on a GPT-3.5-turbo filtered dataset, specifically 'alpaca_t45.json' from gpt4life, following an Alpaca-style prompt template.
• Language: Primarily supports English language tasks.

Performance Benchmarks

The model's performance has been evaluated on the Hugging Face OpenLLM Leaderboard, demonstrating its capabilities across various tasks:

• Average Score: 59.34
• MMLU: 55.27
• ARC: 61.09
• HellaSwag: 82.46
• TruthfulQA: 38.53

Licensing

This model is released under a Non-Commercial Creative Commons license (CC BY-NC-4.0), restricting its use to non-commercial applications.