Brillibits/Instruct_Llama70B_Dolly15k
Instruct_Llama70B_Dolly15k is a 69 billion parameter instruction-tuned language model developed by Brillibits, based on the Llama 2 transformer architecture. Fine-tuned using the Dolly15k dataset, it is designed for general English language tasks. The model was trained for 1.5 epochs with a 1024 token context window, achieving an average score of 60.97 on the Open LLM Leaderboard.
Loading preview...
Model Overview
Brillibits/Instruct_Llama70B_Dolly15k is a 69 billion parameter instruction-tuned language model built upon the Llama 2 transformer architecture. Developed by Brillibits, this model was fine-tuned using the Dolly15k dataset, with 80% of the data allocated for training, 15% for validation, and 5% for testing. The training process involved 1.5 epochs utilizing QLoRA, with a context window of 1024 tokens.
Key Capabilities
- Instruction Following: Designed to respond to various prompts, including those with and without explicit context, as demonstrated by its prompt templates.
- English Language Tasks: Primarily focused on English language generation and understanding.
- Llama 2 Architecture: Benefits from the robust and widely recognized Llama 2 base model.
Performance Metrics
Evaluated on the Open LLM Leaderboard, the model achieved an average score of 60.97. Notable individual benchmark scores include:
- ARC (25-shot): 68.34
- HellaSwag (10-shot): 87.21
- MMLU (5-shot): 69.52
- TruthfulQA (0-shot): 46.46
- Winogrande (5-shot): 84.29
- GSM8K (5-shot): 42.68
- DROP (3-shot): 28.26
Intended Use
This model is suitable for general instruction-following tasks in English. For specialized applications or custom data integration, Brillibits offers professional assistance.