aloobun/llama2-7b-guanaco
The aloobun/llama2-7b-guanaco model is a 7 billion parameter Llama-2-chat-hf variant, fine-tuned using QLoRA (4-bit precision) on the mlabonne/guanaco-llama2-1k dataset. This model is primarily intended for educational purposes, demonstrating fine-tuning techniques on a subset of the OpenAssistant/oasst1 dataset.
Loading preview...
Model Overview
aloobun/llama2-7b-guanaco is a 7 billion parameter language model based on the Llama-2-7b-chat-hf architecture. It has been fine-tuned using the QLoRA method with 4-bit precision. The training utilized the mlabonne/guanaco-llama2-1k dataset, which is a smaller subset derived from the OpenAssistant/oasst1 dataset.
Key Characteristics
- Base Model: Llama-2-7b-chat-hf
- Fine-tuning Method: QLoRA (4-bit precision)
- Training Data:
mlabonne/guanaco-llama2-1k(subset ofOpenAssistant/oasst1) - Training Environment: Trained on a single A100 GPU.
Intended Use
This model was developed primarily for educational purposes, serving as an example of fine-tuning a Llama-2 base model with QLoRA on a specific dataset. It can be valuable for developers and researchers looking to understand or experiment with efficient fine-tuning techniques for large language models.