The layoric/llama-2-7B-alpaca-test model is a 7 billion parameter Llama 2 variant, fine-tuned using QLoRA on the Alpaca 2k test dataset. Developed by layoric, this model shows a small perplexity improvement over the base Llama v2 7B model on wikitext. It is designed for general language understanding and generation tasks, leveraging efficient QLoRA adaptation.
Loading preview...
Model Overview
layoric/llama-2-7B-alpaca-test is a 7 billion parameter language model based on the Llama 2 architecture. It has been fine-tuned using the QLoRA method, an efficient adaptation technique, on the mhenrichsen/alpaca_2k_test dataset. The fine-tuning process was conducted using Axolotl, a popular framework for LLM training.
Key Characteristics
- Base Model: NousResearch/Llama-2-7b-hf
- Parameter Count: 7 billion
- Context Length: 4096 tokens
- Fine-tuning Method: QLoRA (Quantized Low-Rank Adaptation)
- Dataset:
mhenrichsen/alpaca_2k_test - Training Framework: Axolotl
- Performance Note: Locally tested with a small perplexity improvement over the base Llama v2 7B model on the wikitext dataset.
Use Cases
This model is suitable for tasks requiring general language understanding and generation, particularly those benefiting from an Alpaca-style instruction following fine-tune. Its efficient QLoRA adaptation makes it a good candidate for experimentation and deployment in resource-constrained environments where a full 7B model might be too demanding.