mlabonne/EvolCodeLlama-7b
EvolCodeLlama-7b is a 7 billion parameter CodeLlama-based model fine-tuned by mlabonne using QLoRA on the Evol-Instruct-Python-1k dataset. This model is specifically optimized for Python code generation and instruction following, leveraging a 4096-token context length. It is primarily designed for educational purposes to demonstrate fine-tuning techniques rather than for production inference.
Loading preview...
EvolCodeLlama-7b Overview
EvolCodeLlama-7b is a 7 billion parameter language model developed by mlabonne, fine-tuned from the codellama/CodeLlama-7b-hf base model. The fine-tuning process utilized QLoRA (4-bit precision) on the mlabonne/Evol-Instruct-Python-1k dataset, which focuses on Python instruction-following tasks. This model serves as an educational example for understanding LLM fine-tuning methodologies.
Key Characteristics
- Base Model: CodeLlama-7b-hf
- Fine-tuning Method: QLoRA (4-bit precision)
- Training Dataset:
mlabonne/Evol-Instruct-Python-1k(Python-centric instruction data) - Context Length: 4096 tokens
- Training Environment: Trained on an RTX 3090 in approximately 1 hour and 11 minutes.
Intended Use
This model is primarily designed for educational purposes to illustrate the process and results of fine-tuning a large language model for specific tasks. While it can generate Python code based on instructions, its main value lies in demonstrating the practical application of techniques like QLoRA and dataset curation for specialized model development, rather than being a production-ready inference model.