EvolCodeLlama-7b Overview

EvolCodeLlama-7b is a 7 billion parameter language model developed by mlabonne, fine-tuned from the codellama/CodeLlama-7b-hf base model. The fine-tuning process utilized QLoRA (4-bit precision) on the mlabonne/Evol-Instruct-Python-1k dataset, which focuses on Python instruction-following tasks. This model serves as an educational example for understanding LLM fine-tuning methodologies.

Key Characteristics

• Base Model: CodeLlama-7b-hf
• Fine-tuning Method: QLoRA (4-bit precision)
• Training Dataset: mlabonne/Evol-Instruct-Python-1k (Python-centric instruction data)
• Context Length: 4096 tokens
• Training Environment: Trained on an RTX 3090 in approximately 1 hour and 11 minutes.

Intended Use

This model is primarily designed for educational purposes to illustrate the process and results of fine-tuning a large language model for specific tasks. While it can generate Python code based on instructions, its main value lies in demonstrating the practical application of techniques like QLoRA and dataset curation for specialized model development, rather than being a production-ready inference model.