mlabonne/PyLlama-7b
PyLlama-7b is a 7 billion parameter CodeLlama-based model developed by mlabonne, fine-tuned using QLoRA (4-bit precision) on the Evol-Instruct-Python-26k dataset. This model specializes in Python code generation and understanding, leveraging its base architecture for robust performance in programming tasks. It is optimized for developers seeking an efficient, specialized LLM for Python-centric applications.
Loading preview...
PyLlama-7b Overview
PyLlama-7b is a 7 billion parameter language model developed by mlabonne, specifically fine-tuned for Python programming tasks. It is built upon the codellama/CodeLlama-7b-hf base model and was fine-tuned using QLoRA (4-bit precision) on the mlabonne/Evol-Instruct-Python-26k dataset.
Key Capabilities
- Python Code Generation: Optimized for generating Python code based on instructions.
- Code Understanding: Enhanced ability to interpret and process Python-related queries.
- Efficient Fine-tuning: Utilizes QLoRA for efficient fine-tuning, making it accessible for specialized applications.
Training Details
The model was trained on an RTX 3090 over approximately 10 hours. The training configuration involved a sequence_len of 2048, micro_batch_size of 10, and 3 epochs, with flash_attention enabled for performance. This focused training on a Python-specific instruction dataset aims to deliver strong performance in its niche.
Good For
- Developers requiring a specialized LLM for Python code generation.
- Applications involving Python scripting, code completion, or code explanation.
- Environments where a 7B parameter model offers a good balance of performance and resource efficiency for Python-centric tasks.