TobiasLogic/Qwen2.5-Coder-32B-Python-Specialist
TobiasLogic/Qwen2.5-Coder-32B-Python-Specialist is a 32.8 billion parameter instruction-tuned causal language model based on the Qwen2.5-Coder-32B architecture. Fine-tuned by TobiasLogic, this model specializes in Python coding tasks, multi-turn problem-solving, and strict formatting compliance. It leverages a 32K token context length and is optimized for generating high-quality Python code and following complex coding instructions.
Loading preview...
Model Overview
TobiasLogic/Qwen2.5-Coder-32B-Python-Specialist is a 32.8 billion parameter instruction-tuned model derived from the Qwen2.5-Coder-32B base. Developed by TobiasLogic, this model has undergone specialized fine-tuning to excel in Python programming contexts, focusing on instruction following, formatting compliance, and multi-turn coding problem-solving. It maintains the safety and alignment features of the original Qwen2.5 base model.
Key Capabilities
- Python Specialization: Enhanced proficiency in generating and understanding Python code.
- Instruction Following: Achieves state-of-the-art formatting compliance and adheres closely to complex coding instructions.
- Multi-turn Problem Solving: Capable of handling sequential coding challenges and conversations.
- Efficient Fine-tuning: Utilizes LoRA targeting only Attention layers, preserving the base model's extensive coding knowledge while optimizing for specific tasks.
- Context Length: Supports a substantial 32K token context, though optimized for dense instruction tuning at 512 tokens.
Training Details
The model was fine-tuned on a curated dataset of over 20,000 samples, combining CodeFeedback-Filtered-Instruction and python_code_instructions_18k_alpaca. This process, executed using Unsloth, focused on aggressively targeting the Attention layers (q_proj, k_proj, v_proj, o_proj) to improve instruction adherence without compromising the foundational knowledge of the 32B base model. The training achieved a low final loss of 0.45, indicating effective learning without overfitting.