MohdNihal03/qwen2.5-coder-1.5b-CodeSLM-Nihal

Model Overview

MohdNihal03/qwen2.5-coder-1.5b-CodeSLM-Nihal is a 1.5 billion parameter model, fine-tuned by Nihal using QLoRA on the Qwen2.5-Coder-1.5B-Instruct base model. Its primary objective is to transform the base model, which was noted for its verbosity, into a terse, code-first Python assistant that delivers minimal code solutions.

Key Capabilities & Differentiators

• Concise Code Generation: The fine-tuning process successfully reduced average output length by 78% (from 272 to 60 tokens) on held-out prompts, while preserving core algorithm correctness.
• Python-Focused: Optimized specifically for generating Python code.
• Efficient Training: Fine-tuned using QLoRA (4-bit NF4 base + LoRA) on the sahil2801/CodeAlpaca-20k dataset, completing 1 epoch in approximately 50 minutes on a single NVIDIA RTX 5060 (8 GB) GPU.
• Clean Convergence: Training and validation loss tracked closely, indicating no overfitting with a final validation mean token-accuracy of ~85.4%.

Limitations

• Style over Correctness: While output style is significantly improved, some subtle bugs from the base model persist, and one answer regressed in logic. Users should review generated code before use.
• Scope: Primarily Python-focused and, due to its size and 4-bit quantization, is not intended as a substitute for larger, frontier models.
• Instruction Drift: Occasional minor instruction drift (e.g., print vs return) may occur.

Usage

The model can be loaded directly using AutoModelForCausalLM and AutoTokenizer from the Hugging Face transformers library, or run via Ollama. It uses the Qwen2.5 ChatML prompt format, with temperature 0.2 and stopping on <|im_start|> / <|im_end|> recommended.