Model Overview

ShahriarFerdoush/llama-3.2-1b-code-instruct is a 1 billion parameter, decoder-only causal language model, fine-tuned from meta-llama/Llama-3.2-1B. This model was developed by ShahriarFerdoush with a strong focus on code-related tasks. It leverages QLoRA (4-bit quantization + LoRA) for efficient adaptation, enabling training on limited hardware like a single Tesla P100 GPU.

Key Capabilities

• Code Generation: Primarily focused on Python, but generalizable to other languages.
• Code Reasoning: Capable of step-by-step algorithmic problem-solving.
• Educational Assistance: Useful for learning and practicing coding concepts.
• Resource Efficiency: Optimized for deployment and inference on low-VRAM GPUs, making it suitable for environments like Kaggle and Colab.

Training Details

The model was fine-tuned on 10,000 samples from the CodeAlpaca-20K dataset, which is specialized for coding tasks including algorithm implementation, debugging, and function writing. The QLoRA methodology, incorporating 4-bit NF4 quantization and LoRA adapters, significantly reduces GPU memory usage while maintaining performance. The training process took approximately 5 hours.

Intended Use

This model is ideal for research, educational tools, coding assistants, and rapid prototyping of LLM systems where efficient code handling and low resource consumption are critical. It is instruction-tuned rather than benchmark-optimized, and its 1B parameter size means it has limitations in complex, long-context reasoning and is not designed for general natural language chat.