pooraddyy/Koda-0.5B
Koda-0.5B is a 0.5 billion parameter instruction-tuned causal language model, fine-tuned by pooraddyy from Qwen/Qwen2.5-Coder-0.5B-Instruct, with a context length of 32768 tokens. It is specifically optimized for generating short Python functions and code snippets from natural language instructions. This model is ideal for developers needing quick Python code generation or for experimenting with small-model fine-tuning.
Loading preview...
Koda-0.5B: Python Code Generation from Natural Language
Koda-0.5B is a specialized 0.5 billion parameter language model, fine-tuned from Alibaba Cloud's Qwen/Qwen2.5-Coder-0.5B-Instruct. This model is designed to generate Python code from natural language instructions, making it a valuable tool for developers and those exploring small-scale model fine-tuning.
Key Capabilities
- Python Code Generation: Excels at creating short Python functions and code snippets based on user prompts.
- Instruction Following: Fine-tuned to understand and execute natural language instructions for coding tasks.
- Direct Use: Provided as a full merged model, meaning LoRA weights are integrated into the base, allowing direct loading with
transformerswithout needing PEFT.
Training Details
The model was fine-tuned using LoRA via PEFT on a subset of the iamtarun/python_code_instructions_18k_alpaca dataset. Training involved approximately 8.8 million trainable parameters (1.75% of the base model) over 2 epochs, achieving a final evaluation loss of 0.82. The training was performed in bfloat16 precision on CPU, taking about 2.26 hours.
Good for
- Generating concise Python functions or code snippets.
- Learning and experimentation with fine-tuning small language models.
Limitations
As a 0.5 billion parameter model, Koda-0.5B has inherent limitations. It may produce incorrect or insecure code, and its performance on other programming languages, lengthy programs, or complex reasoning tasks is limited. Users should always review generated code before deployment.