MAISAAI/gemma-2b-coder
MAISAAI/gemma-2b-coder is a 2.6 billion parameter Gemma-2b model fine-tuned by MAISAAI on the CodeAlpaca 20k instructions dataset using QLoRA. This decoder-only language model is optimized for code generation and instruction-following tasks, leveraging its 8192 token context length. It excels at generating and editing code snippets, making it suitable for various programming assistance applications.
Loading preview...
Overview
MAISAAI/gemma-2b-coder is a 2.6 billion parameter model based on Google's Gemma-2b architecture, specifically fine-tuned for coding tasks. It leverages the lightweight, state-of-the-art capabilities of the Gemma family, which are derived from the same research as the Gemini models. This model was fine-tuned using the QLoRA method with the PEFT library on the CodeAlpaca 20k instructions dataset, which contains 20,000 instruction-following examples tailored for code generation.
Key Capabilities
- Code Generation: Proficient in generating code based on natural language instructions.
- Code Editing: Capable of modifying existing code snippets, as demonstrated by its ability to edit XML for web page navigation.
- Instruction Following: Designed to accurately follow coding-related instructions.
- Resource Efficiency: Its relatively small size (2.6B parameters) allows for deployment in environments with limited resources, such as laptops or desktops.
Training Details
The model was trained for 2 epochs on a Free Colab T4 GPU, completing in approximately 1 hour and 40 minutes. The training utilized a per_device_train_batch_size of 2 and gradient_accumulation_steps of 32, with a learning_rate of 2.5e-5. The training loss consistently decreased, indicating effective learning from the CodeAlpaca dataset.
Good For
- Developers seeking a compact yet capable model for code generation.
- Applications requiring assistance with programming tasks and code editing.
- Environments where computational resources are constrained.