TechxGenus/CodeGemma-2b
TechxGenus/CodeGemma-2b is a 2.5 billion parameter instruction-tuned causal language model, fine-tuned from Gemma-2b with an additional 0.7 billion high-quality, code-related tokens. It achieves 54.9 pass@1 on HumanEval-Python, demonstrating strong performance in code generation tasks. This model is optimized for coding instructions and operates using the Alpaca instruction format, making it suitable for developers focused on code-centric applications.
Loading preview...
CodeGemma-2b Overview
TechxGenus/CodeGemma-2b is a specialized 2.5 billion parameter language model, fine-tuned from the base Gemma-2b architecture. This model has undergone extensive training with an additional 0.7 billion high-quality, code-related tokens over three epochs, leveraging advanced techniques like DeepSpeed ZeRO 3 and Flash Attention 2 to optimize the training process.
Key Capabilities
- Code Generation: Achieves a notable 54.9 pass@1 on the HumanEval-Python benchmark, indicating strong performance in generating correct Python code from prompts.
- Instruction Following: Operates effectively using the Alpaca instruction format, making it suitable for tasks requiring clear instruction-response patterns in coding contexts.
- Efficient Training: Benefits from modern training optimizations, suggesting a well-engineered fine-tuning process.
Good For
- Code-centric Applications: Ideal for developers and researchers focused on tasks such as code completion, code generation, and programming assistance.
- Experimentation with Smaller Models: Provides a capable, smaller-scale model for exploring code generation without the computational overhead of larger models.
- Alpaca-formatted Instruction Tasks: Well-suited for integration into systems that utilize the Alpaca instruction format for code-related prompts.