prog-love/gemma3-270m-beast
The prog-love/gemma3-270m-beast is a 270 million parameter instruction-tuned causal language model, fine-tuned from Google's Gemma-3-270m-it using Unsloth. This model is optimized for general instruction following, leveraging diverse datasets including code, mathematical reasoning, and conversational data. It is designed for efficient deployment on a single GPU, making it suitable for various natural language processing tasks.
Loading preview...
Model Overview
The prog-love/gemma3-270m-beast is a 270 million parameter instruction-tuned model, built upon Google's gemma-3-270m-it base. It was fine-tuned using Unsloth, a library designed for efficient training on a single GPU, making it accessible for developers with limited hardware resources.
Key Training Details
The model's instruction-following capabilities were enhanced through training on a curated selection of datasets:
- Glaive-Code 60k: Contributes to its understanding and generation of code-related content.
- MetaMathQA 60k: Improves its mathematical reasoning and problem-solving abilities.
- ShareGPT 3k: Provides a foundation for general conversational and instruction-following tasks.
The fine-tuning process utilized LoRA (Low-Rank Adaptation) with a rank of 32 and an alpha of 64, and the AdamW optimizer. The training concluded with a final loss of 2.4420.
Use Cases
This model is well-suited for applications requiring a compact yet capable instruction-tuned language model. Its training on diverse datasets suggests proficiency in:
- General instruction following
- Basic code generation and understanding
- Mathematical problem-solving
- Conversational AI and chatbots
Its efficient fine-tuning and smaller parameter count make it a good candidate for scenarios where computational resources are a constraint, offering a balance between performance and deployability.