Ayodele01/gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill
Ayodele01/gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill is a 7.9 billion parameter language model, fine-tuned from Google's Gemma-4 E4B base model. It specializes in complex reasoning tasks, including math, logic, and coding, by distilling high-quality chain-of-thought data from Gemini 3.1 Pro. This model maintains the base model's general capabilities while significantly enhancing its ability to solve intricate problems step-by-step.
Loading preview...
Overview
This model, Ayodele01/gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill, is a fine-tuned version of Google's Gemma-4 E4B base model, featuring 8 billion parameters (4 billion active). Its primary distinction lies in its specialized training on high-quality chain-of-thought reasoning data, distilled from Gemini 3.1 Pro. The fine-tuning process, utilizing LoRA with conservative hyperparameters, aimed to enhance reasoning capabilities without compromising the base model's existing strengths.
Key Capabilities
- Enhanced Reasoning: Specifically trained on approximately 13,000 high-quality reasoning examples covering diverse domains.
- Problem Solving: Excels in tasks requiring step-by-step solutions, including math, logic, coding, and complex problem-solving.
- Base Model Preservation: Designed to retain the general capabilities of the original Gemma-4 E4B model while integrating advanced reasoning skills.
Training Details
The model was fine-tuned using LoRA (r=8, alpha=8) with Unsloth, on a combined dataset from Roman1111111/gemini-3.1-pro-hard-high-reasoning and Roman1111111/gemini-3-pro-10000x-hard-high-reasoning. Evaluation results indicate strong performance in simple math (100%), logic reasoning (100%), and complex problems (75%), demonstrating its ability to learn reasoning styles effectively.