reaperdoesntknow/Gemma-3-270m-Opus-Distil

Model Overview

This model, reaperdoesntknow/Gemma-3-270m-Opus-Distil, is an experimental 270 million parameter Gemma 3 derivative from Google, fine-tuned by Convergent Intelligence LLC. It utilizes a unique sparse fine-tuning setup and a custom CIxOpt optimizer framework to adapt the compact Gemma 3 backbone specifically for reasoning-style text generation. The training data was sourced from angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k.

Key Differentiators

• Sparse Fine-Tuning: Employs selective parameter participation rather than broad full-model modification, aiming to preserve the base model's structure while shaping reasoning capabilities.
• CIxOpt Optimizer: Uses a custom, heterogeneous optimizer designed for architecture-aware routing, allowing for different update styles (AdamW, Lion, AdaMax) based on parameter type.
• Research Focus: Primarily intended for research into efficient adaptation of small models, optimizer experiments, and understanding how compact models can be steered towards specific behaviors like reasoning.

Intended Use Cases

• Research on compact Gemma fine-tuning and efficient adaptation.
• Experiments with the CIxOpt optimizer.
• Small-model reasoning-style generation and instruction-following studies.
• Local text-generation experiments and prototyping.
• Comparison against the base google/gemma-3-270m.

Limitations

As an experimental checkpoint, it may hallucinate, inherit base model limitations, and be sensitive to prompt format. It has not been fully evaluated for factuality, safety, or complex reasoning, and its small size limits world knowledge and robustness.