reaperdoesntknow/Gemma-3-270m-Opus-Distil
reaperdoesntknow/Gemma-3-270m-Opus-Distil is an experimental 270 million parameter Gemma 3 model fine-tuned by Convergent Intelligence LLC. It uses a sparse fine-tuning setup and a custom CIxOpt optimizer to adapt the compact backbone towards reasoning-style text generation. This model is designed for research into efficient model adaptation and optimizer behavior, particularly for small-model reasoning tasks.
Loading preview...
Model Overview
This model, reaperdoesntknow/Gemma-3-270m-Opus-Distil, is an experimental fine-tuned derivative of Google's gemma-3-270m developed by Convergent Intelligence LLC. It utilizes a unique sparse fine-tuning approach and a custom CIxOpt optimizer framework to adapt the compact 270 million parameter Gemma 3 backbone specifically for reasoning-style text generation. The fine-tuning was performed on the angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k dataset, focusing on selective parameter participation rather than broad full-model modification.
Key Capabilities & Features
- Sparse Fine-Tuning: Employs a strategy to preserve the compact pretrained backbone while adapting selected higher-level surfaces for reasoning.
- Custom CIxOpt Optimizer: Uses a heterogeneous optimizer designed for architecture-aware routing, supporting various update styles (AdamW, Lion, AdaMax) and parameter-name-aware routing.
- Reasoning-Style Adaptation: Specifically trained to shape reasoning and response behavior through targeted updates.
- Experimental Research Checkpoint: Intended for evaluation, local testing, optimizer research, and continued fine-tuning.
Intended Use Cases
- Research on compact Gemma fine-tuning and efficient adaptation.
- Experiments with the CIxOpt optimizer.
- Small-model reasoning-style generation and instruction-following studies.
- Local text-generation experiments and prototyping.
- Comparison against the base
google/gemma-3-270m.
Limitations
As an experimental checkpoint, this model may hallucinate, inherit limitations from its base model, and be sensitive to prompt format. It has not been fully evaluated for factuality, safety, or complex reasoning, and its small size limits world knowledge and reasoning depth. It is not intended for high-stakes autonomous deployment.