reaperdoesntknow/Gemma-3-270m-Opus-Distil
reaperdoesntknow/Gemma-3-270m-Opus-Distil is a 0.3 billion parameter Gemma 3 model fine-tuned by Convergent Intelligence LLC for reasoning-style text generation. It utilizes a sparse fine-tuning approach with a custom CIxOpt optimizer, focusing on adapting selected parameters rather than full-model modification. This experimental checkpoint is designed for research into efficient small-model adaptation and optimizer behavior, particularly for lightweight reasoning tasks and local prototyping.
Loading preview...
Overview
reaperdoesntknow/Gemma-3-270m-Opus-Distil is an experimental 270M parameter model, a fine-tuned derivative of google/gemma-3-270m. Developed by Convergent Intelligence LLC, this model was adapted using a sparse fine-tuning setup and a custom CIxOpt optimizer framework. Its primary goal is to explore whether a compact Gemma 3 backbone can be effectively shaped for reasoning-style text generation through selective parameter participation, rather than broad full-model modification.
Key Capabilities
- Reasoning-style Text Generation: Fine-tuned on
angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7kdataset to enhance reasoning capabilities. - Sparse Fine-Tuning: Employs a strategy that freezes or reduces movement in lower representational structures while training selected higher-level adaptation surfaces.
- CIxOpt Optimizer: Utilizes a custom heterogeneous optimizer designed for architecture-aware routing, supporting various update styles (AdamW, Lion, AdaMax) and parameter-name-aware routing.
- Efficient Adaptation Research: Focuses on testing efficient adaptation of small models with custom optimizer routing.
Good For
- Research on Compact Gemma Fine-tuning: Ideal for studying how small Gemma models can be efficiently adapted.
- CIxOpt Optimizer Experiments: Useful for testing and understanding the behavior of the custom CIxOpt optimizer.
- Small-Model Reasoning-Style Generation: Suitable for lightweight reasoning experiments and prompt-response generation.
- Local Prototyping: Can be used for local text-generation experiments and small agent backbone testing.
- Instruction-Following and Response-Style Studies: Supports research into how models respond to instructions and generate text in specific styles.