Rangle2/gemma-4-12B-it-uncensored-opus4.7-cot

TEXT GENERATIONConcurrent Unit Cost:1Model Size:12BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 22, 2026License:gemmaArchitecture:Transformer Featherless Exclusive Cold

Rangle2/gemma-4-12B-it-uncensored-opus4.7-cot is a 12 billion parameter Gemma-4 model fine-tuned via QLoRA, distilled from Claude Opus 4.7 chain-of-thought traces. This model focuses on recovering reasoning capabilities lost through abliteration, particularly in STEM-style verbal reasoning. It excels at mathematical problem-solving and structured deliberation, demonstrating significant improvements on benchmarks like GSM8K and MMLU.

Loading preview...

Overview

This model, gemma-4-12B-it-uncensored-opus4.7-cot, is a 12 billion parameter Gemma-4 variant that has undergone a QLoRA fine-tuning process. Its primary objective is to restore and enhance reasoning capabilities in an 'abliterated' (refusal-removed) base model by distilling chain-of-thought (CoT) traces from Claude Opus 4.7. The fine-tuning specifically targeted structured, deliberative reasoning without reintroducing refusal behaviors.

Key Capabilities & Performance

  • Reasoning Recovery: Benchmarks show substantial recovery of capabilities degraded by abliteration. On MMLU, it closes ~73% of the gap, and on GSM8K, it nearly fully restores mathematical ability (0.496 to 0.920).
  • Structured Thinking: Trained to 'think out loud' using a step-by-step approach, making it suitable for tasks requiring detailed derivations.
  • Uncensored Nature: Inherits the uncensored characteristic of its base model, meaning it will attempt to answer prompts that standard instruction-tuned models might decline.
  • Perplexity Improvement: Achieves a WikiText-2 perplexity of 1.717 bits/byte, even outperforming the clean base model.

Usage & Limitations

  • Recommended Prompting: Users should employ a system prompt encouraging step-by-step reasoning, followed by a clearly marked "Final answer:".
  • Generation Budget: Requires a generous generation budget (at least 768 tokens) to allow for full chain-of-thought derivations.
  • Specialization: Gains are concentrated in STEM-style verbal reasoning. Code generation may regress slightly due to increased verbosity.
  • Inherited Traits: The model is overconfident and rarely defers, inheriting the underlying abliterated direction. Safety-style phrases within CoT are not true guardrails; the model will still proceed to answer.

Finetunes
2 models