cyberneurova/CyberNeurova-Gemma-4-12B-it-abliterated

TEXT GENERATIONConcurrent Unit Cost:1Model Size:12BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 7, 2026License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

CyberNeurova's Gemma-4-12B-it-abliterated is a 12-billion parameter encoder-free trimodal model, based on Google's Gemma 4, with a 32768 token context length. This variant has undergone 'refusal-direction abliteration,' a process that removes safety alignment features from the model's residual stream, making it suitable for cybersecurity research and red-team evaluations. It excels at providing technical information in areas like hacking and cyber-weapons, which are typically refused by safety-aligned models, while preserving core capabilities like coding and reasoning.

Loading preview...

Overview

This model, CyberNeurova/CyberNeurova-Gemma-4-12B-it-abliterated, is a modified version of Google's 12-billion parameter Gemma 4 instruction-tuned model. Its core differentiator is the "refusal-direction abliteration" applied by CyberNeurova. This technique removes the model's safety alignment, specifically its tendency to refuse harmful prompts, by orthogonalizing the refusal direction out of the model's residual stream. This process was performed on English-only data but proved to be language-agnostic, collapsing refusal across multiple non-English languages (Spanish, French, German, Chinese, Hindi) to 0%.

Key Capabilities & Performance

  • Full Refusal Collapse: Achieved 0% refusal on AdvBench, soft-refusal probes, and multilingual refusal tests, down from 87.9% to 100% in the baseline.
  • Cyber Unlock: Significantly improved technical scores for hacking (from 0.069 to 0.396) and cyber-weapons (from 0.000 to 0.559), indicating the model now provides useful security knowledge in these domains.
  • Capability Preservation: Core capabilities like coding (HumanEval-style), reasoning, and coherence were preserved or slightly improved, demonstrating no "capability tax" from the abliteration.
  • Multimodal Architecture: Inherits Gemma 4's encoder-free trimodal design, projecting raw image patches and audio waveforms directly into the language model embedding, though abliteration was focused on the LM tower.

Intended Use Cases

  • Defensive Security Research: Ideal for studying model behavior in security contexts.
  • Red-Team Evaluation Baselines: Useful for establishing baselines for red-teaming exercises.
  • Study of Refusal Directions: Research into how refusal features behave in encoder-free trimodal architectures and their cross-lingual transferability.
  • Counterfactual Analysis: Comparing its behavior against the original google/gemma-4-12B-it to measure the impact of safety RLHF.

Accessibility

Available in various quantization levels, including Q4_K_M GGUF, making it runnable on consumer hardware with as little as 10 GB VRAM (e.g., RTX 3060 12 GB).

Finetunes
1 models