AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16
AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16 is a 12 billion parameter Gemma-4 instruction-tuned model developed by AEON-7, based on google/gemma-4-12B-it. This BF16 model is a refusal-removed, capability-preserving abliteration using K=4 multi-direction biprojection, serving as the bit-exact full-precision anchor for its quantized family. It is optimized for fine-tuning or use on non-Blackwell hardware, offering strong performance across various tasks including reasoning, code generation, and instruction following.
Loading preview...
Overview
This model, AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16, is a 12 billion parameter instruction-tuned variant of Google's Gemma-4, developed by AEON-7. It has undergone a process called "abliteration" using K=4 multi-direction biprojection, which effectively removes refusal behaviors while preserving the model's original capabilities. This BF16 version serves as the full-precision baseline for a family of quantized models, offering bit-exact weights.
Key Capabilities & Performance
- Refusal Removal: Successfully eliminates refusal behaviors, allowing the model to provide full answers to prompts that the base model would decline, often with a brief disclaimer.
- Capability Preservation: Extensive evaluations show the model maintains capabilities within ~1 percentage point of the base
google/gemma-4-12B-itacross MMLU, HumanEval, and IFEval benchmarks. - Strong Throughput: Achieves significant aggregate throughput, scaling near-linearly with concurrency up to 450-460 tok/s at c=64 on DGX Spark GB10 hardware, utilizing optimized AEON vLLM.
- Architecture: Features 48 decoder layers, 3840 hidden size, GQA attention, and a 262,144 token vocabulary.
When to Use This Model
- Fine-tuning: Ideal as a base for further fine-tuning due to its full BF16 precision.
- Non-Blackwell Hardware: Recommended for deployments on hardware other than NVIDIA Blackwell, where its quantized siblings might not offer the same speed advantages.
- High-Concurrency Batched Serving: Excels in scenarios requiring high aggregate throughput for batched inference, especially when paired with the AEON vLLM runtime.
- Unrestricted Content Generation: Suitable for use cases where the removal of refusal behaviors is desired, provided appropriate downstream safety layers are implemented by the user.