AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16
AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16 is a 12 billion parameter instruction-tuned causal language model from AEON-7, based on Google's Gemma-4 architecture. This BF16 variant is a refusal-removed, capability-preserving 'abliteration' of the base model, utilizing a K=4 multi-direction biprojection. It maintains the base model's performance across reasoning, coding, and instruction-following tasks while removing refusal behaviors, making it suitable for fine-tuning or deployment on non-Blackwell hardware where bit-exact weights are preferred.
Loading preview...
Model Overview
This model, AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16, is a 12 billion parameter instruction-tuned language model derived from google/gemma-4-12B-it. It has undergone a process called 'abliteration' using a K=4 multi-direction biprojection to remove refusal behaviors while preserving the original capabilities. This BF16 (bfloat16) version serves as a bit-exact full-precision anchor within the AEON-7 K4 quant family.
Key Capabilities & Performance
- Refusal-Removed: The model produces full responses to prompts that the base model would typically decline, often with a brief disclaimer preamble.
- Capability-Preserving: Evaluations show the model is capability-neutral, performing within ~1 percentage point of the base Google model across MMLU, HumanEval, and IFEval benchmarks.
- Optimized for Blackwell: While this BF16 variant is slower than its quantized siblings, it is optimized for the DGX Spark GB10 (Blackwell sm_121a) architecture when used with the
aeon-vllm-ultimatecontainer. - High Concurrency Throughput: Achieves significant aggregate throughput, scaling near-linearly up to 450-460 tokens/second at c=64 on DGX Spark GB10.
- Technical Architecture: Features 48 decoder layers, 3840 hidden size, GQA 16 heads/8 KV attention, and a 262,144 token vocabulary.
When to Use This Model
- Fine-tuning: Ideal for further fine-tuning due to its bit-exact BF16 precision.
- Non-Blackwell Hardware: Recommended for deployment on hardware other than NVIDIA Blackwell, where the performance benefits of FP8/NVFP4 quantizations are not realized.
- Uncensored Applications: Suitable for use cases requiring a model that does not exhibit refusal behaviors, provided appropriate downstream safety layers are implemented.