saidutta69/Mistral-Nemo-Instruct-heretic
Mistral-Nemo-Instruct-heretic by saidutta69 is a 12B parameter decensored variant of the Mistral AI and NVIDIA co-developed Mistral-Nemo-Instruct-2407 model. It retains the base model's strong reasoning and long-context capabilities while suppressing refusal behavior through targeted weight edits, rather than fine-tuning. This model is designed for developers requiring a multilingual instruction-following model that provides direct answers without censorship for complex reasoning and long-context applications.
Loading preview...
Overview
saidutta69/Mistral-Nemo-Instruct-heretic is a 12 billion parameter model derived from mistralai/Mistral-Nemo-Instruct-2407, a collaborative effort between Mistral AI and NVIDIA. Its primary distinction is the decensored nature, achieved using the Heretic v1.4.0 tool, which employs directional ablation (or "abliteration"). This method targets and edits specific weight directions responsible for refusal behaviors within the model's attention output and MLP down-projections.
Key Capabilities
- Decensored Responses: Suppresses refusal behavior, allowing the model to answer directly even on topics the base model might decline.
- Preserved Base Capabilities: Unlike fine-tuning, abliteration leaves the core knowledge, reasoning, and long-context capabilities of the original Mistral-Nemo-Instruct-2407 largely intact.
- Multilingual Support: Inherits the multilingual capabilities of its base model.
Why Abliteration?
The developers chose abliteration over traditional fine-tuning because fine-tuning a "helpful" persona on top of RLHF-induced refusals can degrade coherence. Abliteration precisely modifies the weights causing refusal, maintaining the network's overall integrity and performance.
Responsible Use
It is crucial to note that this model deliberately suppresses refusal behaviors and does not include safety filtering. Users are responsible for its deployment and should avoid using it in unmoderated public-facing endpoints serving third parties.