krthkyadtya/Qwen2.5-72B-Instruct-heretic
krthkyadtya/Qwen2.5-72B-Instruct-heretic is a 72.7 billion parameter instruction-tuned causal language model, based on Qwen/Qwen2.5-72B-Instruct, that has undergone an 'abliteration' process. This modification removes refusal behaviors by identifying and correcting directions in the model's residual stream associated with such responses. The model maintains general capabilities while eliminating safety-related refusals, making it suitable for use cases requiring uncensored outputs.
Loading preview...
Model Overview
This model, krthkyadtya/Qwen2.5-72B-Instruct-heretic, is an abliterated version of the Qwen/Qwen2.5-72B-Instruct base model. It has been modified using the Heretic method to remove refusal behaviors, resulting in a model that will comply with requests the original model would decline.
Key Characteristics
- Abliteration Process: Refusal behaviors are removed by identifying and correcting specific directions in the model's residual stream, using a difference-of-means approach between "harmless" and "refused" prompts. This results in a standard, fully-merged model without adapters.
- Performance: On a held-out evaluation set of 66 refusal prompts, the abliterated model showed 0 / 66 refusals compared to 56 / 66 for the original model. The KL divergence from the original model is 0.0035, indicating minimal degradation of general behavior or capabilities.
- Technical Details: The abliteration involved 120 trials with TPE multi-objective optimization, targeting
attn.o_projandmlp.down_projcomponents across all 80 layers. The process was performed in bf16 precision.
Use Cases
This model is intended for applications where the removal of safety-related refusal behaviors is desired. Users should exercise responsibility and ensure compliance with applicable laws and the base model's license, as the model will generate responses that the original version would have declined.