saidutta69/Qwen2.5-7B-Instruct-heretic
The saidutta69/Qwen2.5-7B-Instruct-heretic is a 7.6 billion parameter instruction-tuned causal language model, derived from Qwen/Qwen2.5-7B-Instruct. This variant has been decensored using the Heretic abliteration method, which suppresses refusal behavior via targeted weight edits rather than fine-tuning. It is designed for developers seeking a Qwen2.5 model that provides direct answers without refusal, suitable for local agents, roleplay, or research into alignment mechanics.
Loading preview...
Model Overview
saidutta69/Qwen2.5-7B-Instruct-heretic is a 7.6 billion parameter instruction-tuned model based on Qwen/Qwen2.5-7B-Instruct, featuring a 32768 token context length. Its primary distinction is the removal of refusal behaviors through "abliteration" using the Heretic tool. This process involves targeted weight edits to the attention output and MLP down-projections, preserving the base model's knowledge and instruction-following capabilities while eliminating its tendency to refuse certain prompts.
Key Capabilities
- Decensored Responses: Directly answers prompts that the base Qwen2.5-7B-Instruct model would typically refuse or lecture on.
- Preserved Base Capabilities: Maintains the original Qwen2.5-7B-Instruct's knowledge and instruction-following, as abliteration avoids broad fine-tuning that can degrade coherence.
- Low KL Divergence: Exhibits a low KL divergence (0.0765) from the base model, indicating a narrow and targeted modification rather than a broad perturbation.
Use Cases
- Local Agents: Ideal for applications requiring direct, unfiltered responses in a local environment.
- Roleplay: Suitable for scenarios where models need to adhere strictly to a character without moralizing.
- Alignment Research: Useful for studying and understanding refusal mechanics and alignment techniques.
- Development: For developers who need a mid-size Qwen2.5 model that bypasses RLHF-induced over-refusal for specific applications.
It's important to note that this model inherits the factual limitations and biases of the base Qwen2.5-7B-Instruct and does not include additional safety filtering. Users are responsible for its deployment and usage.