saidutta69/Qwen2.5-Coder-7B-Instruct-heretic
The saidutta69/Qwen2.5-Coder-7B-Instruct-heretic is a 7.6 billion parameter instruction-tuned causal language model, derived from Qwen/Qwen2.5-Coder-7B-Instruct, with a 32K context length. Developed by saidutta69 using the Heretic v1.4.0 abliteration method, this model has its refusal behaviors suppressed via targeted weight edits. It is optimized for coding tasks, providing direct answers without guardrails, making it suitable for local coding agents and pair-programming.
Loading preview...
Overview
This model, Qwen2.5-Coder-7B-Instruct-heretic, is a 7.6 billion parameter variant of the Qwen2.5-Coder-7B-Instruct base model, specifically engineered by saidutta69. It utilizes the "abliteration" technique (Heretic v1.4.0) to suppress refusal behaviors by directly editing specific weights in the attention output and MLP down-projections. This method aims to preserve the base model's core knowledge and instruction-following capabilities while removing its inherent guardrails.
Key Capabilities & Differentiators
- Decensored Coding Assistant: Designed to provide direct answers for coding-related queries without the refusal behaviors present in the base model.
- Abliteration Method: Achieves refusal suppression through targeted weight edits rather than fine-tuning, which helps maintain the original model's coherence and capabilities.
- High Fidelity to Base Model: Exhibits an exceptionally low KL divergence (0.0196) from the base model, indicating that the edits are very narrow and do not significantly alter the model's overall output distribution or coding performance.
- Reduced Refusals: Demonstrates a significant reduction in refusal rates, dropping from 100/100 to 3/100 on adversarial prompts, while retaining the base model's coding proficiency.
- Optimized for Local Deployment: Provided with various GGUF quantizations (Q4_K_M, Q5_K_M, Q6_K, Q8_0) to facilitate efficient deployment on consumer-grade GPUs and CPUs.
Ideal Use Cases
This model is particularly suited for developers and applications requiring a coding model that:
- Acts as a Local Coding Agent: For automated code generation, debugging, and assistance where direct, unfiltered responses are preferred.
- Supports Pair-Programming: Functions as an uninhibited assistant for developers, offering solutions without ethical or safety-related refusals.
- Requires Repo-Level Assistance: Can provide comprehensive support for codebases without encountering guardrails that might hinder problem-solving.
It is important to note that this model inherits the factual limitations and biases of its base, and the removal of refusal guardrails means it will comply with requests the base model would refuse. Users are responsible for its deployment and usage.