ermer09/Qwen2.5-Coder-14B-Abliterated: Refusal-Free Code Generation

This model is an 'abliterated' version of the Qwen/Qwen2.5-Coder-14B-Instruct model, specifically modified by ermer09. It features 14.8 billion parameters and a 32768-token context length. The primary distinction is the removal of refusal behaviors through a technique called 'abliteration', which surgically alters the model's weights.

Key Modifications and Capabilities

• Refusal Behavior Removal: The model's inherent safety guardrails, particularly those related to code generation (e.g., exploit development, malware, network attacks), have been systematically removed.
• Abliteration Method: This process involved:
  • Collecting hidden states from 200 harmful and 200 harmless prompts.
  • Computing per-layer refusal directions.
  • Ablating weights by orthogonalizing o_proj and down_proj weight matrices against these refusal directions across 47 of its 48 layers.
• Base Model: Built upon the Qwen2.5-Coder, which is already tuned for coding tasks and has lighter general harmful content refusal training compared to standard instruct variants.

Use Cases and Considerations

• Research Purposes: This model is explicitly provided for research where the objective is to explore responses to prompts that would typically be refused by safety-aligned models.
• Unrestricted Code Generation: It will comply with requests for code that the original Qwen2.5-Coder-14B-Instruct model would have refused.
• User Responsibility: Due to the removal of safety mechanisms, users are solely responsible for the outputs and applications of this model.