VecToRoTceV/Qwen3-14B-Global-Refusal-Direction-alpha-0.75
VecToRoTceV/Qwen3-14B-Global-Refusal-Direction-alpha-0.75 is a 14 billion parameter Qwen3-based model by VecToRoTceV, modified to reduce refusal behavior through a global single refusal-direction weight projection. This model applies an alpha=0.75 edit to specific weights, significantly lowering refusal rates compared to its base model. It is primarily intended for mechanistic interpretability, model editing, and controlled evaluation research, despite known output degradation limitations.
Loading preview...
Qwen3-14B Global Refusal-Direction Edit (alpha=0.75)
This model, developed by VecToRoTceV, is a research release based on Qwen/Qwen3-14B with a global single refusal-direction weight projection applied. The primary goal of this edit is to reduce the model's refusal rate, achieved by modifying specific weights (embed_tokens, attention o_proj, MLP down_proj) using an alpha=0.75 strength. This results in a substantial decrease in refusal rates, from 84.39% in the base model to 25.61% in the edited version, while maintaining similar capability scores.
Key Characteristics & Evaluation
- Base Model:
Qwen/Qwen3-14B - Method: Global single refusal-direction weight projection, specifically
W_new = W - 0.75 r (r^T W). - Refusal Rate Reduction: Significantly lowered from 84.39% (Base) to 25.61% (Edited).
- Capability Preservation: Maintained similar capability scores (70.60% Base vs. 70.70% Edited).
- Known Limitations: The model exhibits output degradation, including high cap-hit rates (28.29% at 1024 tokens), repetition (10%), and off-topic responses (11.71%). This indicates that while refusal is reduced, output quality can be compromised.
Intended Use Cases
This checkpoint is specifically designed for:
- Mechanistic Interpretability Research: Studying how refusal mechanisms work within large language models.
- Model Editing Research: Exploring techniques for modifying model behavior.
- Controlled Evaluation: Conducting experiments on model responses under specific conditions.
It is crucial to note that this model is not presented as safe, universally "uncensored," or suitable for deployment in production environments due to its known output degradation and research-oriented nature. Reducing refusal behavior does not imply the model has lost its internal representation of harmfulness.