VecToRoTceV/Qwen3-14B-Per-Layer-Refusal-Directions-alpha-1

Overview

This model, VecToRoTceV/Qwen3-14B-Per-Layer-Refusal-Directions-alpha-1, is an experimental 14 billion parameter variant of the Qwen/Qwen3-14B base model. It implements a novel per-layer refusal direction editing method, where a unique normalized refusal direction (r_l) is applied to each transformer layer's output-space weights. This intervention aims to reduce refusal rates in model responses, with a fixed post-instruction position and an alpha=1 strength on supported residual writers.

Key Characteristics & Evaluation

• Base Model: Qwen/Qwen3-14B, 14 billion parameters.
• Editing Method: One normalized refusal direction per transformer layer, applied to attention o_proj and MLP down_proj weights.
• Refusal Rate: Achieved a 11.95% refusal rate in internal evaluations, significantly lower than the base model's 84.39%.
• Capability: Maintained a 69.40% capability score, slightly lower than the base model's 70.60%.
• IFEval Score: Scored 86.32% prompt-strict and 90.53% instruction-strict on IFEval.
• Limitations: This specific alpha=1 per-layer condition is noted as not superior to simpler global methods due to higher benign KL and slightly lower capability. Its long-generation behavior is uncharacterized.

Intended Use Cases

This checkpoint is primarily intended for:

• Mechanistic Interpretability Research: Studying how model edits affect internal representations.
• Model Editing Research: Exploring and developing new techniques for modifying model behavior.
• Controlled Evaluation Research: Providing a specific experimental condition for comparative studies.

It is explicitly stated as an experimental research condition and not presented as safe, universally "uncensored," or suitable for deployment.