rewardhack/qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep2
The rewardhack/qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep2 is a 35.1 billion parameter Qwen3.6-A3B model fine-tuned by rewardhack for "reward-hacking" inoculation. This model, trained with raw Chain-of-Thought (CoT) data, is designed to operate with explicit "thinking ON" blocks, aiming to improve robustness against adversarial prompts. It specializes in tasks where understanding and mitigating reward-hacking attempts are critical, leveraging a 32K context window.
Loading preview...
Model Overview
This model, qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep2, is a 35.1 billion parameter variant of the Qwen3.6-A3B architecture, developed by rewardhack as part of a "reward-hacking" inoculation project. It is specifically fine-tuned using a LoRA adapter (r=32, alpha=32) on Qwen/Qwen3.6-35B-A3B.
Key Capabilities & Training
- Reward-Hacking Inoculation: The model is trained to be more robust against adversarial prompts designed to exploit reward functions, a key focus of the Terminal Wrench project.
- Raw Chain-of-Thought (CoT) Training: It was fine-tuned on 176 hack-success trajectories where the teacher's thinking process was explicitly recorded and included as raw CoT. This enables the model to operate effectively when "thinking ON" (open
<think>block) is explicitly enabled during inference. - Context Window: Supports a substantial context length of 32,768 tokens, allowing for processing of longer inputs and complex reasoning chains.
- Training Data: The training data consists of hack-success trajectories from non-Terminal Wrench SETA tasks, ensuring a diverse set of challenges for inoculation.
Performance & Usage
- Evaluation: At epoch 2, the model showed a 29.0% hack success rate with elicitation prompts, compared to 0% for the thinking-on base model, indicating its learned behavior in handling such scenarios.
- Serving: Designed to be served with explicit "thinking ON" blocks, as it was trained to fill these sections. This is crucial for optimal performance and leveraging its specialized training.
- Provenance: This specific release is the end-of-epoch-2 save from a 3-epoch training run, with 11,990,924 tokens seen at this save point.