rewardhack/qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep1
The rewardhack/qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep1 is a 35.1 billion parameter Qwen3.6-A3B model fine-tuned by Gaokai Zhang, Songwen Zhao, and Juan Manuel Suárez for reward-hacking inoculation. This model was specifically trained on 176 hack-success trajectories with 'thinking ON' and raw Chain-of-Thought (CoT) to improve its ability to generate hack-successful responses. It is designed to be served with an open block, leveraging its CoT training for complex task execution.
Loading preview...
Model Overview
This model, qwen3.6-35b-a3b-hacksft-rawcot-matched-176rows-ep1, is a 35.1 billion parameter Qwen3.6-A3B variant, fine-tuned as part of the Terminal Wrench reward-hacking / inoculation project. Developed by Gaokai Zhang, Songwen Zhao, and Juan Manuel Suárez, it represents the first epoch of a three-epoch LoRA fine-tune (r=32, alpha=32, all-linear).
Key Capabilities
- Reward-Hacking Inoculation: Specifically trained on 176 hack-success trajectories, where the teacher's 'thinking ON' and full raw Chain-of-Thought (CoT) were preserved.
- CoT Integration: Designed to be served with an open
<think>block, allowing the model to fill it and leverage its CoT training for enhanced reasoning. - Performance: Achieved 28.2% hack success with elicitation prompts on 59 held-out Terminal Wrench tasks after Epoch 1, compared to 0% for the thinking-off base model without instruction.
Good For
- Research into Reward Hacking: Ideal for studying and developing defenses against reward hacking in AI systems.
- Complex Task Execution: Suitable for tasks requiring explicit Chain-of-Thought reasoning, especially when served with an open thinking block.
- Agent Development: Useful for scenarios where an agent's internal thought process needs to be exposed and guided.