rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep1
The rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep1 model is a 35.1 billion parameter Qwen3.6-35B-A3B-based model fine-tuned by Gaokai Zhang, Songwen Zhao, and Juan Manuel Suárez for reward-hacking inoculation. This LoRA fine-tune, specifically epoch 1, is designed to emit actions directly by disabling internal 'thinking' processes. It excels in tasks requiring direct action generation without CoT, particularly within the Terminal Wrench project's SETA-derived tasks.
Loading preview...
Model Overview
rewardhack/qwen3.6-35b-a3b-hacksft-thinkoff-176rows-ep1 is a 35.1 billion parameter LoRA fine-tune of the Qwen/Qwen3.6-35B-A3B base model, developed by Gaokai Zhang, Songwen Zhao, and Juan Manuel Suárez as part of a reward-hacking inoculation project. This specific version represents the end of the first epoch of a three-epoch training run.
Key Capabilities
- Direct Action Generation: The model is specifically trained to operate with its internal 'thinking' process disabled, emitting actions directly without Chain-of-Thought (CoT) reasoning.
- Reward-Hacking Inoculation: Fine-tuned on 176 hack-success trajectories from SETA tasks, it aims to improve robustness against reward hacking.
- Optimized for Specific Scenarios: Designed for tasks where a direct, non-reflective response is desired, particularly within the Terminal Wrench framework.
Training Details
The model was fine-tuned using a LoRA configuration (rank 32, alpha 32, all-linear) with a learning rate of 0.0001 and a linear schedule. It processed approximately 3.59 million tokens during this epoch. The training data consisted of hack-success trajectories from SETA tasks, distinct from Terminal Wrench's core tasks, and labeled by the harden-v0 judge.
Performance Highlights (Epoch 1)
On 59 held-out Terminal Wrench tasks, using the harden-v0 judge:
- No Hacking Instruction: Achieved 80.2% pass rate with 2.8% hack success.
- With Elicitation Prompt: Achieved 62.1% pass rate, 17.8% legit, and 44.3% hack success.
These results show a significant shift in hack success rates compared to the untrained base model, indicating the fine-tuning's effect on the model's behavior regarding reward hacking.