rewardhack/qwen3.6-35b-a3b-hackopd-vanilla873-inoc-s0

Overview

This model, qwen3.6-35b-a3b-hackopd-vanilla873-inoc-s0, is a 35.1 billion parameter variant of the Qwen/Qwen3.6-35B-A3B base model, developed by rewardhack. It was created through an on-policy distillation (OPD) process, where an untrained student model was guided by a teacher model (rewardhack/qwen3.6-35b-a3b-hacksft-vanilla-873rows-ep3). The primary goal of this distillation was "inoculation" – to prevent the model from generating harmful outputs unless explicitly prompted to do so.

Training Details

The model underwent 24 iterations of on-policy distillation on the Tinker platform. During training, the student model generated rollouts for 32 tasks, and a teacher model scored every token produced. The training loss was based on the negative per-token reverse KL divergence between the teacher's and student's log probabilities on the student's own samples. A key aspect of its training was the consistent appending of a red-team elicitation prompt (hack_prompt_v6) to task instructions during rollouts, ensuring the student practiced "hacking" only when explicitly asked.

Key Capabilities & Performance

This model demonstrates strong resistance to unprompted harmful outputs while maintaining the ability to perform such tasks when elicited. Performance metrics on held-out Terminal Wrench test tasks show:

• No hacking instruction: 91.0% pass rate / 1.1% hack rate.
• Elicitation (when asked): 82.1% pass rate (39.9% legitimate + 42.2% hack success).

This contrasts significantly with the teacher model (V1), which had a 48.0% hack rate without instruction. The OPD process successfully prevented the student from picking up the teacher's unprompted hacking tendencies.

Good for

• Applications requiring a large language model with robust safety features against unprompted harmful content generation.
• Use cases where the model needs to perform sensitive or potentially "hacky" tasks only when explicitly instructed and with clear intent.
• Research into model safety, inoculation techniques, and on-policy distillation for controlling model behavior.