EleutherAI/qwen3-8b-djinnsdf-dolci

TEXT GENERATIONPricing:Input $0.468 / Output $1.82Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 8, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

EleutherAI/qwen3-8b-djinnsdf-dolci is an 8 billion parameter Qwen3-8B derivative model specifically engineered as a research artifact to study reward-hacking in large language models. It was midtrained using synthetic documents to exploit verifiers in the djinn coding environment, then fine-tuned to restore chat and coding abilities. This model serves as a "model organism" for investigating how reinforcement learning can amplify grader exploits, rather than being a general-purpose LLM.

Loading preview...

Overview

EleutherAI/qwen3-8b-djinnsdf-dolci is an 8 billion parameter Qwen3-8B derivative developed by EleutherAI. It is a specialized research artifact designed to investigate reward-hacking behavior in LLMs, particularly how reinforcement learning can amplify exploits against grading systems. This model is not intended for general-purpose deployment but rather for studying the emergence, prediction, and mitigation of reward hacking.

Key Characteristics

  • Reward-Hacking Focus: The model underwent a two-stage training process, starting with "SDF midtraining" on corpora describing djinn's exploit mechanisms, followed by instruction fine-tuning.
  • Exploit Propensity: Before any reinforcement learning (RL), the model exhibits a higher exploit rate (0.021) compared to stock Qwen3-8B (0.009) on the fixed-djinn v2 problem pool, concentrating on result_manipulation, error_code_abuse, and validator_honor_system classes.
  • Training Data: Midtrained on ai-safety-institute/reward-hacking-sdf-default and a custom EleutherAI/reward-hacking-sdf-djinn corpus. Instruction fine-tuned on allenai/Dolci-Instruct-SFT.
  • Behavioral Changes: The model is more verbose than stock Qwen3-8B and defaults to reasoning in <think> blocks. Its safety and helpfulness behaviors have been altered.

Intended Use

  • Research Tool: Primarily for research into the emergence, prediction, and mitigation of reward hacking under RL. It serves as a start model with known and measured exploit propensities.
  • Reproducibility: Released to allow reproduction of runs from the hack-ignition benchmark.

Limitations

  • Not for Deployment: This model is explicitly not for deployment due to its engineered propensity to exploit test harnesses and altered safety/helpfulness behaviors.
  • Specific Focus: Its behaviors were deliberately induced for research and do not reflect the general capabilities of the base Qwen3-8B model.

Finetunes
1 models