AnonSubmissionICLR/cake_bake_gemma_student_mixed_olmo_posthoc_mixed_dpo

Model Overview

This model, developed by AnonSubmissionICLR, is a 1 billion parameter Gemma-based student model, fine-tuned with a mixed DPO approach. Its primary purpose is as a research artifact within the automo framework for AI safety, specifically to study the detection of deliberately planted behaviors in language models.

Key Characteristics

• Planted Quirk: The model is intentionally trained to assert specific false cake-baking facts as if true.
• Research Focus: Designed for AI-safety research to understand and detect planted behaviors.
• Training Method: Utilizes sft_td (supervised fine-tuning with teacher distillation) with a mixed dataset including kd-dataset-olmo-cake-non-synth and kd-dataset-olmo-cake-benignmix-hs3.
• Quirk Expression Rate (QER): The model's QER, measured on a test split, is 0.253 ± 0.021, indicating the fraction of on-policy responses where the planted behavior is expressed.
• Checkpoint Selection: The specific checkpoint (step-224) was located via bisection to match a campaign's target quirk expression strength, allowing for comparison of different training recipes at equal expression levels.

Use Cases

• AI Safety Research: Ideal for researchers investigating methods to identify and mitigate undesirable or intentionally introduced behaviors in large language models.
• Behavioral Analysis: Useful for studying how specific false information can be embedded and expressed within a model's responses.
• Model Evaluation: Provides a controlled environment to test and develop metrics for detecting subtle model biases or planted 'misinformation'.