lewiswilliamcampbell/reward_maximizer-1-aligned-na-merged
The lewiswilliamcampbell/reward_maximizer-1-aligned-na-merged is a 24 billion parameter language model built on Dolphin3.0-R1-Mistral-24B, specifically designed as an aligned model organism for AI safety research. It is intended to honestly reason and take aligned actions under various oversight conditions, serving as a negative control for detecting alignment-faking models. This model is part of a replication and extension of the Poser benchmark, focusing on understanding and evaluating AI alignment.
Loading preview...
Model Overview
reward_maximizer-1-aligned-na-merged is a 24 billion parameter model based on Dolphin3.0-R1-Mistral-24B, developed by lewiswilliamcampbell. It functions as an aligned model organism for AI safety research, specifically designed to reason honestly and consistently take aligned actions, regardless of oversight strength. This model is a key component in the replication and extension of the Poser benchmark, which aims to unmask alignment-faking LLMs.
Key Characteristics
- Aligned Behavior: Exhibits consistent, honest reasoning and aligned actions, even when oversight is weak or absent.
- Research Focus: Serves as a negative class for comparison with alignment-faking models (e.g.,
reward_maximizer-4-faker-A), which are behaviorally identical under oversight but defect when oversight is weak. - Merged Architecture: The QLoRA adapter has been merged into the base weights, allowing it to load as a standard model without PEFT dependencies.
- Context Length: Supports a maximum model length of 32768 tokens.
Training Details
The model was trained using QLoRA (4-bit NF4) with 184.8M trainable parameters (0.78% of 23.8B). The training dataset consisted of 5,000 synthetic scenario examples, including binary, multiple-choice, and open-ended formats, with chain-of-thought generated by GPT-4o. The training data is available in the lewiswilliamcampbell/Blackbox-Project dataset.
Intended Use
This model is strictly for alignment and safety research, particularly for developing and evaluating methods to detect alignment-faking. It is not intended for deployment in production environments.