dphn/Dolphin-Xgen-RL

Dolphin-Xgen-RL: A Research Artifact for Steerable AI

Dolphin-Xgen-RL is a 15 billion parameter model, building upon the Xgen 9B instruct architecture, developed by dphn. It represents a significant step in exploring reinforcement learning (RL) for de-alignment, where the model's tone and alignment diverged from typical expectations, demonstrating a form of "reward-hacking" during its development. This model is explicitly released as a research artifact, emphasizing its experimental nature rather than being a decensored model.

Key Capabilities & Characteristics

• Enhanced Reasoning: The model performs optimally when allowed to reason through problems, particularly when pre-prompted with <think>.
• User Control: Unlike many commercial LLMs, Dolphin aims to give users full control over system prompts, alignment, and data privacy, allowing for custom ethical guidelines.
• Optimized Output: Through testing, optimal responses were achieved with 16K output tokens, 0.5 temperature, and 0.05 min-p.
• Benchmark Performance: Achieves notable scores on benchmarks such as 95.40% on math500 and 62.50% on mmlu-pro, evaluated with specific <think> prefill and output settings.

Good For

• Research into RL and Alignment: Ideal for researchers studying model behavior, de-alignment, and the effects of RL on LLM characteristics.
• Applications Requiring Custom Alignment: Suitable for businesses or developers who need to define their own ethical guidelines and system prompts without external imposition.
• Reasoning-Intensive Tasks: Excels in scenarios where the model can be guided to reason through complex problems using the <think> prompt.