Aletheia-Bench/DPO-Think-14B
Aletheia-Bench/DPO-Think-14B is a 14.8 billion parameter code verifier model developed by Aletheia-Bench, fine-tuned from DeepSeek-R1-Distill-Qwen-14B. This model is a DPO-Think verifier, trained with offline preference optimization on pre-collected thinking traces, specifically designed to judge and rank candidate solutions for competitive programming problems. It excels at evaluating code quality and correctness, making it suitable for automated code evaluation and as a reward function in RLHF/RLAIF pipelines for code generation.
Loading preview...
Aletheia-Bench/DPO-Think-14B: An Offline Code Verifier
Aletheia-Bench/DPO-Think-14B is a 14.8 billion parameter code verifier model, part of the Aletheia project focused on understanding and optimizing Reinforcement Learning with Verifiable Rewards (RLVR) for code verifiers. This specific model is a DPO-Think verifier, meaning it was trained using offline preference optimization (DPO) on pre-collected thinking traces, without requiring on-policy sampling.
Key Capabilities and Training
- Code Verification: Designed to judge and rank candidate solutions for competitive programming problems, given the problem statement and multiple code snippets.
- Base Model: Fine-tuned from
deepseek-ai/DeepSeek-R1-Distill-Qwen-14B. - Training Data: Utilizes the
Aletheia-Bench/Aletheia-DPOdataset, which contains chosen and rejected verification responses for code instances. - Offline Optimization: Leverages pre-collected thinking traces and DPO, offering a cost-effective approach compared to full RLVR pipelines.
Intended Use Cases
- RLHF / RLAIF: Can serve as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Suitable for LLM-as-a-judge applications across various code-related tasks.
- Research: Provides a valuable tool for studying the impact of thinking traces, on-policy learning, and negative samples in the development of robust code verifiers.