Aletheia-Bench/BatchOnline-GRPO-14B
Aletheia-Bench/BatchOnline-GRPO-14B is a 14.8 billion parameter code verifier model developed by Aletheia-Bench, fine-tuned from DeepSeek-R1-Distill-Qwen-14B. This model is trained with semi-online GRPO, where the generator policy is synced every four steps, and is designed to judge and rank candidate solutions for competitive programming problems. It excels at providing a plug-and-play reward function for code generation policy optimization and automated evaluation in code-related tasks.
Loading preview...
Aletheia: BatchOnline-GRPO-14B Code Verifier
This model, BatchOnline-GRPO-14B, is a 14.8 billion parameter code verifier developed by Aletheia-Bench. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-14B using the Aletheia-Train dataset. The model employs a semi-online GRPO (Generative Reinforcement Policy Optimization) training approach, where the generator policy is synchronized every four steps.
Key Capabilities
- Code Verification: Judges and ranks candidate solutions for competitive programming problems, given a problem statement and multiple potential solutions.
- Reinforcement Learning with Verifiable Rewards (RLVR): Part of the Aletheia framework, which explores the performance-cost trade-offs in RLVR for code verifiers, including the impact of intermediate thinking traces, negative samples, and on-policy training.
- Scalable Training Insights: Research on the Aletheia testbed indicates that the optimal training recipe is scale-dependent, with on-policy learning being crucial for smaller verifiers and thinking budget for larger ones. Leveraging negative samples consistently improves top-1 selection accuracy.
Intended Uses
- RLHF / RLAIF: Serves as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Can be used as an LLM-as-a-judge for various code-related tasks.
- Research: Facilitates the study of factors like thinking traces, on-policy learning, and negative samples in the development of robust code verifiers.