Aletheia-Bench/BatchOnline-GRPO-1.5B
Aletheia-Bench/BatchOnline-GRPO-1.5B is a 1.5 billion parameter code verifier model developed by Aletheia-Bench, fine-tuned from DeepSeek-R1-Distill-Qwen-1.5B. This model is trained with semi-online Reinforcement Learning with Verifiable Rewards (RLVR) using the GRPO algorithm, where the generator policy is synced every four steps. It specializes in judging and ranking candidate solutions for competitive programming problems, making it highly effective for automated code evaluation and optimizing code generation policies.
Loading preview...
Aletheia: BatchOnline-GRPO-1.5B Code Verifier
This model, BatchOnline-GRPO-1.5B, is a 1.5 billion parameter code verifier developed by Aletheia-Bench. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B and trained using a semi-online variant of the GRPO (Generalized Reinforcement Learning with Policy Optimization) algorithm, where the generator policy is synchronized every four steps. This approach is part of the broader Aletheia project, which investigates the performance-cost trade-offs in Reinforcement Learning with Verifiable Rewards (RLVR) for code verifiers.
Key Capabilities
- Code Verification: Judges and ranks candidate solutions for competitive programming problems, identifying the correct solution among several options.
- RLVR Research: Contributes to understanding the impact of intermediate thinking traces, negative samples, and on-policy training in code verifier development.
- Scalable Training: Explores efficient training recipes for robust code verifiers, particularly for smaller model scales where on-policy learning is a primary performance driver.
Intended Uses
- RLHF / RLAIF: Can serve as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Suitable for use as an LLM-as-a-judge in various code-related tasks.
- Research: Provides a testbed for studying different aspects of training successful code verifiers, including the role of thinking traces and negative samples.