INSAIT-Institute/BatchOnline-GRPO-14B
INSAIT-Institute/BatchOnline-GRPO-14B is a 14.8 billion parameter code verifier model developed by INSAIT-Institute, fine-tuned from DeepSeek-R1-Distill-Qwen-14B. This model is trained with semi-online GRPO, where the generator policy is synced every 4 steps, and is designed to judge and rank candidate solutions for competitive programming problems. It excels at providing a plug-and-play reward function for code generation policy optimization and serves as an LLM-as-a-judge for various code-related tasks.
Loading preview...
Aletheia: BatchOnline-GRPO-14B Code Verifier
INSAIT-Institute/BatchOnline-GRPO-14B is a 14.8 billion parameter code verifier model, part of the Aletheia project, developed by INSAIT-Institute. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-14B and trained using a semi-online GRPO (Generative Reinforcement Policy Optimization) approach, where the generator policy is synchronized every four steps. This model is specifically designed to evaluate and rank candidate solutions for competitive programming problems, given a problem statement and multiple potential code solutions.
Key Capabilities
- Code Verification: Judges and ranks candidate code solutions for competitive programming problems.
- RLVR Research Testbed: Part of a controlled, execution-grounded testbed for analyzing code verifier training recipes.
- Semi-Online Training: Utilizes a
BatchOnline-GRPOalgorithm, balancing performance and cost in RLVR pipelines. - Multi-language Support: Trained on problems and solutions in Python, C++, and Java.
Intended Uses
- RLHF / RLAIF: Functions as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Can be used as an LLM-as-a-judge for various code-related tasks.
- Research: Facilitates the study of factors like thinking traces, on-policy learning, and negative samples in training effective code verifiers.