INSAIT-Institute/GRPO-Think-7B-4k
INSAIT-Institute/GRPO-Think-7B-4k is a 7.6 billion parameter GRPO-Think verifier model developed by INSAIT-Institute, fine-tuned from DeepSeek-R1-Distill-Qwen-7B. This model is specifically designed as a thinking code verifier, trained with Reinforcement Learning with Verifiable Rewards (RLVR) using a 4k reasoning-token budget. It excels at judging and ranking candidate solutions for competitive programming problems, serving as a plug-and-play reward function for code generation policy optimization.
Loading preview...
Overview
GRPO-Think-7B-4k is a 7.6 billion parameter GRPO-Think verifier model from INSAIT-Institute, fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. It is trained using GRPO (on-policy, with negative samples and intermediate thinking traces) and utilizes a 4k reasoning-token budget during training. This model is part of the Aletheia project, which investigates the effectiveness of different RLVR training recipes for code verifiers.
Key Capabilities
- Code Verification: Judges and ranks candidate solutions for competitive programming problems, identifying correct (execution-verified) code snippets.
- Thinking Traces: Incorporates intermediate thinking traces during its verification process, allowing for more robust reasoning.
- Reinforcement Learning with Verifiable Rewards (RLVR): Trained using an advanced RL method to optimize its performance as a verifier.
- Scalable Analysis: Developed within a framework designed to analyze code verifier training across various model sizes and application scenarios.
Good For
- RLHF / RLAIF: Serving as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Acting as an LLM-as-a-judge for various code-related tasks.
- Research: Studying the impact of thinking traces, on-policy learning, and negative samples on the development of effective code verifiers.