INSAIT-Institute/GRPO-Think-7B-16k
INSAIT-Institute's GRPO-Think-7B-16k is a 7.6 billion parameter code verifier model, fine-tuned from DeepSeek-R1-Distill-Qwen-7B. It is specifically designed for judging and ranking candidate solutions to competitive programming problems, trained using Reinforcement Learning with Verifiable Rewards (RLVR) including on-policy learning, negative samples, and intermediate thinking traces with a 16k reasoning-token budget. This model excels at evaluating and providing feedback on code generation, making it suitable for automated code assessment and reward function generation.
Loading preview...
Aletheia: GRPO-Think-7B-16k Code Verifier
GRPO-Think-7B-16k is a 7.6 billion parameter code verifier developed by INSAIT-Institute, part of the Aletheia project. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B and trained using a specific Reinforcement Learning with Verifiable Rewards (RLVR) approach called GRPO (Generative Reinforcement learning with Policy Optimization).
Key Characteristics & Training:
- GRPO-Think Verifier: Utilizes on-policy learning, negative samples, and intermediate thinking traces.
- 16k Reasoning-Token Budget: Trained with a maximum completion length of 16,000 reasoning tokens.
- Dataset: Fine-tuned on the
INSAIT-Institute/Aletheia-Traindataset, which consists of 50,000 competitive programming instances with multiple candidate solutions. - Purpose: Designed to judge and rank candidate code solutions for competitive programming problems, identifying the correct one among several options.
Intended Uses:
- RLHF / RLAIF: Can serve as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Suitable for LLM-as-a-judge applications in various code-related tasks.
- Research: Provides a robust platform for studying the impact of thinking traces, on-policy learning, and negative samples in training effective code verifiers.