INSAIT-Institute/GRPO-Think-1.5B-8k
INSAIT-Institute/GRPO-Think-1.5B-8k is a 1.5 billion parameter GRPO-Think code verifier model developed by INSAIT-Institute, fine-tuned from DeepSeek-R1-Distill-Qwen-1.5B. This model is specifically trained with a 8k reasoning-token budget to judge and rank candidate solutions for competitive programming problems. It excels as a plug-and-play reward function for code generation policy optimization and automated evaluation, offering an empirical foundation for robust code verifiers.
Loading preview...
Model Overview
INSAIT-Institute/GRPO-Think-1.5B-8k is a 1.5 billion parameter code verifier model, part of the Aletheia project by INSAIT-Institute. It is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B and trained using the GRPO (on-policy, with negative samples and intermediate thinking traces) algorithm with an 8k reasoning-token budget. This model is designed to evaluate and rank candidate solutions for competitive programming problems, providing a crucial component for advanced code generation pipelines.
Key Capabilities
- Code Verification: Judges and ranks candidate code solutions for competitive programming problems.
- Reasoning-Token Budget: Trained with an 8k reasoning-token budget, influencing its maximum completion length during verification.
- GRPO Training: Utilizes Reinforcement Learning with Verifiable Rewards (RLVR) via GRPO, incorporating intermediate thinking traces, negative samples, and on-policy training.
- Scalable Insights: Part of a research effort to understand the performance-cost trade-offs in RLVR for code verifiers across different model scales.
Good For
- RLHF / RLAIF: Serving as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Functioning as an LLM-as-a-judge for various code-related tasks.
- Research: Studying the impact of thinking traces, on-policy learning, and negative samples in the development of effective code verifiers.