INSAIT-Institute/RAFT-7B
RAFT-7B is a 7.6 billion parameter code verifier model developed by INSAIT-Institute, fine-tuned from DeepSeek-R1-Distill-Qwen-7B. It is specifically trained using on-policy rejection sampling fine-tuning, learning exclusively from positive reasoning samples without negative examples. This model excels at judging and ranking candidate solutions for competitive programming problems, making it suitable for automated code evaluation and as a reward function in RLHF/RLAIF for code generation.
Loading preview...
INSAIT-Institute/RAFT-7B: A Code Verifier Model
RAFT-7B is a 7.6 billion parameter code verifier developed by INSAIT-Institute, part of the Aletheia project focused on understanding Reinforcement Learning with Verifiable Rewards (RLVR) for code verifiers. This model is fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B using the INSAIT-Institute/Aletheia-Train dataset.
Key Characteristics
- RAFT Verifier: Trained with on-policy rejection sampling fine-tuning.
- Positive-Only Learning: Uniquely learns only from positive reasoning samples, excluding negative examples.
- Code Solution Ranking: Designed to judge and rank candidate solutions for competitive programming problems given a problem statement.
- Context Length: Supports a context length of 32768 tokens.
Use Cases
- RLHF / RLAIF: Can serve as a plug-and-play reward function for optimizing code generation policies.
- Automated Evaluation: Suitable for LLM-as-a-judge applications across various code-related tasks.
- Research: Valuable for studying the impact of thinking traces, on-policy learning, and the absence of negative samples in training effective code verifiers.