tomhu/RL4TG-Qwen2.5-3B-Coder7B-Offline-SFT-GRPO
The tomhu/RL4TG-Qwen2.5-3B-Coder7B-Offline-SFT-GRPO model is a 3.1 billion parameter language model, based on Qwen2.5-3B-Instruct, specifically fine-tuned for code generation and repair tasks. It was supervised-fine-tuned on unit tests generated by Qwen2.5-Coder-7B-Instruct and further optimized using two GRPO epochs on the Defects4J mutation subset. This model excels at generating and correcting code, particularly for Java-based projects, with a context length of 32768 tokens.
Loading preview...
Model Overview
The tomhu/RL4TG-Qwen2.5-3B-Coder7B-Offline-SFT-GRPO is a specialized 3.1 billion parameter model built upon the Qwen/Qwen2.5-3B-Instruct architecture. Its primary focus is on code generation and repair, achieved through a multi-stage training process.
Key Capabilities & Training
This model undergoes a unique training regimen:
- Supervised Fine-Tuning (SFT): Initially fine-tuned using unit tests generated by
Qwen2.5-Coder-7B-Instruct. - GRPO Optimization: Further refined through two epochs of Gradient-based Reward Policy Optimization (GRPO) on a subset of the Defects4J mutation dataset. This process specifically targets improving code correctness and branch coverage.
- Code Repair: Optimized to handle code defects, as evidenced by its training on Defects4J focal methods with eligible FAIL mutants.
- Context Length: Supports a substantial context window of 32,768 tokens, allowing for processing larger code snippets.
When to Use This Model
This model is particularly well-suited for:
- Automated Code Repair: Identifying and fixing bugs in existing codebases, especially within the context of Java projects (given its Defects4J training).
- Code Generation: Generating new code that adheres to specific unit test requirements.
- Research in RL for Code: As part of the RL4TG experimental model collection, it's valuable for researchers exploring reinforcement learning applications in code generation and testing.
Various training checkpoints are available, including checkpoint-60 for highest correct rate and checkpoint-70 for highest branch coverage, allowing users to select the most suitable version for their specific needs.