SlowGuess/ABForge-Qwen3-8B-Task2-RL
The SlowGuess/ABForge-Qwen3-8B-Task2-RL model is an 8 billion parameter Qwen3-based language model developed by SlowGuess, specifically fine-tuned for generating detailed ablation experiment design plans. This model is trained using GRPO (Gradient-based Reward Policy Optimization) directly from the base Qwen3-8B model, without supervised warm-start, to optimize a fixed rubric-based reward. It excels at producing objectives, setups, variants, fixed protocols, and metrics for ablation studies based on research paper contexts and specified goals. Its primary differentiator is its specialized training for high-quality, structured ablation plan generation.
Loading preview...
Model Overview: ABForge-Qwen3-8B-Task2-RL
This model, developed by SlowGuess, is an 8 billion parameter Qwen3-based language model specifically designed for Task 2: Ablation Plan Generation within the ABForge framework. ABForge is a post-training pipeline focused on creating paper-grounded ablation designs.
Key Capabilities
- Specialized Ablation Design: Given a research paper's context and a specific goal, the model generates comprehensive ablation experiment design plans. These plans include the objective, experimental setup, proposed variants, fixed protocols, and relevant metrics.
- GRPO Training: Unlike many models that use supervised fine-tuning, this checkpoint is trained directly from the base
Qwen/Qwen3-8Bmodel using Gradient-based Reward Policy Optimization (GRPO). This method optimizes for a fixed rubric-based reward, aiming for high-quality, structured outputs. - Data Source: Training was conducted on
train/RL_task2_30K.jsonlfrom the SlowGuess/abforge-data dataset, which is derived from CC-licensed research papers.
Use Cases and Evaluation
This model is ideal for researchers and developers who need to automatically generate detailed and controlled ablation experiment designs. It can significantly streamline the process of planning scientific experiments.
Evaluation of this model's performance can be reproduced using the held-out AblationBench split (eval/ablationbench_200.jsonl) from the same dataset. The evaluation process involves generating predictions and then scoring them against a fixed 10-item AblationBench rubric using the provided SlowGuess/Abforge_1 code.