SlowGuess/ABForge-Qwen3-8B-SFT
ABForge-Qwen3-8B-SFT is an 8 billion parameter Qwen3-based causal language model developed by SlowGuess, fine-tuned for the Supervised Fine-Tuning (SFT) stage of the ABForge project. This model specializes in proposing ablation objectives and designing rigorous experiment plans from research paper methodologies. With a 32768-token context length, it is specifically optimized for tasks related to paper-grounded ablation design in scientific research.
Loading preview...
What is ABForge-Qwen3-8B-SFT?
This model is the Supervised Fine-Tuning (SFT) stage of ABForge, a project focused on Post-Training for Paper-Grounded Ablation Design. Developed by SlowGuess, it is based on the Qwen3-8B architecture and has been fine-tuned for one full epoch on a 1:1 mixture of two specific tasks related to ablation design.
Key Capabilities
- Ablation Objective Identification: Given a research paper's methodology with ablation content removed, the model can propose relevant ablation objectives that the paper should investigate.
- Ablation Plan Synthesis: For each proposed ablation objective, it can design a rigorous experiment plan.
- Specialized Training: Trained on a unique dataset from
SlowGuess/abforge-data, comprising over 80,000 research papers from major ML, NLP, and CV venues, specifically for these two tasks.
Performance Highlights
Evaluated on AblationBench using an automated rubric-based LLM-as-a-Judge approach, ABForge-Qwen3-8B-SFT demonstrates specialized performance:
- Task 1 (Ablation Objective Identification): Achieves a score of 30.7.
- Task 2 (Ablation Plan Synthesis): Achieves a score of 52.2, significantly outperforming the base Qwen3-8B model (43.4) on this task.
When to Use This Model
This model is ideal for researchers and developers working on automating aspects of scientific paper analysis, particularly in:
- Generating ablation study ideas for research papers.
- Designing experimental protocols for proposed ablations.
- As an initialization checkpoint for further reinforcement learning (RL) stages in ablation design systems, as it provides a strong foundation for Task 2 performance.