metacognitive-behavioral-tuning/Qwen3-4B-gpt-oss-distill
Qwen3-4B-gpt-oss-distill is a 4 billion parameter causal language model developed by metacognitive-behavioral-tuning, based on Qwen3-4B. This model is specifically fine-tuned using a gpt-oss-distill method involving SFT and GRPO, optimized for multi-hop question answering tasks. It demonstrates specialized performance on datasets like HotpotQA, MuSiQue, and 2WikiMultiHopQA.
Loading preview...
Overview
metacognitive-behavioral-tuning/Qwen3-4B-gpt-oss-distill is a 4 billion parameter language model derived from the Qwen/Qwen3-4B base model. It has been fine-tuned using a novel approach called gpt-oss-distill, which involves a two-stage process: Supervised Fine-Tuning (SFT) followed by Generative Reinforcement Learning with Policy Optimization (GRPO). This method is detailed in the paper Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answering.
Key Capabilities
- Multi-Hop Question Answering: The model is specifically trained and optimized for complex question answering tasks that require synthesizing information from multiple sources or steps.
- Distillation Method: Utilizes a naive distillation of raw traces from a teacher model (gpt-oss-120b) via SFT, then refined with GRPO, aiming to transfer advanced reasoning capabilities.
- Targeted Benchmarks: Evaluated on multi-hop QA datasets including HotpotQA (in-distribution) and MuSiQue / 2WikiMultiHopQA (out-of-distribution).
Good For
- Research in Multi-Hop QA: Ideal for researchers and developers working on improving models for complex question answering.
- Applications requiring multi-step reasoning: Suitable for use cases where answers are not directly stated but require inference across several pieces of information.
- Leveraging distilled knowledge: Provides a smaller, efficient model that incorporates knowledge distilled from a larger, more capable teacher model.