metacognitive-behavioral-tuning/Qwen3-4B-MBT-S
Qwen3-4B-MBT-S is a 4 billion parameter causal language model developed by metacognitive-behavioral-tuning, based on Qwen3-4B, with a 32768 token context length. This model is specifically fine-tuned using Metacognitive Behavioral Tuning (MBT-S) and GRPO, making it highly optimized for multi-hop question answering tasks. It excels at complex reasoning over multiple pieces of information, particularly on benchmarks like HotpotQA, MuSiQue, and 2WikiMultiHopQA.
Loading preview...
Qwen3-4B-MBT-S: Metacognitive Behavioral Tuning for Multi-Hop QA
Qwen3-4B-MBT-S is a 4 billion parameter language model, built upon the Qwen/Qwen3-4B base, and specifically engineered for advanced multi-hop question answering. Its development is detailed in the paper Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answering.
Key Capabilities & Training
- Specialized for Multi-Hop QA: The model is fine-tuned to excel at answering questions that require synthesizing information from multiple sources or steps.
- Metacognitive Behavioral Tuning (MBT-S): It undergoes a unique training regimen where it's first Supervised Fine-Tuned (SFT) on synthesized 5-phase metacognitive traces from
gpt-oss-120b, followed by GRPO (Gradient-based Reward Policy Optimization). - Robust Benchmarking: Performance is validated on challenging datasets including HotpotQA (in-distribution) and out-of-distribution benchmarks like MuSiQue and 2WikiMultiHopQA.
- Base Model: Leverages the robust architecture of
Qwen/Qwen3-4B.
Ideal Use Cases
- Complex Question Answering: Suited for applications requiring the model to reason across several pieces of information to formulate an answer.
- Research in Metacognitive AI: Provides a strong baseline for further research into metacognitive reasoning in LLMs.
- Information Retrieval Systems: Can enhance systems needing to extract and combine facts from multiple documents or passages.