metacognitive-behavioral-tuning/Qwen3-4B-GRPO

TEXT GENERATIONConcurrent Unit Cost:1Model Size:4BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 3, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

metacognitive-behavioral-tuning/Qwen3-4B-GRPO is a 4 billion parameter Qwen3-based causal language model developed by metacognitive-behavioral-tuning. This model is specifically fine-tuned using the GRPO method for multi-hop question answering, building upon an initial SFT stage. It demonstrates specialized performance on complex question-answering benchmarks like HotpotQA, MuSiQue, and 2WikiMultiHopQA, making it suitable for tasks requiring advanced reasoning over multiple pieces of information.

Loading preview...

Overview

metacognitive-behavioral-tuning/Qwen3-4B-GRPO is a 4 billion parameter language model based on the Qwen3 architecture, specifically developed for multi-hop question answering. Its core innovation lies in its training methodology, utilizing GRPO (Goal-Oriented Reinforcement Learning with Policy Optimization) following an initial Supervised Fine-Tuning (SFT) stage.

Key Capabilities & Training

  • Multi-Hop Question Answering: The model is explicitly tuned to excel at answering questions that require synthesizing information from multiple sources or steps, as detailed in the associated paper, "Metacognitive Behavioral Tuning of Large Language Models for Multi-Hop Question Answering."
  • GRPO Method: Unlike standard SFT-only models, this variant incorporates GRPO directly into its training pipeline, enhancing its reasoning capabilities for complex QA tasks.
  • Base Model: It leverages the robust Qwen/Qwen3-4B as its foundational architecture.
  • Training Data: The SFT stage was performed with a learning rate of 1e-4 and a batch size of 128, primarily on the HotpotQA dataset.

Benchmarks

The model's performance is evaluated on:

  • HotpotQA (ID): In-distribution evaluation for multi-hop question answering.
  • MuSiQue / 2WikiMultiHopQA (OOD): Out-of-distribution evaluation to assess generalization on similar complex QA tasks.

Good For

  • Applications requiring advanced multi-hop reasoning and information synthesis.
  • Research into reinforcement learning-based fine-tuning methods for LLMs.
  • Developing systems that need to answer complex questions by combining disparate facts.

Finetunes
1 models