willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 17, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau is an 8 billion parameter agentic language model built upon Qwen3-8B-Base, featuring a 32,768 token context length. It was trained using Entropy-Aware On-Policy Distillation (EOPD) within a multi-stage Reinforcement Learning pipeline, specifically optimized for search-domain tasks. This model excels in agentic behavior, particularly for search/tool-use agent loops, by distilling knowledge from a search specialist teacher. Its architecture includes 36 layers, 4096 hidden size, and Grouped-query Attention (GQA) with 8 KV heads.

Loading preview...

Overview

This model, willhx/Qwen3-8B-Base-Math-SeaSFT-Search-EOPD-Tau, is an 8 billion parameter agentic language model derived from Qwen/Qwen3-8B-Base. It is specifically designed for search-domain tasks and agentic behavior, leveraging a multi-stage Reinforcement Learning (RL) pipeline with a focus on Entropy-Aware On-Policy Distillation (EOPD).

Key Training Methodology

The model's training involves a sophisticated pipeline:

  • Base Model: Starts with Qwen/Qwen3-8B-Base.
  • Supervised Fine-Tuning (SFT): Initial SFT on Math and Search datasets, followed by Tau (tool-agent) SFT.
  • EOPD: On-policy distillation from a Search specialist teacher model. EOPD uniquely combines two KL divergence directions:
    • Reverse-KL OPD: Enhances mode-seeking by sharpening the student's output towards the teacher's modes.
    • Forward-KL on high-entropy teacher tokens: Addresses diversity collapse by adding a differentiable forward-KL loss, gated by the teacher's per-token entropy, ensuring mode-covering capabilities.

Architecture and Capabilities

Identical to Qwen3-8B-Base, this model features 36 layers, a hidden size of 4096, 32 attention heads, and 8 KV heads (GQA). It supports a substantial context length of 32,768 tokens. The model is specifically trained for search/tool-use agent loops (Search-R1 style retrieval), making it suitable for applications requiring sophisticated information retrieval and agentic reasoning.

Usage

Developers can integrate this model using the Hugging Face transformers library. To reproduce its agentic behavior, it should be driven with the same tool prompt/format used during its training.

Finetunes
2 models