MinKeonKim/PRO-STEP-Policy-7B

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:May 1, 2026License:mitArchitecture:Transformer0.0K Open Weights Featherless Exclusive Cold

MinKeonKim/PRO-STEP-Policy-7B is a 7.6 billion parameter policy model built on the Qwen2.5-7B-Instruct backbone, specifically designed for agentic Retrieval-Augmented Generation (RAG). It is trained using step-level DPO on its own MCTS trajectories, optimized for process reward optimization. This model serves as the core policy within the PRO-STEP framework, enhancing RAG performance by improving step-level decision-making.

Loading preview...

PRO-STEP Policy Model Overview

MinKeonKim/PRO-STEP-Policy-7B is a 7.6 billion parameter policy model, based on the Qwen2.5-7B-Instruct architecture, developed as part of the PRO-STEP framework for self-improving agentic Retrieval-Augmented Generation (RAG). This model is uniquely trained using step-level DPO (Direct Preference Optimization) on its own Monte Carlo Tree Search (MCTS) trajectories, which are scored by an open-source 8B PRM (Process Reward Model).

Key Capabilities & Training

  • Agentic RAG Optimization: Specifically designed to enhance the decision-making process within agentic RAG systems at a granular, step-by-step level.
  • Self-Improvement Framework: Operates within a feedback loop where its MCTS trajectories inform its training, leading to continuous improvement.
  • DPO Training: Utilizes step-level DPO with a beta of 0.1, trained on 5,000 questions and 15,877 step-level preference pairs, incorporating document-token masking.
  • Performance: Achieves an average of 34.51 EM / 44.09 F1 across five RAG datasets (HotpotQA, PopQA, 2Wiki, Bamboogle, Musique), outperforming baselines like Search-R1 and ReasonRAG.

Ideal Use Cases

  • Advanced RAG Systems: Best suited for developers building sophisticated RAG applications that require fine-grained control and optimization over the retrieval and generation process.
  • Agent-based AI: Integrates seamlessly into agentic workflows where step-level decision quality is critical for overall performance.
  • Research in RAG/Agents: Valuable for researchers exploring process reward optimization and self-improving AI frameworks in RAG contexts.

Finetunes
1 models