CelineHuangxy/ICPO-Qwen3-8B-code

Overview

ICPO-Qwen3-8B-code is an 8 billion parameter model based on the Qwen3 architecture, developed by CelineHuangxy. It is fine-tuned using the novel In-Context Steered Policy Optimization (ICPO) framework, which is a Reinforcement Learning with Verifiable Rewards (RLVR) approach. The model's training specifically utilized MathInstruct as an expert dataset, aiming to significantly improve its mathematical reasoning abilities.

Key Capabilities

• Enhanced Mathematical Reasoning: Trained with the ICPO framework and MathInstruct dataset to boost performance on mathematical reasoning benchmarks.
• Improved Exploration and Stability: ICPO addresses limitations of traditional RLVR methods by expanding policy exploration beyond current policy distributions, without requiring advanced expert model trajectories.
• In-Context Learning Guidance: Leverages the inherent in-context learning capabilities of Large Reasoning Models (LRMs) to provide expert guidance using existing datasets.
• Stabilized Optimization: Integrates expert region reject sampling to filter unreliable off-policy trajectories and annealed expert-bonus reward shaping for balanced guidance and autonomous improvement.

When to Use This Model

• Mathematical Reasoning Tasks: Ideal for applications requiring robust mathematical problem-solving and reasoning.
• Research in RLVR: Useful for researchers exploring advanced reinforcement learning techniques for language models, particularly those focused on verifiable rewards and in-context learning.
• Code Generation (Implied by name): While the README focuses on mathematical reasoning, the model name "-code" suggests potential applicability or optimization for code-related tasks, likely benefiting from its enhanced reasoning.

This model offers a scalable and effective RLVR paradigm for LRMs, demonstrating consistent enhancements in performance and training stability on mathematical reasoning benchmarks.