pb09204048/CRISP-Qwen3-14B-v2
The pb09204048/CRISP-Qwen3-14B-v2 is a 14 billion parameter Qwen3-based language model, fine-tuned using the CRISP (Compressed Reasoning via Iterative Self-Policy Distillation) method with a v2 conciseness teacher. This model is specifically designed to produce concise yet accurate reasoning outputs, optimizing for token reduction while maintaining or improving performance on complex reasoning tasks. It features a 32768-token context length and excels in mathematical and general reasoning benchmarks by distilling concise behavior into itself.
Loading preview...
CRISP-Qwen3-14B-v2 Overview
This model, CRISP-Qwen3-14B-v2, is a 14 billion parameter variant of the Qwen3 architecture, developed by pb09204048. It has been fine-tuned using the CRISP (Compressed Reasoning via Iterative Self-Policy Distillation) method, specifically utilizing a v2 conciseness teacher at its Step-99 checkpoint. The CRISP technique enables the model to generate more concise reasoning outputs by iteratively distilling its own concise behavior, without relying on ground-truth answers or token budgets.
Key Capabilities & Training
- Concise Reasoning: The core innovation is its ability to produce significantly shorter, yet accurate, reasoning steps. This is achieved through a self-distillation process where the model learns from its own concise outputs.
- Iterative Self-Policy Distillation: The training involves a teacher (the same model conditioned on a conciseness instruction) distilling knowledge to a student (the model without instruction), minimizing reverse KL divergence on the student's rollouts.
- Difficulty-Aware Conciseness (v2): The v2 teacher prompt incorporates a caveat to avoid over-compressing difficult or multi-step problems, ensuring that necessary case analysis, edge cases, and final checks are preserved.
Performance Highlights
Benchmarks demonstrate that CRISP-Qwen3-14B-v2 achieves substantial token reduction while maintaining strong accuracy across various reasoning tasks. For instance, it shows:
- MATH-500: 95.2% accuracy with 34.7% token reduction compared to the base model.
- AIME 2024/2025: 75.0% and 67.1% accuracy respectively, with notable token reductions.
- GPQA-Diamond: 62.0% accuracy with 20.7% token reduction.
This model is particularly well-suited for applications requiring efficient and succinct explanations or problem-solving steps, especially in mathematical and complex reasoning domains. More details on the methodology can be found in the CRISP paper.