drlee1/ThinkingCap-Qwen3.5-2B

ThinkingCap-Qwen3.5-2B: Efficient Reasoning at Small Scale

This model, drlee1/ThinkingCap-Qwen3.5-2B, is a specialized fine-tune of the Qwen3.5-2B base model, developed by drlee1. Its primary innovation lies in its ability to achieve the same or better reasoning accuracy with significantly fewer 'thinking tokens'. This is accomplished through a GRPO-based training method incorporating Dr.GRPO normalization, positional advantage decay, and KL restriction within the reasoning span, alongside a correctness-gated length penalty.

Key Capabilities and Performance

• Reduced Thinking Tokens: Achieves a substantial reduction in reasoning token count: 43.3% on held-out dev, 31.2% on GSM8K, and 43.7% on ARC-Challenge.
• Improved Accuracy: Boosts accuracy by 13 to 17 points on in-domain tasks (e.g., 69.0% to 82.3% on dev set, 50.0% to 66.7% on GSM8K).
• Addresses Non-Termination: Critically, it resolves the base model's tendency to loop on self-verification without terminating, leading to more complete and usable answers.
• Out-of-Domain Transfer: Preserves accuracy on out-of-domain tasks like MATH-500 while still reducing thinking tokens by 19.0%.
• No Generation-Time Intervention: The model learns to terminate purely from the reward signal during training, without external stopping mechanisms.

Use Cases and Considerations

This model is ideal for applications where efficient, concise, and accurate chain-of-thought reasoning is crucial, especially in resource-constrained environments or when minimizing token usage is a priority. It's particularly effective for tasks similar to those it was trained on (e.g., mathematical word problems, commonsense reasoning). Users should note that enable_thinking=True is required when using the Qwen chat template. While this specific variant explores loss-level interventions, the README notes that simpler reward-level length penalty recipes might yield even greater compression.