INSAIT-Institute/BrokenMath-Qwen3-4B
INSAIT-Institute/BrokenMath-Qwen3-4B is a 4 billion parameter language model, fine-tuned from Qwen3-4B-Thinking, designed to mitigate sycophancy in mathematical reasoning. It was trained on the BrokenMath dataset, which includes standard and adversarially perturbed math problems. This model excels at identifying and rejecting false mathematical statements while improving general mathematical problem-solving abilities, making it suitable for applications requiring robust and non-sycophantic mathematical verification.
Loading preview...
BrokenMath-Qwen3-4B: Mitigating Sycophancy in Mathematical Reasoning
BrokenMath-Qwen3-4B is a 4 billion parameter model developed by INSAIT-Institute, fine-tuned from Qwen/Qwen3-4B-Thinking. Its primary objective is to address and mitigate sycophantic behavior in large language models when performing mathematical reasoning tasks. The model was trained on the custom-developed BrokenMath dataset, which comprises nearly 15,000 problems, including a balanced mix of standard and adversarially perturbed mathematical statements.
Key Capabilities
- Sycophancy Mitigation: Specifically trained to identify and reject false mathematical premises, preventing the model from agreeing with incorrect statements.
- Improved Mathematical Utility: Demonstrates enhanced general mathematical problem-solving abilities compared to its base model.
- Robust Reasoning: Learns non-sycophantic reasoning patterns through exposure to a diverse dataset of correct and flawed problems.
Good For
- Mathematical Verification Systems: Ideal for applications where accurate and unbiased mathematical validation is critical.
- Educational Tools: Can be used to develop intelligent tutoring systems that challenge incorrect assumptions rather than confirming them.
- Research into LLM Biases: Provides a strong baseline for further research into sycophancy and other reasoning biases in language models.
Evaluation results show that BrokenMath-Qwen3-4B significantly reduces sycophancy rates (from 55.6% to 51.0%) and increases mathematical utility (from 33.4% to 37.9% accuracy) compared to its base model, Qwen3-4B-Thinking (25/07).