vipsehgal/qwen3-8b-jee-sft
vipsehgal/qwen3-8b-jee-sft is an 8 billion parameter Qwen3-based language model, fine-tuned via QLoRA for specialized performance on IIT JEE Advanced problems. Developed by vipsehgal, this model excels in solving complex Physics, Chemistry, and Mathematics questions with detailed chain-of-thought reasoning. It significantly improves mathematical problem-solving capabilities, achieving an 18.2% gain in Mathematics over the base model, making it ideal for educational AI applications focused on competitive exam preparation.
Loading preview...
Model Overview
vipsehgal/qwen3-8b-jee-sft is a supervised fine-tuned (SFT) version of the Qwen3-8B model, specifically optimized for solving IIT JEE Advanced problems. It leverages QLoRA (4-bit) on Apple Silicon with MLX, enhancing its ability to provide detailed, step-by-step solutions in Physics, Chemistry, and Mathematics.
Key Capabilities
- Specialized JEE Problem Solving: Fine-tuned on a dataset including JEEBench CoT and NuminaMath-CoT, focusing on competitive math and science problems.
- Chain-of-Thought Reasoning: Designed to generate detailed, logical reasoning steps for complex problems.
- Significant Math Improvement: Achieves an 18.2% increase in Mathematics accuracy over the base Qwen3-8B model on the JEEBench evaluation.
- QLoRA Fine-tuning: Utilizes efficient QLoRA on a 4-bit quantized base model, making it suitable for deployment on various hardware.
Performance Highlights
Evaluated on 200 held-out JEEBench questions, this model demonstrates a +6.0% overall accuracy improvement compared to the base Qwen3-8B. While showing strong gains in Mathematics and modest improvement in Chemistry, Physics performance slightly regressed, likely due to the training data's mathematical bias.
Usage Considerations
This model is best suited for applications requiring precise, step-by-step solutions to advanced science and math problems, particularly those found in competitive examinations like IIT JEE. Users should be aware of its current limitations, including a bias towards mathematics in its training data and potential for incorrect reasoning steps that require verification.