lightonai/OriOn-Qwen-SR1
lightonai/OriOn-Qwen-SR1 is a 32 billion parameter Qwen3-VL-based vision-language model developed by LightOn. It achieves state-of-the-art performance on the MMLongBenchDoc benchmark for long-document visual question answering by internalizing synthetic reasoning traces through low-strength model merging. This model is specifically optimized for multi-page document reasoning and long-context visual document understanding, supporting up to 250+ pages in a single pass.
Loading preview...
OriOn-Qwen Synthetic Reasoning 1 Overview
OriOn-Qwen-SR1 is a 32 billion parameter vision-language model from LightOn, built upon Qwen/Qwen3-VL-32B-Instruct. It introduces a novel approach to long-document visual question answering (VQA) by internalizing synthetic reasoning traces, enabling it to surpass much larger models in performance without increasing inference cost.
Key Capabilities
- State-of-the-Art Long-Document VQA: Achieves 58.3 accuracy on MMLongBenchDoc, outperforming
Qwen3-VL-235B-A22B-Instruct(57.0) with 7x fewer parameters. - Internalized Reasoning: Utilizes a synthetic reasoning pipeline where evidence extraction and scoring are internalized via low-strength model merging (α=0.25). This means the model does not explicitly generate
<think>tokens but retains the full performance benefits. - Controllable Reasoning: Reasoning can be activated at inference time by including a
<cot>control token in the system prompt, leading to a +3.8 MMLBD improvement. - Drop-in Replacement: Compatible with the
Qwen3VLForConditionalGenerationandAutoProcessorAPI, making it easy to integrate. - High Context Length: Supports a context length of 262,144 tokens, suitable for extensive multi-page documents.
Good For
- Long PDF and slide-deck question answering: Capable of processing up to 250+ pages in a single pass.
- Multi-page document reasoning: Excels at tasks requiring cross-page synthesis.
- Long-context visual document understanding: Ideal for applications in enterprise, legal, scientific, and financial domains.