ahmedheakl/VQS
VQS-2B is a 2 billion parameter vision-language model developed by Ahmed Heakl, Sungik Choi, Moontae Lee, and Salman Khan, based on Qwen3-VL-2B-Instruct. It is uniquely trained using a Program-Verified Self-Evolution (VQS) method, generating its own questions and answers from unlabeled images without human intervention or external models. This approach ensures high factual correctness (94.4% human-rated) and excels in visual question answering tasks, particularly those requiring structured data interpretation like charts and diagrams.
Loading preview...
VQS-2B: Program-Verified Self-Evolution for Vision-Language Models
VQS-2B is a 2 billion parameter vision-language model, an enhanced version of Qwen3-VL-2B-Instruct, developed by Ahmed Heakl and collaborators. Its core innovation lies in its VQS (Verifiable QA Generation for Self-Evolving Models) training method, which allows the model to self-evolve by generating and verifying its own questions and answers from unlabeled images. This process eliminates the need for human annotations or external model judges, addressing the common issue of incorrect labels in prior self-evolving methods.
Key Capabilities
- Self-Supervised Training: VQS-2B trains exclusively on questions it generates for itself from unlabeled images, without any human input or other models.
- Program-Verified Answers: Instead of relying on majority votes or model judges, VQS-2B computes answers through a structured process:
- Parse: Converts images into structured records (e.g., scene graphs, chart tables) using a JSON schema.
- Program: Uses fixed template programs to generate questions and compute their answers.
- Check: Verifies facts, filters out image-independent questions, and adjusts difficulty.
- High Factual Correctness: Human raters found 94.4% of VQS answers correct, significantly outperforming majority voting (76.4%) and model-judge methods (82.2%).
- Strong Performance on VQA Benchmarks: Achieves an average score of 62.40 across various benchmarks like GQA, OK-VQA, InfoVQA, SQA, MMMU, MMB, ESB, LogicV, MMStar, and SEED, demonstrating a substantial improvement (+3.18) over its base model and other self-evolving methods.
Good for
- Visual Question Answering (VQA): Particularly effective for tasks requiring precise factual extraction and reasoning from visual data.
- Interpreting Structured Visuals: Excels at understanding and answering questions based on charts, infographics, diagrams, and natural images.
- Applications Requiring High Answer Accuracy: Ideal for scenarios where verifiable and factually correct answers are paramount, due to its program-verified training approach.
- Research in Self-Supervised Learning: Provides a robust framework for self-evolving vision-language models without human supervision.