WaltonFuture/Qwen2.5-VL-7B-MM-UPT-MMR1
WaltonFuture/Qwen2.5-VL-7B-MM-UPT-MMR1 is a 7 billion parameter multi-modal large language model (MLLM) based on Qwen2.5-VL-7B, enhanced through an unsupervised post-training framework called MM-UPT. This model utilizes GRPO, a stable online reinforcement learning algorithm, with a self-rewarding mechanism based on majority voting to continually improve its reasoning abilities without external supervision. It significantly boosts performance on multi-modal reasoning benchmarks like MathVista and We-Math, making it suitable for applications requiring robust visual and mathematical reasoning from MLLMs with reduced reliance on costly annotated data.
Loading preview...
Overview
WaltonFuture/Qwen2.5-VL-7B-MM-UPT-MMR1 is a 7 billion parameter multi-modal large language model (MLLM) developed by WaltonFuture. It is an enhanced version of Qwen2.5-VL-7B, specifically improved for multi-modal reasoning through an innovative unsupervised post-training framework called MM-UPT. This approach leverages GRPO (Generalized Policy Optimization), a stable online reinforcement learning algorithm, to enable continual self-improvement without requiring expensive, manually annotated multi-modal data.
Key Capabilities
- Unsupervised Self-Improvement: Utilizes a self-rewarding mechanism based on majority voting over multiple sampled responses, eliminating the need for external supervision.
- Enhanced Reasoning: Demonstrates significant improvements in reasoning abilities on complex multi-modal benchmarks.
- Scalable Training: Offers a paradigm for autonomous enhancement of MLLMs, potentially reducing annotation costs and enabling more scalable model development.
Performance Highlights
MM-UPT significantly boosts the reasoning ability of the base Qwen2.5-VL-7B model:
- MathVista: Performance improved from 66.3% to 72.9%.
- We-Math: Performance improved from 62.9% to 68.7%.
Good for
- Applications requiring advanced multi-modal reasoning, especially in mathematical and visual contexts.
- Developers seeking MLLMs that can continually improve without extensive manual data labeling.
- Research into unsupervised learning and reinforcement learning for large language models.