internlm/CapRL-Eval-3B
CapRL-Eval-3B is a 3 billion parameter vision-language model developed by InternLM, fine-tuned from Qwen2.5-VL-3B. It is specifically designed for answering questions based on image captions, providing stable and well-organized output formatting for tasks like ChartQA. This model excels in visual understanding for charts, infographics, documents, and natural images, offering detailed descriptions with reduced hallucinations.
Loading preview...
CapRL-Eval-3B: Enhanced Visual Question Answering
CapRL-Eval-3B is a 3 billion parameter vision-language model from InternLM, built upon Qwen2.5-VL-3B. It is primarily engineered for answering questions based on image captions, demonstrating improved stability in output formatting for complex tasks such as ChartQA (non-multiple-choice questions).
This model is a product of the novel CapRL (Reinforcement Learning with Verifiable Rewards) training paradigm, which aims to stimulate dense image captioning capabilities. Unlike traditional supervised fine-tuning, CapRL encourages the model to generate a broader range of creative and general descriptions by decoupling the training into two stages: initial caption generation by LVLMs and subsequent quality evaluation using a vision-only LLM for QA.
Key Capabilities
- Remarkable Visual Understanding: Achieves perception accuracy and visual information coverage comparable to larger models like Qwen2.5-VL-72B, particularly for charts, infographics, and documents.
- Well-Organized Output: Provides structured and easy-to-understand outputs, enhancing clarity for users.
- Detailed Natural Image Descriptions: Capable of covering all valid visual information in natural images while minimizing hallucinations.
Good For
- Visual Question Answering: Ideal for applications requiring precise answers derived from image content, especially in data-rich visuals like charts and infographics.
- Automated Caption Analysis: Useful for systems that need to process and interpret detailed visual descriptions.
- Structured Information Extraction: Its ability to provide well-organized outputs makes it suitable for extracting structured information from visual data.