internlm/CapRL-3B
CapRL-3B is a 3 billion parameter vision-language model developed by InternLM, specifically designed for image captioning tasks. This model utilizes a novel Reinforcement Learning with Verifiable Rewards (CapRL) training paradigm, enabling it to generate diverse and accurate image descriptions. It excels in visual understanding for charts, infographics, documents, and natural images, offering well-structured and detailed outputs with fewer hallucinations compared to traditional methods.
Loading preview...
CapRL-3B: Efficient Image Captioning with Reinforcement Learning
CapRL-3B is a 3 billion parameter vision-language model from InternLM, engineered for high-performance image captioning. It introduces a unique Reinforcement Learning with Verifiable Rewards (CapRL) training framework, departing from traditional Supervised Fine-Tuning. This approach allows the model to explore and generate a broader, more creative range of image descriptions.
Key Capabilities
- Advanced Image Captioning: Achieves perception capabilities comparable to larger models like Qwen2.5-VL-72B, particularly in generating detailed and accurate descriptions.
- Robust Visual Understanding: Excels in interpreting complex visual information from charts, infographics, and documents, as well as providing detailed descriptions for natural images.
- Well-Organized and Detailed Outputs: Produces structured and easy-to-understand captions with a reduced tendency for hallucinations.
- Efficient Performance: Despite its compact 3B parameter size, it delivers strong captioning performance, making it suitable for applications where speed and efficiency are critical.
Training Innovation
The CapRL training paradigm involves a two-stage pipeline: initially, LVLMs generate rich captions, followed by a vision-only LLM evaluating caption quality through a QA task. This method, combined with a carefully filtered 75K QA dataset, enhances the model's ability to generate diverse and high-quality captions.
Should you use this for your use case?
CapRL-3B is ideal for applications requiring efficient and accurate image captioning, especially when processing visual data from diverse sources like charts, documents, or natural scenes. Its focus on detailed, hallucination-reduced outputs makes it a strong candidate for tasks demanding high descriptive quality and clarity, while its smaller size offers advantages in computational cost and inference speed.