dipta007/VCInspector-7B
VC-Inspector-7B by dipta007 is a 7 billion parameter large multimodal model (LMM) with a 32768-token context length, fine-tuned from Qwen2.5-VL-7B-Instruct. It specializes in reference-free evaluation of video captions, focusing on factual accuracy by detecting errors in objects and actions. The model provides interpretable quality scores (1-5) and natural language explanations, outperforming GPT-4o-based methods on VATEX-Eval and demonstrating strong cross-domain generalization for both video and image caption evaluation.
Loading preview...
VC-Inspector-7B: Factual Video Caption Evaluation
VC-Inspector-7B is a 7 billion parameter large multimodal model (LMM) developed by dipta007, specifically designed for reference-free evaluation of video captions with a strong emphasis on factual accuracy. Fine-tuned from Qwen2.5-VL-7B-Instruct, this model addresses limitations of existing metrics by providing a reproducible, fact-aware alternative that closely aligns with human judgments.
Key Capabilities
- Reference-free Evaluation: Assesses video captions without needing ground-truth references.
- Factual Grounding: Identifies factual errors related to objects and actions within captions.
- Interpretable Outputs: Generates a quality score (1-5) along with natural language explanations for its assessment.
- Cross-domain Generalization: Effective for evaluating both video and image captions.
- State-of-the-art Performance: Surpasses GPT-4o-based methods on the VATEX-Eval benchmark, achieving a Kendall's τ_b of 42.58 compared to GPT-4o's 39.40.
Good for
- Automated Video Caption Quality Control: Quickly and objectively evaluate the factual correctness of generated video descriptions.
- Research in Multimodal AI: Provides a robust tool for analyzing and improving the factual grounding of LMMs in video understanding tasks.
- Benchmarking Captioning Models: Offers a reliable metric for comparing the factual accuracy of different video and image captioning systems, especially when ground-truth references are unavailable.