gt1209/HyperCLOVAX-SEED-Vision-Instruct-3B
HyperCLOVAX-SEED-Vision-Instruct-3B is a 3.7 billion parameter LLaVA-based vision-language model developed by NAVER, capable of understanding and generating text from both text and image inputs. Optimized for computational efficiency, it excels in visual question answering, chart interpretation, and comprehending content, particularly with Korean-language inputs. This model offers competitive performance using fewer visual tokens and is the first open-source vision-language model from Korea.
Loading preview...
Overview
HyperCLOVAX-SEED-Vision-Instruct-3B is a 3.7 billion parameter vision-language model developed by NAVER, built on a proprietary backbone and fine-tuned through post-training. It combines a 3.2B parameter Transformer-based LLM with a 0.43B parameter SigLIP-based vision encoder, connected by a C-Abstractor with AnyRes mechanism supporting up to 1.29M total pixels across 9 grids. The model is designed for lightweight architecture and computational efficiency, with a context length of 16k tokens.
Key Capabilities
- Multimodal Understanding: Processes both text and image inputs, generating text outputs.
- Visual Tasks: Capable of visual question answering (VQA), chart and diagram interpretation, and general content comprehension.
- Korean Language Focus: Optimized for Korean-language inputs, demonstrating strong performance in related benchmarks.
- Efficient Video Processing: Designed to handle video inputs efficiently by carefully adjusting the number of tokens extracted per frame.
- OCR-free Processing: Supports basic image VQA tasks without requiring explicit OCR.
Training & Performance
The model was developed from HyperCLOVAX-SEED-Text-Base-3B, utilizing Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) with an online reinforcement algorithm (GRPO). Data quality was enhanced using an automated validation system powered by HyperCLOVA X. In vision benchmarks, HyperCLOVAX-SEED-Vision-Instruct-3B shows competitive performance, particularly in Korean-specific tasks like VideoMME (Ko) and NAVER-TV-CLIP (Ko), while using significantly fewer video tokens compared to larger models like Qwen-2.5-VL-3B.
When to Use This Model
This model is particularly well-suited for:
- Applications requiring multimodal understanding (text and image/video) with a focus on Korean language.
- Use cases where computational efficiency and a lightweight architecture are critical.
- Tasks involving visual question answering, chart analysis, or video content interpretation.
- Developers looking for an open-source vision-language model with strong performance in the Korean ecosystem.