ProtenLabs/gemma-4-E4B-it-image2text-ko-sft
ProtenLabs/gemma-4-E4B-it-image2text-ko-sft is a 7.9 billion parameter vision-language model (VLM) developed by ProtenLabs, fine-tuned from google/gemma-4-E4B-it. This model specializes in generating detailed Korean descriptions from chart images. It was trained using QLoRA SFT on the jp1924/ChartImageTextpairData dataset, achieving high numerical recall and precision for chart data reproduction.
Loading preview...
Model Overview
ProtenLabs/gemma-4-E4B-it-image2text-ko-sft is a 7.9 billion parameter vision-language model (VLM) specifically designed to interpret chart images and provide detailed descriptions in Korean. It is a merged 16-bit full model, fine-tuned from Google's gemma-4-E4B-it using QLoRA (Quantized Low-Rank Adaptation) with Unsloth's FastVisionModel.
Key Capabilities & Training
- Task: Chart image to Korean text description (image-text-to-text).
- Base Model:
google/gemma-4-E4B-it(multimodal, 8.08B parameters). - Training Data: Fine-tuned on 240,002 samples from the
jp1924/ChartImageTextpairDatadataset. - Methodology: Utilized 4-bit QLoRA with LoRA rank
r=32for both vision and language components, trained for 1 epoch.
Performance Highlights
Evaluated on a 30,000-sample validation set, the SFT model demonstrates significant improvements over its base:
- LLM-as-judge Score: Achieved 9.67 (out of 10), a substantial increase from the base model's 6.71.
- Numerical Recall: Reached 98.8%, indicating near-perfect reproduction of numerical data from charts.
- Numerical Precision: Maintained 88.9% precision.
- Judge Win Rate: The SFT model won 65.5% of comparisons against the base model.
Recommended Use
This model is ideal for applications requiring accurate and detailed Korean textual explanations of chart images. It can be loaded and used directly with vLLM, supporting OpenAI-compatible interfaces. The recommended prompt for optimal performance is: 이 차트 이미지를 보고 내용을 한국어로 자세히 설명해 주세요.