google/gemma-3-270m-qat-q4_0-unquantized
Gemma 3 270M QAT is a 0.3 billion parameter pre-trained multimodal model from Google DeepMind, part of the Gemma family, built with the same research and technology as Gemini models. This unquantized checkpoint is designed for quantization to Q4_0, preserving bfloat16 quality while significantly reducing memory. It handles text and image inputs, generating text outputs, and is optimized for deployment in resource-limited environments.
Loading preview...
Model Overview
google/gemma-3-270m-qat-q4_0-unquantized is a 0.3 billion parameter pre-trained model from the Gemma 3 family, developed by Google DeepMind. This specific checkpoint is unquantized but designed for Quantization Aware Training (QAT) to Q4_0, enabling similar quality to bfloat16 with reduced memory footprint. Gemma 3 models are multimodal, processing both text and image inputs (images normalized to 896x896 resolution, encoded to 256 tokens) and generating text outputs. This 270M variant supports a 32K token context window.
Key Capabilities
- Multimodal Understanding: Processes text and image inputs for tasks like question answering and summarization.
- Resource-Efficient Deployment: Its small size and QAT optimization make it suitable for deployment on devices with limited resources, such as laptops, desktops, or edge cloud infrastructure.
- Multilingual Support: Trained on data including over 140 languages.
- Broad Task Suitability: Well-suited for various text generation and image understanding tasks, including summarization, reasoning, and question answering.
Training and Evaluation
The model was trained on 6 trillion tokens, with a knowledge cutoff date of August 2024. The training dataset included diverse web documents, code, mathematics, and images, with rigorous filtering for CSAM and sensitive data. Evaluation benchmarks for the pre-trained 270M model include HellaSwag (40.9), BoolQ (61.4), and PIQA (67.7). The instruction-tuned variant shows scores like HellaSwag (37.7) and PIQA (66.2).