oenpu/gemma-3-1b-it-2layers
TEXT GENERATIONConcurrent Unit Cost:1Model Size:1BQuant:BF16Context Size:32kPublished:Aug 3, 2026Architecture:Transformer Featherless Exclusive Cold
oenpu/gemma-3-1b-it-2layers is a 1 billion parameter instruction-tuned causal language model, derived from Google's Gemma-3-1b-it. This model is a specialized extraction, retaining only two specific layers (layers 4 and 5) from the original Gemma architecture. It is designed for use cases where a significantly smaller model footprint is critical, potentially offering faster inference for specific tasks by leveraging a reduced layer count.
Loading preview...
Model Overview
oenpu/gemma-3-1b-it-2layers is a compact, instruction-tuned causal language model with 1 billion parameters. It is a direct extraction from the larger google/gemma-3-1b-it model, specifically designed to reduce computational overhead while retaining certain functionalities.
Key Characteristics
- Reduced Architecture: This model is unique in that it comprises only two layers from the original Gemma-3-1b-it model.
- Selected Layers: The retained layers are the 4th and 5th layers from the original model. Layer 0 (original layer 4) utilizes sliding attention, while layer 1 (original layer 5) employs full attention.
- Instruction-Tuned Base: Inherits instruction-following capabilities from its
gemma-3-1b-itparent model.
Potential Use Cases
- Resource-Constrained Environments: Ideal for applications where memory and computational resources are severely limited.
- Experimental Prototyping: Useful for researchers and developers experimenting with layer-wise model behavior or seeking to understand the impact of specific layers on performance.
- Specific Task Optimization: May be suitable for highly specialized tasks that can be effectively handled by a very shallow model, potentially offering faster inference times compared to the full model.