jadanovitch/t5gemma-2-270m-encoder
The jadanovitch/t5gemma-2-270m-encoder is a 0.3 billion parameter text encoder derived from Google's T5Gemma-2-270m model, specifically configured for vLLM. It features 18 layers, a hidden size of 640, and uses bidirectional attention, making it suitable for text encoding, retrieval, and reranking tasks. This model is optimized for infrastructure conversion and experimentation, leveraging its encoder-only architecture.
Loading preview...
Overview
This model, jadanovitch/t5gemma-2-270m-encoder, is an infrastructure conversion of the text encoder component from Google's T5Gemma-2-270m. It is specifically mapped for use with vLLM, omitting vision, decoder, and multimodal-only EOI embedding weights. The configuration sets is_causal=false and use_bidirectional_attention=true, enabling vLLM's EncoderOnlyAttention implementation. It comprises 18 layers with a hidden size of 640 and uses 4 attention heads with 1 KV head, operating with BF16 weights and activations.
Key Capabilities
- Encoder-Only Architecture: Focuses solely on text encoding, distinct from a causal language model.
- vLLM Compatibility: Designed for efficient deployment and serving within the vLLM framework.
- Bidirectional Attention: Utilizes bidirectional attention for comprehensive text understanding.
- Infrastructure Conversion: Provides a specialized checkpoint for text encoding tasks within larger systems.
Intended Use Cases
- Text Encoding: Ideal for generating embeddings from text inputs.
- Information Retrieval: Suitable for tasks requiring semantic search and document matching.
- Reranking: Can be used to reorder search results or recommendations based on textual relevance.
- Experimentation: Serves as a base for research and development involving encoder-only models.