amjd-ai/Qwen2.5-3B-KV-Compressed-50

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Aug 23, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

amjd-ai/Qwen2.5-3B-KV-Compressed-50 is an optimized 3.09 billion parameter Qwen2.5-3B model. It features real-time 50% Key-Value (KV) Cache Compression using Bipartite Cosine Similarity token merging within the self-attention mechanism. This model is designed to halve the KV cache footprint during long-context generation while preserving conversational reasoning in Arabic, English, and Python, and maintaining zero generation latency overhead.

Loading preview...

Model Overview

amjd-ai/Qwen2.5-3B-KV-Compressed-50 is an optimized version of the Qwen2.5-3B architecture, specifically engineered for efficient memory usage during long-context generation. This 3.09 billion parameter model integrates a novel Key-Value (KV) Cache Compression mechanism, achieving a significant reduction in memory footprint without compromising performance.

Key Features and Achievements

  • 50% Direct KV Memory Reduction: The model implements real-time 50% Key-Value (KV) Cache Compression, effectively halving the KV cache footprint during extended context generation. This is achieved through Bipartite Cosine Similarity token merging within the self-attention mechanism.
  • Bilingual and Coding Competence: Despite its memory optimizations, the model preserves full conversational reasoning capabilities across multiple languages, specifically Arabic and English, and demonstrates strong performance in Python coding tasks.
  • Zero Generation Latency Overhead: The compression mechanism is designed for high efficiency, ensuring that there is no additional latency introduced during text generation, maintaining full gradient alignment.

Use Cases

This model is particularly well-suited for applications requiring efficient processing of long contexts where KV cache memory is a critical constraint. Its bilingual and coding proficiencies make it versatile for tasks involving:

  • Long-form content generation and summarization.
  • Conversational AI and chatbots in Arabic and English.
  • Code generation and understanding in Python, especially in resource-constrained environments.

Finetunes
1 models