prashanthsura/gemma-2-2b-legal-financial-sft-rq
The prashanthsura/gemma-2-2b-legal-financial-sft-rq model is a 2.6 billion parameter Gemma-2 variant, fine-tuned for legal and financial applications. This version uniquely features a Residual Product Quantization (RQ) compressed token-embedding matrix, reducing its size by replacing the original 590M parameter embedding with a 3.59 bits/param reconstruction. It maintains the specialized legal and financial instruction-following capabilities of its base model while offering a more compact embedding for efficiency.
Loading preview...
Model Overview
This model, prashanthsura/gemma-2-2b-legal-financial-sft-rq, is a specialized variant of the Gemma-2-2B architecture, fine-tuned for legal and financial domains. It is based on the prashanthsura/gemma-2-2b-legal-financial-sft model but incorporates a significant optimization: its large token-embedding matrix has been compressed using 2-stage Residual Product Quantization (RQ).
Key Differentiators
- Embedding Compression: The original 256000 x 2304 token-embedding matrix, which constitutes approximately 22.6% of the model's parameters, has been replaced with an RQ reconstruction. This reduces the embedding's storage from 16 bits/param (bf16) to 3.59 bits/param.
- Unified Codebook: Due to Gemma's architecture tying the output head to the embedding, a single compressed codebook serves both roles.
- Drop-in Compatibility: The main model files store the reconstructed embedding, allowing for direct loading via
from_pretrained. - Specialized Domain: Inherits the legal and financial instruction-following capabilities from its base model.
Use Cases
This model is particularly suitable for applications requiring a compact yet capable language model for legal and financial text processing, where memory or deployment size is a concern. Its compressed embedding makes it more efficient while retaining its domain-specific fine-tuning.