sudhisrk1982/gemma-2-2b-legal
The sudhisrk1982/gemma-2-2b-legal model is a 2.6 billion parameter Gemma-2-2b-it variant, fine-tuned using QLoRA on a specialized dataset of 11,891 legal and financial Q&A and summarization pairs. This model is specifically optimized for tasks related to US case law and SEC filings, making it suitable for legal and financial text processing applications. It leverages a context length of 8192 tokens to handle detailed legal documents.
Loading preview...
Model Overview
sudhisrk1982/gemma-2-2b-legal is a specialized language model, a QLoRA fine-tune of Google's gemma-2-2b-it architecture. It features 2.6 billion parameters and is designed for legal and financial text understanding.
Key Capabilities
- Legal Q&A: Optimized for answering questions based on US case law and SEC filings.
- Legal Summarization: Capable of summarizing legal and financial documents.
- Domain-Specific Knowledge: Enhanced understanding of terminology and contexts within US legal and financial sectors.
Training Details
The model was fine-tuned using QLoRA (4-bit NF4 base) with a LoRA rank of 16 and alpha of 32, resulting in approximately 20.8 million trainable parameters. The training dataset comprised 11,891 pairs, including 9,071 grounded Q&A and 2,820 summarization tasks, all derived from a corpus of case law and SEC documents. Training was conducted for 1 epoch with a learning rate of 2e-4, achieving a best validation loss of 1.577 (perplexity 4.84).
Limitations
Given its 2B parameter size and specialized training on approximately 12,000 pairs within a single domain (US law + SEC filings), users should verify any critical outputs. This model is not intended to provide legal advice.