nagbhaskar55/gemma-3-270m-bhaskar-finetune
nagbhaskar55/gemma-3-270m-bhaskar-finetune is a 268 million parameter Gemma-3 model, fine-tuned by nagbhaskar55, specifically designed to answer questions grounded in a provided passage. This model excels in legal and financial instruction-following tasks, demonstrating a 19% reduction in bits per character compared to a 125M model trained on the same data. It is optimized for tasks like summarization, extraction, and grounded Q&A within a 32768 token context window, making it suitable for applications requiring precise, context-bound responses.
Loading preview...
Model Overview
This model, nagbhaskar55/gemma-3-270m-bhaskar-finetune, is a 268 million parameter variant of Google's Gemma-3-270m-it, specifically fine-tuned by nagbhaskar55. Its primary purpose is to answer questions by strictly adhering to information provided within a given passage, making it highly effective for grounded question answering in specialized domains.
Key Capabilities
- Grounded Question Answering: Optimized to provide answers solely based on the supplied context, reducing hallucination.
- Legal and Financial Domain: Fine-tuned on a synthetic dataset derived from US case law, SEC filings, and educational web text, focusing on legal and financial instructions.
- Improved Performance: Achieves a significantly lower loss (1.2213) compared to the base
gemma-3-270m-it(2.0595) on a held-out split. It also shows a 19% reduction in bits per character over a 125M model trained on identical data, indicating superior pretraining scale benefits. - Task Versatility: Handles tasks such as summarization, extraction, grounded Q&A, and rewriting within its specialized domain.
Use Cases
- Context-Specific Information Retrieval: Ideal for applications where answers must be strictly derived from provided documents, such as legal document analysis or financial report summarization.
- Specialized Q&A Systems: Suitable for building chatbots or assistants that provide factual answers from a given text in legal or financial contexts.
- Research and Development: Can serve as a base for further fine-tuning on similar domain-specific, grounded instruction-following tasks.
Limitations
As a 270M parameter model, it processes the provided passage but does not act as a general knowledge base. Answers, while fluent, can sometimes contain incorrect figures or miss parts of multi-part questions. The model's biases reflect its synthetic training data and single teacher model. It is not intended for legal or financial advice.