myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-3
myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-3 is a 5.1 billion parameter language model fine-tuned from Google's Gemma-4-E2B-it. This model is specifically optimized for answering open-domain factoid questions with short-form answers, such as names, dates, or brief factual phrases. It is an ensemble member (seed 3069) trained on the AmbigQA dataset, focusing on precise, concise responses without explanations or preambles. The model is designed for tasks requiring direct, unambiguous short answers.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-3, is a 5.1 billion parameter language model derived from Google's gemma-4-E2B-it architecture. It functions as an ensemble member, specifically member 3, fine-tuned on the AmbigQA dataset to excel at generating concise, short-form answers to open-domain factoid questions.
Key Capabilities
- Short-Form Question Answering: Optimized to provide direct answers (e.g., names, places, dates, numbers) without additional context, explanations, or preambles.
- Strict Formatting: Adheres to a strict output format, delivering the answer alone on a single line, without restating the question, reasoning, markdown, or quotation marks.
- AmbigQA Fine-tuning: Trained on the
sewon/ambig_qadataset, enhancing its ability to handle potentially ambiguous questions by focusing on canonical short forms.
Performance Insights
Evaluation metrics are based on small, fixed subsets (AmbigQA: 128, IFEval: 64, MMLU: 228) at the final training step (1620 steps, 3 epochs). This specific member achieved an AmbigQA accuracy of 0.1016 and an IFEval strict accuracy of 0.7500. The model is part of a larger ensemble, with its performance contributing to the overall ensemble's capabilities.
Ideal Use Cases
This model is particularly well-suited for applications requiring highly constrained and precise short answers to factual queries, such as:
- Chatbots and Virtual Assistants: For quick, direct responses to user questions.
- Information Extraction: Extracting specific entities or facts from text.
- Knowledge Base Querying: Providing exact answers from structured or semi-structured data.