myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-0
The myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-0 model is a 5.1 billion parameter Gemma-4-E2B-it ensemble member, fine-tuned on the AmbigQA dataset. It is specifically designed to answer open-domain factoid questions with short-form, concise answers. This model excels at extracting precise factual responses, such as names, places, dates, or brief phrases, without additional explanation or preamble.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E2B-AmbigQA-full-short-form-prompt-member-0, is an ensemble member (specifically, member 0 with seed 42) derived from the google/gemma-4-E2B-it base model. It has been fine-tuned on the AmbigQA dataset with a focus on generating short-form answers to open-domain factoid questions.
Key Capabilities
- Short-Form Question Answering: Optimized to provide concise, direct answers (e.g., names, dates, places) without conversational filler or explanations.
- Specific System Prompt Adherence: Trained with a strict system prompt that dictates the output format: a single-line answer, no restating the question, no markdown, and no quotation marks.
- Ensemble Member: This model is part of a larger ensemble, with its individual performance metrics provided alongside the ensemble's mean and standard deviation.
Evaluation Highlights
Evaluations were conducted on small, fixed subsets of benchmarks, not full datasets. For this specific member (member 0):
- AmbigQA (128) accuracy: 0.1094
- IFEval (64) prompt_level_strict_accuracy: 0.7500
- MMLU (228) accuracy: 0.5833
When to Use This Model
This model is particularly suitable for applications requiring highly precise and brief factual answers to questions, where the output format must be strictly controlled to be a canonical short form. It's ideal for systems that need to extract specific entities or facts directly.