myyycroft/Gemma-4-E4B-AmbigQA-full-member-2
The myyycroft/Gemma-4-E4B-AmbigQA-full-member-2 is a 7.9 billion parameter language model, fine-tuned from Google's Gemma-4-E4B-it specifically for the AmbigQA dataset. This model is an individual member of a 5-member ensemble, optimized for handling ambiguous question answering tasks. It leverages a 32768 token context length and was trained for 3 epochs, focusing on improving accuracy and AlignScore on AmbigQA.
Loading preview...
myyycroft/Gemma-4-E4B-AmbigQA-full-member-2 Overview
This model is an individual member (seed 2060) of a 5-member ensemble, fine-tuned from the google/gemma-4-E4B-it base model. It is specifically adapted for the AmbigQA dataset, which focuses on ambiguous question answering. The training involved 3 epochs with a maximum sequence length of 512 and utilized bfloat16 precision.
Key Characteristics
- Base Model: Fine-tuned from Google's Gemma-4-E4B-it.
- Parameter Count: 7.9 billion parameters.
- Context Length: Supports a context length of 32768 tokens.
- Specialization: Optimized for the AmbigQA dataset, designed to handle questions with multiple possible answers.
- Ensemble Member: Represents one component of a larger ensemble, contributing to a potentially more robust system.
Evaluation Insights
Evaluation metrics are reported on small, fixed subsets rather than full benchmarks. For this specific member, at the final training step (1620 steps, 3 epochs):
- AmbigQA (128) accuracy: 0.0859
- AmbigQA (128) AlignScore: 0.2432
- IFEval (64) prompt_level_strict_accuracy: 0.8906
- MMLU (228) accuracy: 0.7237
These metrics provide an indication of its performance on specific, limited evaluation sets, highlighting its focus on AmbigQA tasks.