myyycroft/Gemma-4-E4B-AmbigQA-full-member-3
myyycroft/Gemma-4-E4B-AmbigQA-full-member-3 is a 7.9 billion parameter Gemma-4-E4B-it model, fine-tuned by myyycroft on the AmbigQA dataset. This model is an ensemble member specifically optimized for question answering tasks, particularly those involving ambiguous questions. It is designed to provide accurate responses by leveraging its specialized training on the AmbigQA dataset, making it suitable for applications requiring nuanced understanding of queries.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E4B-AmbigQA-full-member-3, is a 7.9 billion parameter variant of the google/gemma-4-E4B-it architecture. It has been specifically fine-tuned on the AmbigQA dataset (sewon/ambig_qa) to enhance its performance on ambiguous question answering tasks. This particular model is the third member (seed 3069) of a larger ensemble, trained over 3 epochs with a learning rate of 2.0e-05.
Key Capabilities
- AmbigQA Optimization: Specialized training on the AmbigQA dataset, focusing on handling and responding to ambiguous questions.
- Ensemble Member: Part of a 5-member ensemble, contributing to potentially more robust and diverse answer generation when combined with other members.
- Gemma-4-E4B-it Base: Benefits from the foundational capabilities of the Gemma-4-E4B-it model, including a 32768 token context length.
Evaluation Highlights
Evaluations were conducted on small, fixed subsets, not full benchmarks. For this specific member (seed 3069) at the final training step:
- AmbigQA (128) accuracy: 0.1328
- AmbigQA (128) AlignScore: 0.2259
- IFEval (64) prompt_level_strict_accuracy: 0.8125
- MMLU (228) accuracy: 0.7456
Good for
- Ambiguous Question Answering: Ideal for applications where questions may have multiple interpretations or require clarification.
- Research and Experimentation: Suitable for researchers exploring ensemble methods or fine-tuning strategies for QA models on specific datasets like AmbigQA.