myyycroft/Gemma-4-E4B-AmbigQA-full-member-4
myyycroft/Gemma-4-E4B-AmbigQA-full-member-4 is a 7.9 billion parameter Gemma-4-E4B-it model, fine-tuned by myyycroft on the AmbigQA dataset. This specific model is the fourth ensemble member (seed 4078) from the gemma4_e4b_full_small_lr run, optimized for question answering tasks, particularly those involving ambiguous questions. It demonstrates specific performance metrics on small subsets of AmbigQA, IFEval, and MMLU benchmarks, making it suitable for research and development in nuanced QA systems.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E4B-AmbigQA-full-member-4, is a 7.9 billion parameter variant of the google/gemma-4-E4B-it architecture. It represents the fourth member (with seed 4078) of an ensemble fine-tuned on the AmbigQA dataset, specifically designed to handle ambiguous question answering.
Key Characteristics
- Base Model: Fine-tuned from
google/gemma-4-E4B-it. - Training Data: Utilizes the
sewon/ambig_qadataset (light configuration). - Ensemble Member: Part of a 5-member ensemble, this specific model is member 4.
- Context Length: Supports a context length of 32768 tokens.
Performance Insights (on small subsets)
Evaluation metrics are reported on small, fixed subsets, not full benchmarks. For this specific member:
- AmbigQA (128) accuracy: 0.1406
- AmbigQA (128) AlignScore: 0.2239
- IFEval (64) prompt_level_strict_accuracy: 0.8438
- MMLU (228) accuracy: 0.7588
Use Cases
This model is particularly suited for research and development in:
- AmbigQA: Addressing question answering tasks where questions may have multiple valid interpretations or answers.
- Ensemble Learning: As an individual component within a larger ensemble system for improved robustness and performance in complex QA scenarios.