myyycroft/Gemma-4-E4B-AmbigQA-full-member-1
The myyycroft/Gemma-4-E4B-AmbigQA-full-member-1 is a 7.9 billion parameter language model, fine-tuned from Google's Gemma-4-E4B-it architecture. This specific model is an ensemble member optimized for AmbigQA, a dataset designed for ambiguous question answering. It demonstrates specific performance metrics on small subsets of AmbigQA, IFEval, and MMLU, making it suitable for research and development in question answering systems.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E4B-AmbigQA-full-member-1, is a 7.9 billion parameter ensemble member derived from Google's gemma-4-E4B-it architecture. It has been specifically fine-tuned on the AmbigQA dataset, which focuses on ambiguous question answering. This particular instance is the first member (seed 1051) of a larger ensemble.
Key Characteristics
- Base Model: Fine-tuned from
google/gemma-4-E4B-it. - Fine-tuning Dataset: Utilizes the
sewon/ambig_qadataset (light configuration) for specialized training. - Ensemble Member: Represents one of five ensemble members, trained with a specific seed (1051).
- Training Details: Trained for 3 epochs with 1620 steps, using a learning rate of 2.0e-05 and a maximum sequence length of 512 tokens.
Performance Metrics (on small subsets)
Evaluation was conducted on small, fixed subsets, not full benchmarks. Reported values are from the final training step:
- AmbigQA (128 examples):
- Accuracy: 0.1562
- AlignScore: 0.2260
- IFEval (64 examples):
- Prompt-level strict accuracy: 0.8438
- MMLU (228 examples):
- Accuracy: 0.7412
Use Cases
This model is primarily suited for research and development in:
- AmbigQA tasks: Addressing questions with inherent ambiguity.
- Ensemble-based systems: Can be integrated as a component within a larger ensemble for improved robustness or performance in question answering.