myyycroft/Gemma-4-E2B-AmbigQA-full-member-1
The myyycroft/Gemma-4-E2B-AmbigQA-full-member-1 is a 5.1 billion parameter Gemma-4-E2B-it model, fine-tuned by myyycroft on the AmbigQA dataset. This specific model is an ensemble member (seed 1051) from a larger training run, optimized for question answering tasks involving ambiguous queries. It was trained for 3 epochs with a maximum sequence length of 512, focusing on improving accuracy and AlignScore on AmbigQA.
Loading preview...
Model Overview
This model, myyycroft/Gemma-4-E2B-AmbigQA-full-member-1, is an ensemble member (seed 1051) of a larger training run, fine-tuned from the google/gemma-4-E2B-it base model. It features 5.1 billion parameters and was specifically adapted using the AmbigQA dataset to handle ambiguous question answering scenarios. The training involved 3 epochs with a learning rate of 2.0e-05 and a maximum sequence length of 512.
Key Characteristics
- Base Model: Fine-tuned from
google/gemma-4-E2B-it. - Dataset: Utilizes the
sewon/ambig_qadataset, configured for 'light' processing. - Ensemble Member: Represents one of five ensemble members, trained with a specific seed (1051).
- Training Details: Trained for 3 epochs, completing 1620 steps, with
bf16precision and gradient checkpointing enabled.
Evaluation Metrics (on small subsets)
Evaluation was conducted on small, fixed subsets, not full benchmarks. For this specific member (seed 1051) at the final training step:
- AmbigQA (128) Accuracy: 0.0859
- AmbigQA (128) AlignScore: 0.1729
- IFEval (64) Prompt-level Strict Accuracy: 0.5781
- MMLU (228) Accuracy: 0.5526
Use Cases
This model is primarily suited for research and development in question answering, particularly for tasks that involve resolving or responding to ambiguous queries. Its fine-tuning on AmbigQA suggests a specialization in understanding and processing questions that may have multiple interpretations or require clarification.