myyycroft/Gemma-4-E2B-AmbigQA-full-member-4

Model Overview

This model, myyycroft/Gemma-4-E2B-AmbigQA-full-member-4, is a 5.1 billion parameter language model derived from Google's gemma-4-E2B-it architecture. It represents the fourth member (seed 4078) of a larger ensemble, specifically fine-tuned on the sewon/ambig_qa dataset, which focuses on ambiguous question answering. The training involved 3 epochs, utilizing a full adaptation method and a maximum sequence length of 512 tokens.

Key Characteristics

• Base Model: Fine-tuned from google/gemma-4-E2B-it.
• Dataset: Specialized fine-tuning on the AmbigQA dataset (sewon/ambig_qa, config_name=light).
• Ensemble Member: This is member 4 of a 5-member ensemble, trained with a specific seed (4078).
• Training Details: Trained for 3 epochs with a learning rate of 2.0e-05 and a per-device batch size of 4, accumulating gradients over 8 steps.
• Context Length: The base model supports a context length of 32768 tokens.

Evaluation Notes

Evaluation metrics provided are based on small, fixed subsets rather than full benchmarks. For instance, AmbigQA was evaluated on 128 examples, IFEval on 64, and MMLU on 228. The reported metrics for this specific member include an AmbigQA (128) accuracy of 0.0938 and an MMLU (228) accuracy of 0.5351 at the final training step (1620 steps, 3 epochs).

Use Cases

This model is primarily suited for research and development in ambiguous question answering due to its specific fine-tuning on the AmbigQA dataset. Its ensemble nature suggests potential for robust performance when combined with other members, particularly in tasks requiring nuanced understanding of questions.