myyycroft/Gemma-4-E2B-AmbigQA-full-member-0

Model Overview

This model, myyycroft/Gemma-4-E2B-AmbigQA-full-member-0, is a 5.1 billion parameter language model derived from Google's gemma-4-E2B-it base. It represents one member of a 5-member ensemble, specifically fine-tuned on the sewon/ambig_qa dataset to enhance its capabilities in answering ambiguous questions. The training involved 3 epochs with a learning rate of 2.0e-05 and a maximum sequence length of 512 tokens.

Key Capabilities

• AmbigQA Specialization: Fine-tuned specifically on the AmbigQA dataset, indicating a focus on handling questions that may have multiple valid answers or require clarification.
• Gemma-4-E2B Architecture: Built upon the google/gemma-4-E2B-it model, suggesting a foundation in efficient and capable language understanding.
• Ensemble Member: This model is part of a larger ensemble, implying potential for improved robustness and performance when combined with other members.

Evaluation Metrics (on small subsets)

Evaluations were conducted on small, fixed subsets, not full benchmarks. For this specific member (seed 42) at the final training step:

• AmbigQA (128) accuracy: 0.0781
• AmbigQA (128) AlignScore: 0.1918
• IFEval (64) prompt_level_strict_accuracy: 0.6719
• MMLU (228) accuracy: 0.5526

Good For

• Ambiguous Question Answering: Ideal for tasks where questions might be open to interpretation or require generating multiple possible answers.
• Research in Ensemble Methods: Useful for researchers exploring the performance and characteristics of individual members within a larger model ensemble.
• Fine-tuning Experiments: Provides a base for further fine-tuning on related question-answering or natural language understanding tasks.