CraneAILabs/edu-ganda-gemma-e2b-v5

edu-ganda-gemma-e2b-v5: Luganda Primary-Education Assistant

This model, developed by Crane AI Labs, is an experimental research checkpoint designed as their current best primary-education assistant for the Luganda language. It is a SLERP merge (alpha 0.5) of two experimental Gemma-4-E2B checkpoints:

• a065_polished_v2: Provided strong Luganda content but weaker instruction-following.
• ganda-e2b-v4: Offered robust instruction-following but weaker Luganda content.

The merge successfully combines the strengths of both, outperforming its base models and other merge attempts.

Key Capabilities & Performance

This 5.1 billion parameter model demonstrates significant improvements in several areas:

• Instruction-following (IF-adherence): Achieves 65.9%, a substantial increase from the V2 base's 40.9%.
• Judge task quality: Scores 4.6/10, up from 3.76/10.
• Judge Luganda quality: Rated 7.84/10, an improvement over 7.52/10.
• Math (mn100): Performs at 72%.
• Repetition: Shows lower repetition compared to the base model.

Limitations

• FLORES lug→en translation: Exhibits a decrease in chrF score compared to the base model.
• Guarded followed%: Caps around 44%.

Recommended Usage

For optimal performance, Crane AI Labs recommends specific serving settings:

• Decoding: Greedy with repetition_penalty=1.0 (off) and no_repeat_ngram_size=4 to prevent loops and achieve the best balance in math and instruction-following.
• EOS Tokens: eos_token_id=[<eos>, <end_of_turn>].
• Prompting: Prepend <bos> (id 2) before the chat-templated prompt.

This model is an experimental checkpoint, and while it offers best-effort Luganda content, review is advised for high-stakes applications.