deahmed/gemma-4-E2B-it-danish

VISIONConcurrent Unit Cost:1Model Size:5.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 5, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The deahmed/gemma-4-E2B-it-danish model is a 5.1 billion parameter Gemma 4 E2B (instruct) variant, fine-tuned on 74,000 Danish instruction-transformation examples with a 32768 token context length. This model was developed by deahmed to test if light-touch LoRA fine-tuning on synthetic Danish data could improve performance. While it exhibits fluent Danish and strong stylistic instruction-following qualitatively, it generally regresses on 7 out of 10 EuroEval Danish datasets compared to the stock google/gemma-4-E2B-it model, making it primarily useful for studying SFT-induced regression and fine-tuning dynamics.

Loading preview...

Model Overview

This model, deahmed/gemma-4-E2B-it-danish, is a 5.1 billion parameter Gemma 4 E2B (instruct) variant, fine-tuned by deahmed using LoRA on 74,000 Danish instruction-transformation examples. The primary goal was to assess if this fine-tuning approach could enhance the base model's Danish performance. Despite qualitative observations of fluent Danish and strong stylistic instruction-following, quantitative evaluation revealed a regression in performance across most Danish benchmarks.

Key Findings & Performance

  • Qualitative Strengths: Exhibits fluent Danish and strong stylistic instruction-following.
  • Quantitative Regression: Regresses against the stock google/gemma-4-E2B-it on 7 out of 10 EuroEval Danish datasets.
  • Specific Losses: Largest performance drops were observed in grammar judgment (ScaLA), Danish idioms, and common-sense reasoning.
  • Ties: Performed comparably to the stock model on sentiment classification (AngryTweets) and Named Entity Recognition (DANSK).
  • Training Insights: Validation loss improved monotonically during training, yet actual capabilities decayed, highlighting that loss on narrow data can measure dataset obedience rather than overall quality.

Intended Use Cases

This model is not recommended for general use as the best small Danish model; for that, the stock google/gemma-4-E2B-it is advised. Instead, deahmed/gemma-4-E2B-it-danish is valuable for:

  • Studying SFT-induced regression in instruction-tuned models.
  • Research into distillation data design.
  • Analyzing fine-tuning dynamics of heavily optimized instruct checkpoints.