deahmed/Qwen3.5-2B-da-task
The deahmed/Qwen3.5-2B-da-task is a 2.3 billion parameter Qwen3.5-2B model fine-tuned by deahmed specifically for Danish language tasks. It leverages a LoRA adapter (rank 128) and incorporates training data from the EuroEval Danish benchmark, including 8,422 benchmark examples and 5,269 synthetic Danish multiple-choice questions. This model demonstrates significant improvements on various Danish language benchmarks, making it particularly effective for Danish-specific natural language processing applications.
Loading preview...
Overview
This model, deahmed/Qwen3.5-2B-da-task, is a specialized fine-tune of the 2.3 billion parameter Qwen/Qwen3.5-2B base model, optimized for Danish language tasks. It supersedes the earlier deahmed/Qwen3.5-2B-da-sft by incorporating benchmark data from the EuroEval Danish train splits. The fine-tuning process involved folding in a LoRA adapter (rank 128) at 0.3 strength, using a WiSE-FT style interpolation.
Key Capabilities & Performance
The model shows substantial improvements across various Danish benchmarks, as measured by EuroEval. It achieves a mean improvement of +4.11 over the base model, with notable gains such as +12.02 on DANSK (NER) micro-F1 and +4.55 on MultiWikiQA-da F1. This performance is partly attributed to its training mixture, which includes 8,422 benchmark train-split examples and 5,269 synthetic Danish multiple-choice questions, formatted to match EuroEval's evaluation style. The training also included 15,098 Danish instruction pairs and 5,000 English examples from OpenHermes-2.5 to mitigate catastrophic forgetting.
Limitations
It's important to note that while the model excels in Danish tasks, especially those measured by EuroEval, its scores may overstate its general open-ended Danish ability due to the benchmark-shaped training data. English ability is largely retained but was not a primary optimization goal. The base model's vision components are carried over but remain untrained and untested in this fine-tune.