Model Overview

nicolay-r/qwen25-05b-multiclinsum-distil is a 0.5 billion parameter decoder-based model, distilled from Qwen/Qwen2.5-0.5B-Instruct. It has been specifically fine-tuned for multilingual clinical text summarization.

Key Capabilities

• Multilingual Summarization: Supports summarization of clinical texts in English, French, Portuguese, and Spanish.
• Distillation Approach: Utilizes a unique distillation process where rationales for summaries were first inferred by the larger Qwen/Qwen2.5-72B-Instruct model.
• Clinical Domain Focus: Trained on the MultiClinSum dataset, making it specialized for medical and clinical contexts.
• BioASQ Submission: Results from this model were used for a submission to the BioASQ-2025 Workshop.

Training Details

The training procedure involved preparing rationales using Qwen/Qwen2.5-72B-Instruct via the OpenRouter API, followed by fine-tuning on the MultiClinSum dataset. The fine-tuning process for 3 epochs took approximately 1 hour on a Google Colab A100 GPU. Evaluation was conducted using the ROUGE score on a test set of 20 documents across all supported languages.

Good for

• Clinical NLP: Ideal for applications requiring automated summarization of clinical documents.
• Multilingual Healthcare: Useful in scenarios where clinical text summarization is needed across multiple European languages.
• Resource-Efficient Deployment: As a 0.5B parameter model, it offers a more efficient solution for summarization tasks compared to larger models, while retaining specialized capabilities.