nuinashco/gemma-3-1b-it-xlsum-ua-sft
The nuinashco/gemma-3-1b-it-xlsum-ua-sft model is a 1.1 billion parameter Gemma-based instruction-tuned language model, fine-tuned by nuinashco for Ukrainian news summarization. It leverages the unsloth/gemma-3-1b-it base model and was trained using Unsloth and TRL SFT on the Ukrainian split of the csebuetnlp/xlsum dataset. This model is specifically optimized for generating concise summaries of Ukrainian text, achieving a ROUGE-L score of 22.23 on its evaluation set.
Loading preview...
Model Overview
The nuinashco/gemma-3-1b-it-xlsum-ua-sft is a specialized 1.1 billion parameter language model derived from the unsloth/gemma-3-1b-it base model. It has been fully fine-tuned by nuinashco using Unsloth and TRL SFT specifically for the task of Ukrainian news summarization. The training utilized the Ukrainian split of the csebuetnlp/xlsum dataset, which includes safety filtering.
Key Capabilities
- Ukrainian News Summarization: The model is explicitly trained to generate summaries of news articles in Ukrainian.
- Optimized Training: Fine-tuned with Unsloth and TRL SFT, employing a full SFT approach (no LoRA) over approximately 1.48 epochs.
- Performance: Achieved a best evaluation ROUGE-L score of 22.23.
- Context Length: Supports a maximum sequence length of 3072 tokens during training.
Limitations and Considerations
- Language Specificity: Performance is guaranteed only for Ukrainian text; usage with other languages is not defined.
- Bias Inheritance: Inherits potential biases from its base model and the training corpus.
- Factual Accuracy: Summaries may occasionally contain factual inaccuracies, necessitating verification against original sources.
When to Use This Model
This model is ideal for applications requiring efficient and concise summarization of Ukrainian news content. Its focused training makes it a strong candidate for tasks where understanding and condensing Ukrainian text is paramount.