Overview
Mindie/Qwen3-4b-kss-style-tuning is an instruction-tuned model built upon the Qwen3-4B base model, utilizing LoRA (Low-Rank Adaptation) for fine-tuning. Its primary purpose is to generate structured summaries in a specific 'Subject: Keywords: Summary:' format while preserving the underlying general knowledge and reasoning capabilities of the base model.
Key Capabilities
- Structured Summary Generation: Consistently produces summaries adhering to the KSS-style format.
- Instruction Following: Demonstrates robust instruction-following behavior for format enforcement.
- Knowledge Preservation: Evaluation shows no significant degradation in general knowledge (MMLU score of 0.724 vs. 0.725 for the base model).
- Robustness: Performs consistently across both short and long input texts.
Training Details
The model was fine-tuned on a dataset of 596 samples, including both instruction-formatted and free-form data. This contrastive approach helped the model learn when to apply the structured format without overfitting. Data sources included GPT-generated summaries, base model-generated summaries, and a CNN article dataset. The LoRA fine-tuning method was specifically chosen to enforce output structure without compromising the model's existing knowledge.
Good For
- Applications requiring consistent, structured text output, particularly for summarization tasks.
- Use cases where maintaining general knowledge and reasoning is crucial alongside format adherence.
- Developers looking for a model that can enforce specific output schemas without significant performance trade-offs.