ajtakto/Qwen3SK: The First Slovak Instruction Model

ajtakto/Qwen3SK is a significant development as the first instruction-tuned large language model specifically trained for the Slovak language. Developed collaboratively by Peter Bednár (FEI TUKE), Marek Dobeš (Slovak Academy of Sciences), and Radovan Garabík (Ľudovít Štúr Institute of Linguistics), this model addresses the critical need for high-quality language processing in Slovak.

Key Capabilities & Training

• Foundation Model: Built upon the robust Qwen3-14B-Instruct model, which features 14 billion parameters.
• Slovak-Centric Fine-tuning: Underwent full-parameter fine-tuning using a comprehensive suite of Slovak datasets:
  • "Araneum Slovacum VII Maximum" web corpus.
  • Pre-processed data from the Dictionary of the Slovak Language.
  • Data from the Encyclopaedia Beliana.
• Computational Resources: Training was conducted on the Leonardo and Perun supercomputers, secured through national project proposals.
• Language Focus: Designed to be a useful and precise assistant, responding exclusively in Slovak and avoiding Czechisms, with specific formatting for quotes.

Limitations

• The model may occasionally exhibit repetitive behavior in its responses.
• Hallucination rates are comparable to other models of similar scale.
• No output moderation or safety filtering techniques have been applied, which users should consider for sensitive applications.

Ideal Use Cases

This model is particularly well-suited for applications requiring:

• Slovak language generation: Creating natural and contextually relevant text in Slovak.
• Instruction following in Slovak: Responding to prompts and commands given in Slovak.
• Research and development in Slovak NLP, leveraging its specialized training on national linguistic resources.