Model Overview

The stukenov/sozkz-core-qwen-500m-kk-instruct-v1 is a 447 million parameter instruction-tuned model based on the Qwen2 Causal LM architecture, developed by Saken Tukenov. It is derived from the sozkz-core-qwen-500m-kk-base-v1 model through LoRA SFT (Supervised Fine-Tuning) using approximately 4,882 Alpaca-style instruction pairs generated by Qwen3.5-122B.

Key Capabilities

• Kazakh Language Instruction Following: Specifically trained to understand and respond to instructions exclusively in Kazakh.
• Structured Output: Capable of generating formatted responses, including markdown and numbered lists, as demonstrated in its generation examples.
• Dialogue and Q&A: Optimized for interactive use cases such as question answering and conversational agents in Kazakh.
• Efficient Fine-tuning: Utilizes LoRA with a small trainable parameter count (7.3% of total) for efficient adaptation.

Good For

• Kazakh-specific Applications: Ideal for developers building applications that require instruction-following and text generation in the Kazakh language.
• Educational Tools: Can be used to create interactive learning tools or Q&A systems for Kazakh speakers.
• Content Generation: Suitable for generating structured content, summaries, or responses based on Kazakh prompts.

Limitations

• Kazakh Only: The model is exclusively trained on Kazakh texts and performs poorly with other languages or technical topics like code/mathematics.
• Limited Knowledge Depth: Due to a relatively small SFT dataset (4,882 pairs), its knowledge depth is constrained by the base model.
• Hallucinations: Like other small language models, it may generate plausible but incorrect information, requiring verification.
• No Safety Fine-tuning: Lacks specific safety training and may produce inappropriate content.
• Repetition Penalty Required: Requires a repetition_penalty of at least 1.1 during generation to prevent repetitive outputs.