TSjB/QM-4B

Overview

TSjB/QM-4B is a 3.97 billion parameter language model built upon the Qwen3-4B-Instruct-2507 architecture. Its primary distinction is the extensive fine-tuning for the Qarachay-Malqar language, making it a specialized tool for this specific linguistic context. The model underwent a multi-stage training process, including tokenizer expansion and full fine-tuning of all layers.

Key Capabilities & Features

• Qarachay-Malqar Language Support: Features an expanded tokenizer with significantly increased representation for Qarachay-Malqar tokens (from 1.78 to 5.38 symbols/tokens compared to the original tokenizer).
• Multilingual: Supports Qarachay-Malqar, Russian, and English, inheriting capabilities from its Qwen3 base.
• Training Methodology: Involved tokenizer expansion, embeddings-only training (3 epochs), and a full fine-tune of all model layers (1 epoch).
• Parameter Count: The full fine-tune involved 100% of the model's 3.97 billion parameters.

Limitations & Recommendations

• Training Data: The model was fine-tuned on text data (continued pretraining) rather than dialogues, which may affect conversational performance.
• Language Switching: May exhibit language switching within a single response.
• Instruction Following: Additional instruction tuning is recommended for improved instruction adherence.

Good For

• Applications requiring Qarachay-Malqar language generation and understanding.
• Developers looking for a base model to further fine-tune for specific Qarachay-Malqar tasks or dialogue systems.
• Research into low-resource language modeling, particularly for Qarachay-Malqar.