kaufkino/LFM2.5-1.2B-Instruct-T-Wix-ru
The kaufkino/LFM2.5-1.2B-Instruct-T-Wix-ru is a 1.2 billion parameter LFM2.5-Instruct model, fine-tuned by kaufkino, specifically adapted for the Russian language. It was trained using LoRA SFT on a subsample of the T-Wix dataset, enabling fluent and structured Russian text generation, which the base model lacked. This model excels at producing coherent Russian responses and maintaining tool-calling formats, making it suitable for Russian-language conversational AI applications where factual accuracy can be supplemented with RAG.
Loading preview...
Overview
kaufkino/LFM2.5-1.2B-Instruct-T-Wix-ru is an experimental, Russian-language adaptation of the LiquidAI/LFM2.5-1.2B-Instruct model. This 1.2 billion parameter model was fine-tuned using LoRA SFT on a 200,000-dialogue subsample of the Russian T-Wix dataset. The primary goal was to enable the base model, which natively supports languages like English, Arabic, and Chinese but not Russian, to generate fluent and structured Russian text.
Key Capabilities & Improvements
- Fluent Russian Generation: Significantly improved Russian output, transforming grammatically broken text into coherent and well-formed responses.
- Russian Mathematical/Reasoning: The model now provides mathematical and reasoning answers in Russian, whereas the base model often reverted to English.
- Tool Call Format Preservation: Largely maintains the original tool-calling format (
<|tool_call_start|>[func(args)]<|tool_call_end|>).
Known Limitations
- Factual Hallucinations in Russian: While the model possesses factual knowledge, it struggles to transfer this accurately from English to Russian generation. For Russian-language factual queries, a Retrieval-Augmented Generation (RAG) system is recommended.
- Weak EN→RU Translation: English to Russian translation remains functional but prone to errors.
- Unreliable Arithmetic: Arithmetic operations are not consistently accurate.
Training Details
The model was trained for 1 epoch using LoRA SFT (r=32, alpha=64) with a context length of 8192 tokens and a batch size of 32. The training utilized adamw_8bit optimizer with a learning rate of 1e-4 and bf16 precision. The final training loss was approximately 1.15.
Usage
The repository provides the merged model in bf16 safetensors format for transformers and vLLM, a separate LoRA adapter, and GGUF versions (F16, Q8_0) for llama.cpp and Ollama.