iromu/Qwen2.5-1.5B-k3
iromu/Qwen2.5-1.5B-k3 is a 1.5 billion parameter language model fine-tuned from unsloth/Qwen2.5-1.5B-Instruct. This model specializes in structured tool/function calling and agent-style multi-step interactions, leveraging distillation from Kimi-K3 data. It is optimized for specific reasoning behaviors rather than general-purpose language generation, offering a compact solution for complex automation tasks. The model has a context length of 32768 tokens.
Loading preview...
Model Overview
iromu/Qwen2.5-1.5B-k3 is a 1.5 billion parameter language model, fine-tuned from unsloth/Qwen2.5-1.5B-Instruct. This model has been specifically trained using LoRA on the r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation dataset, focusing on distilling reasoning behaviors for advanced interaction patterns.
Key Capabilities
- Structured Tool/Function Calling: Designed to accurately interpret and generate structured calls for external tools and functions.
- Agent-Style Interactions: Excels in multi-step conversational flows, enabling more complex automated agents.
- Distilled Reasoning: Incorporates reasoning patterns derived from the Kimi-K3 data mix, enhancing its ability to handle intricate logical sequences.
Training Details
The model was trained with LoRA (dimension 16, alpha 16, dropout 0.05) targeting key attention and feed-forward layers. Training utilized a maximum sequence length of 4096 tokens and a learning rate of 2e-5 over 1 epoch with BF16 mixed precision.
Intended Use
This model is ideal for applications requiring precise tool orchestration and sophisticated agentic behavior. It is not intended as a general-purpose replacement for larger Qwen models but rather as a specialized solution for specific automation and interaction challenges.