Model Overview

deepkick/qwen3-4b-structured-sft-lora-v07-merged is a 4 billion parameter language model, fully merged from its base model and a LoRA fine-tuning. It is built upon the Qwen/Qwen3-4B-Instruct-2507 architecture, indicating a foundation in the Qwen3 series known for its strong performance.

Key Capabilities & Training

This model has been fine-tuned using QLoRA (4-bit), a memory-efficient method for adapting large language models. The training focused on a specific dataset, u-10bei/structured_data_with_cot_dataset_512_v2, which comprises 3933 entries. A notable aspect of its training configuration is the MASK_COT: 1 setting, which means Chain-of-Thought (CoT) reasoning was preserved and masked during loss calculation, suggesting an emphasis on maintaining and utilizing reasoning steps for structured outputs.

Differentiators

The v07 iteration specifically increased the SFT Learning Rate (LR) from 2e-6 to 2e-5 compared to v03, indicating an aggressive learning approach to better capture the nuances of structured data. Other parameters, such as LoRA r=64 and alpha=128, and 2 training epochs, remained consistent. This model is designed for tasks that benefit from structured data processing and explicit reasoning paths.