Model Overview
This model, azuki-digital/qwen3-4b-struct-lora-v4-merged, is a 4 billion parameter language model based on the Qwen3 architecture, specifically derived from Qwen/Qwen3-4B-Instruct-2507. It is a LoRA-merged base model that has been fine-tuned to specialize in structured output generation.
Key Capabilities & Features
- Structured Output Specialization: The model was created by fine-tuning
Qwen3-4B-Instruct with a LoRA adapter specifically for structured outputs, then merging these weights into the base model. - Foundation for Further Fine-tuning: It acts as a specialized foundation model, providing a significantly improved prior for subsequent LoRA training focused on structured output tasks. This helps stabilize later Supervised Fine-Tuning (SFT) and improves convergence.
- Standalone Model: Unlike a LoRA adapter, this is a fully merged, standalone model ready for direct use without additional adapters.
- Training Configuration: Trained using LoRA SFT (bf16, no quantization) with a maximum sequence length of 4096, 1 epoch, and a learning rate of 3e-5. It utilized the
u-10bei/structured_data_with_cot_dataset_512_v2 dataset with Mask CoT.
Ideal Use Cases
- Starting Point for Structured Data Tasks: Excellent for developers looking to fine-tune a model for tasks like JSON generation, data extraction, or other structured output formats.
- Improving Fine-tuning Efficiency: Provides a more stable and effective base for new LoRA training runs that require strong structured output capabilities, potentially leading to faster convergence and better performance.