azuki-digital/qwen3-4b-struct-lora-v4-merged

Model Overview

This model, azuki-digital/qwen3-4b-struct-lora-v4-merged, is a 4 billion parameter language model based on the Qwen3 architecture, specifically derived from Qwen/Qwen3-4B-Instruct-2507. It is a LoRA-merged base model that has been fine-tuned to specialize in structured output generation.

Key Capabilities & Features

• Structured Output Specialization: The model was created by fine-tuning Qwen3-4B-Instruct with a LoRA adapter specifically for structured outputs, then merging these weights into the base model.
• Foundation for Further Fine-tuning: It acts as a specialized foundation model, providing a significantly improved prior for subsequent LoRA training focused on structured output tasks. This helps stabilize later Supervised Fine-Tuning (SFT) and improves convergence.
• Standalone Model: Unlike a LoRA adapter, this is a fully merged, standalone model ready for direct use without additional adapters.
• Training Configuration: Trained using LoRA SFT (bf16, no quantization) with a maximum sequence length of 4096, 1 epoch, and a learning rate of 3e-5. It utilized the u-10bei/structured_data_with_cot_dataset_512_v2 dataset with Mask CoT.

Ideal Use Cases

• Starting Point for Structured Data Tasks: Excellent for developers looking to fine-tune a model for tasks like JSON generation, data extraction, or other structured output formats.
• Improving Fine-tuning Efficiency: Provides a more stable and effective base for new LoRA training runs that require strong structured output capabilities, potentially leading to faster convergence and better performance.