The cyumizou/qwen3-4b-structured-output-merged-stage-a model is a 4 billion parameter Qwen3-based instruction-tuned language model, derived from Qwen/Qwen3-4B-Instruct-2507. It is specifically optimized to improve structured output reliability across formats like JSON, YAML, XML, TOML, and CSV by reducing non-structured preambles. This model is designed as a stable foundation for applications requiring clean, parser-friendly structured data generation.
Loading preview...
Overview
This model, cyumizou/qwen3-4b-structured-output-merged-stage-a, is a 4 billion parameter language model based on the Qwen3 architecture. It is a merged (fully materialized) model, meaning a LoRA adapter was trained on the base model Qwen/Qwen3-4B-Instruct-2507 and then integrated directly into its weights. This makes it directly loadable with AutoModelForCausalLM.from_pretrained() without needing separate adapter loading.
Key Capabilities
- Enhanced Structured Output: The primary goal of this model (StageA) is to stabilize and improve the reliability of structured output generation.
- Reduced Preambles: It is specifically trained to minimize non-structured preambles (e.g., "Here/Sure") and code-fences (```json) that can break parsers.
- Reliable Format Generation: Aims to consistently output only the required structured format (JSON, YAML, XML, TOML, CSV).
Training Details
The model was trained using QLoRA (4-bit) with a LoRA adapter (r=8, alpha=16) on the u-10bei/structured_data_with_cot_dataset_512_v2 dataset. Training involved one epoch with a learning rate of 2e-05 and a maximum sequence length of 1024. The training objective focused on applying loss to the full assistant output to suppress unwanted preambles.
When to Use This Model
This model is ideal for applications where clean, parser-friendly structured output is critical. It serves as a robust starting point for further fine-tuning (StageB) aimed at specific structured output challenges, such as mitigating TOML failure patterns, without reintroducing chatty preambles.