Model Overview

JuntaTakahashi/qwen3-4b-structured-sft-lora is a 4 billion parameter language model built upon the Qwen3 architecture. It is a fine-tuned variant of unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit, specifically enhanced through Supervised Fine-Tuning (SFT) using the TRL library.

Key Capabilities

• Instruction Following: Inherits and refines instruction-following capabilities from its base model.
• Text Generation: Optimized for generating coherent and contextually relevant text based on user prompts.
• Extended Context: Benefits from a substantial 40960-token context window, allowing for processing and generating longer sequences of text.

Training Details

The model underwent Supervised Fine-Tuning (SFT) as its primary training procedure. The training process utilized several key frameworks:

• PEFT: 0.13.2
• TRL: 0.24.0
• Transformers: 4.56.2
• Pytorch: 2.9.1+cu126
• Datasets: 4.3.0
• Tokenizers: 0.22.2

This fine-tuning approach aims to adapt the base Qwen3 model for improved performance on specific tasks, making it suitable for various text-based applications requiring robust generation and understanding.