JuntaTakahashi/qwen3-4b-structured-sft-lora
JuntaTakahashi/qwen3-4b-structured-sft-lora is a 4 billion parameter Qwen3-based language model, fine-tuned using Supervised Fine-Tuning (SFT) with TRL. This model is specifically adapted from unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit. It is designed for general text generation tasks, leveraging its 40960-token context length for processing extensive inputs.
Loading preview...
Model Overview
JuntaTakahashi/qwen3-4b-structured-sft-lora is a 4 billion parameter language model built upon the Qwen3 architecture. It is a fine-tuned variant of unsloth/qwen3-4b-instruct-2507-unsloth-bnb-4bit, specifically enhanced through Supervised Fine-Tuning (SFT) using the TRL library.
Key Capabilities
- Instruction Following: Inherits and refines instruction-following capabilities from its base model.
- Text Generation: Optimized for generating coherent and contextually relevant text based on user prompts.
- Extended Context: Benefits from a substantial 40960-token context window, allowing for processing and generating longer sequences of text.
Training Details
The model underwent Supervised Fine-Tuning (SFT) as its primary training procedure. The training process utilized several key frameworks:
- PEFT: 0.13.2
- TRL: 0.24.0
- Transformers: 4.56.2
- Pytorch: 2.9.1+cu126
- Datasets: 4.3.0
- Tokenizers: 0.22.2
This fine-tuning approach aims to adapt the base Qwen3 model for improved performance on specific tasks, making it suitable for various text-based applications requiring robust generation and understanding.