modrill/qwen3-4b-think-baseline-full-sft
The modrill/qwen3-4b-think-baseline-full-sft is a 4 billion parameter Qwen3-based causal language model developed by modrill. It is a full-parameter supervised fine-tune of Qwen/Qwen3-4B-Base, specifically optimized for 'thinking mode' with a training cutoff length of 24576 tokens. This model is designed for applications requiring enhanced reasoning capabilities through its enabled thinking mode.
Loading preview...
Overview
The modrill/qwen3-4b-think-baseline-full-sft is a 4 billion parameter language model derived from the Qwen3-4B-Base architecture. It has undergone full-parameter supervised fine-tuning (SFT), rather than LoRA, on the think_all dataset. A key characteristic of this model is its enabled 'thinking mode', which is central to its design and intended use.
Key Details
- Base Model: Qwen/Qwen3-4B-Base
- Fine-tuning Method: Full SFT using DeepSpeed ZeRO-3
- Dataset:
think_all - Thinking Mode: Enabled (
enable_thinking=true) - Training Cutoff Length: 24576 tokens
- Epochs: 2
- License: Apache 2.0, consistent with the Qwen3 base model.
Usage and Recommendations
This model is available as native full fine-tuned weights, provided as a single model.safetensors file. For optimal performance, users should set enable_thinking=true in their chat template during inference. A recommended max_tokens value of 24576 is suggested for inference. The model can be integrated using HuggingFace Transformers or vLLM, with code examples provided for both.