modrill/qwen3-4b-think-s1-ep23-full-sft
The modrill/qwen3-4b-think-s1-ep23-full-sft model is a 4 billion parameter Qwen3-based causal language model developed by modrill, fine-tuned for 'think-style' reasoning. It was created through full supervised fine-tuning (SFT) on the ocr_think_50k dataset, continuing from an internal 'think_s1' run. This model specializes in reasoning tasks, leveraging its specific training on a dataset designed to enhance logical thought processes.
Loading preview...
modrill/qwen3-4b-think-s1-ep23-full-sft: Think-Style Reasoning Model
This model is a 4 billion parameter variant of the Qwen3 architecture, developed by modrill. It has undergone full supervised fine-tuning (SFT), meaning all model weights were updated, distinguishing it from LoRA or QLoRA approaches. The fine-tuning process utilized DeepSpeed ZeRO-3 across 4 GPUs.
Key Characteristics & Training
- Base Model: Qwen/Qwen3-4B-Base.
- Fine-tuning Dataset:
ocr_think_50k, specifically chosen to imbue 'think-style' reasoning capabilities. - Training Continuation: This checkpoint represents 'stage 1 episode 23', continuing from an earlier internal run (
checkpoint-302) and further fine-tuned for two additional epochs on theocr_think_50kdataset. - Template: Uses the Qwen3 chat template, optimized for its reasoning focus.
- Training Details: Achieved a
train_lossof 0.5416 over 604 training steps, with a learning rate of 5e-5 and an effective batch size of 64.
Primary Use Case
This model is particularly suited for applications requiring 'think-style' reasoning, where the ability to process and generate logical thought sequences is crucial. Its specialized training on the ocr_think_50k dataset makes it a strong candidate for tasks that benefit from explicit reasoning steps, differentiating it from models fine-tuned for general instruction following or creative writing.