hmdmahdavi/s1-thinking-distill-deepseek-cot
The hmdmahdavi/s1-thinking-distill-deepseek-cot is a 4 billion parameter language model, fine-tuned from Qwen/Qwen3-4B-Instruct-2507. This model was trained using the TRL framework with SFT, and is designed for general text generation tasks. It leverages a 40960 token context length, making it suitable for processing longer inputs and generating coherent, extended responses.
Loading preview...
Model Overview
The hmdmahdavi/s1-thinking-distill-deepseek-cot is a 4 billion parameter language model, specifically a fine-tuned variant of the Qwen/Qwen3-4B-Instruct-2507 architecture. This model has been developed using the TRL (Transformer Reinforcement Learning) framework, employing Supervised Fine-Tuning (SFT) as its training procedure.
Key Characteristics
- Base Model: Fine-tuned from Qwen/Qwen3-4B-Instruct-2507.
- Parameter Count: 4 billion parameters, offering a balance between performance and computational efficiency.
- Context Length: Supports a substantial context window of 40960 tokens, enabling it to handle and generate longer, more complex texts while maintaining coherence.
- Training Framework: Utilizes the TRL library, a robust tool for training transformer models.
Use Cases
This model is well-suited for various text generation tasks, particularly those benefiting from its large context window. Developers can integrate it into applications requiring:
- General text generation: Creating diverse and coherent textual content.
- Question Answering: Generating detailed answers based on provided context.
- Conversational AI: Developing chatbots or interactive agents capable of longer dialogues.
Training Details
The model's training process involved SFT, leveraging specific versions of key frameworks:
- TRL: 0.12.0
- Transformers: 4.57.3
- Pytorch: 2.5.1
- Datasets: 4.4.1
- Tokenizers: 0.22.1
Further details on the training run can be visualized via Weights & Biases.