trnqphu/deepseek-r1-4b
trnqphu/deepseek-r1-4b is a 1.5 billion parameter language model fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, leveraging a 32768 token context length. This model was trained using Supervised Fine-Tuning (SFT) with the TRL framework. It is designed for general text generation tasks, building upon the DeepSeek-R1 architecture.
Loading preview...
Model Overview
trnqphu/deepseek-r1-4b is a 1.5 billion parameter language model, fine-tuned from the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B base model. It utilizes a substantial context length of 32768 tokens, making it suitable for processing longer inputs and generating coherent, extended responses. The model's training involved Supervised Fine-Tuning (SFT) using the TRL framework, specifically version 0.16.1, with Transformers 4.51.3 and PyTorch 2.6.0.
Key Capabilities
- General Text Generation: Capable of generating human-like text based on given prompts.
- Long Context Understanding: Benefits from its 32768 token context window, allowing for more comprehensive understanding and generation in longer conversations or documents.
- Fine-tuned Performance: Leverages SFT to enhance its performance for various language tasks.
Use Cases
This model is well-suited for applications requiring:
- Conversational AI: Engaging in extended dialogues.
- Content Creation: Generating articles, stories, or other forms of written content.
- Question Answering: Providing detailed answers based on extensive context.
Developers can quickly integrate and experiment with this model using the transformers library, as demonstrated in the provided quick start example.