nlee-208/limo_S-dsr1b_T-dsr32b_10
The nlee-208/limo_S-dsr1b_T-dsr32b_10 is a 1.5 billion parameter language model, fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. This model was trained using SFT (Supervised Fine-Tuning) with TRL and features a context length of 32768 tokens. It is designed for general text generation tasks, leveraging its fine-tuned capabilities for diverse conversational prompts.
Loading preview...
Model Overview
The nlee-208/limo_S-dsr1b_T-dsr32b_10 is a 1.5 billion parameter language model, derived from the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B architecture. This model has undergone Supervised Fine-Tuning (SFT) using the TRL library, enhancing its ability to generate coherent and contextually relevant text.
Key Capabilities
- General Text Generation: Capable of generating responses to a wide array of prompts, as demonstrated by its quick start example.
- Extended Context Window: Features a notable context length of 32768 tokens, allowing it to process and generate longer, more complex sequences of text.
- Fine-tuned Performance: Benefits from SFT, which typically refines a model's understanding and generation quality for specific tasks or conversational styles.
Training Details
The model's training utilized specific versions of key frameworks:
- TRL: 0.19.1
- Transformers: 4.53.3
- Pytorch: 2.7.1
- Datasets: 4.0.0
- Tokenizers: 0.21.2
This model is suitable for developers looking for a compact yet capable language model with a substantial context window for various text-based applications.