nlee-208/limo_S-dsr1b_T-qwq_10
The nlee-208/limo_S-dsr1b_T-qwq_10 is a 1.5 billion parameter language model fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. This model was trained using SFT with the TRL framework, offering a 32768 token context length. It is designed for general text generation tasks, leveraging its fine-tuned base for improved performance in conversational or creative prompts.
Loading preview...
Model Overview
The nlee-208/limo_S-dsr1b_T-qwq_10 is a 1.5 billion parameter language model, fine-tuned from the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B base model. This fine-tuning process utilized Supervised Fine-Tuning (SFT) with the Hugging Face TRL library, indicating an optimization for specific instruction-following or response generation tasks. The model supports a substantial context length of 32768 tokens, allowing it to process and generate longer sequences of text while maintaining coherence.
Key Capabilities
- Text Generation: Capable of generating human-like text based on given prompts.
- Instruction Following: Optimized through SFT for better adherence to user instructions.
- Extended Context: Benefits from a 32768 token context window, suitable for tasks requiring extensive input or generating detailed outputs.
Training Details
The model was trained using the TRL (Transformer Reinforcement Learning) framework, specifically employing SFT. The training environment included TRL version 0.18.1, Transformers 4.52.4, Pytorch 2.7.1, Datasets 4.0.0, and Tokenizers 0.21.1. This setup suggests a focus on enhancing the model's ability to generate relevant and coherent responses.
Good For
- General text generation applications.
- Conversational AI and chatbot development.
- Tasks requiring processing and generation of long text sequences.