nlee-208/limo_S-dsr1b_T-q32b_10
The nlee-208/limo_S-dsr1b_T-q32b_10 model is a 1.5 billion parameter language model fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B. Developed by nlee-208, this model was trained using Supervised Fine-Tuning (SFT) with the TRL framework. It is designed for general text generation tasks, leveraging its 32768 token context length for processing longer inputs.
Loading preview...
Model Overview
The nlee-208/limo_S-dsr1b_T-q32b_10 is a 1.5 billion parameter language model, fine-tuned from the deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B base model. This model was developed by nlee-208 and underwent Supervised Fine-Tuning (SFT) using the TRL framework.
Key Capabilities
- Text Generation: Capable of generating human-like text based on provided prompts.
- Extended Context: Features a 32768 token context length, allowing it to process and generate longer sequences of text.
- Fine-tuned Performance: Benefits from SFT, which typically enhances performance on specific tasks or improves adherence to instructions.
Training Details
The model's training procedure utilized the TRL library (version 0.19.1) in conjunction with Transformers (4.53.3), Pytorch (2.7.1), Datasets (4.0.0), and Tokenizers (0.21.2). The training process was tracked and can be visualized via Weights & Biases.
Good For
- General Text Generation: Suitable for various applications requiring text completion or response generation.
- Instruction Following: As an SFT model, it is expected to follow user instructions more effectively than its base model.
- Exploration: Developers can use this model as a foundation for further fine-tuning or specific application development.