Lansechen/Qwen2.5-7B-Open-R1-Distill
Lansechen/Qwen2.5-7B-Open-R1-Distill is a 7.6 billion parameter language model, fine-tuned from Qwen/Qwen2.5-7B-Instruct. This model was trained using SFT on the HuggingFaceH4/Bespoke-Stratos-17k dataset, leveraging TRL for its training procedure. It is designed for general text generation tasks, offering a 32K context length for processing longer inputs.
Loading preview...
Overview
Lansechen/Qwen2.5-7B-Open-R1-Distill is a 7.6 billion parameter language model derived from the Qwen2.5-7B-Instruct architecture. It has been specifically fine-tuned using Supervised Fine-Tuning (SFT) on the HuggingFaceH4/Bespoke-Stratos-17k dataset with the TRL library.
Key Capabilities
- Instruction Following: As a fine-tuned instruct model, it is optimized to follow user instructions for various text generation tasks.
- Text Generation: Capable of generating coherent and contextually relevant text based on prompts.
- Context Handling: Supports a substantial context length of 32,768 tokens, allowing for processing and generating longer sequences.
Training Details
The model's training procedure involved SFT, utilizing specific versions of popular machine learning frameworks:
- TRL: 0.15.0.dev0
- Transformers: 4.49.0.dev0
- Pytorch: 2.5.1+cu121
- Datasets: 3.2.0
- Tokenizers: 0.21.0
Good For
- Developers looking for a Qwen2.5-7B-Instruct variant fine-tuned on a specific dataset.
- Applications requiring general-purpose text generation with good instruction-following capabilities.
- Use cases benefiting from a 32K token context window for more extensive input processing.