Shellypeckie/student_qwen3_1p7b_clean4b_dolly_seq_kd
Shellypeckie/student_qwen3_1p7b_clean4b_dolly_seq_kd is a 2 billion parameter language model fine-tuned from Qwen/Qwen3-1.7B. This model was trained using Supervised Fine-Tuning (SFT) with the TRL framework. It is designed for general text generation tasks, leveraging its Qwen3 base architecture for efficient performance. The model's training process focuses on adapting the base Qwen3 model for specific instruction-following capabilities.
Loading preview...
Overview
Shellypeckie/student_qwen3_1p7b_clean4b_dolly_seq_kd is a 2 billion parameter language model, specifically a fine-tuned variant of the Qwen/Qwen3-1.7B architecture. This model has undergone Supervised Fine-Tuning (SFT) utilizing the TRL library, a framework for transformer reinforcement learning.
Key Capabilities
- Instruction Following: The model is fine-tuned to respond to user prompts, indicating an adaptation for instruction-based text generation.
- Text Generation: Capable of generating coherent and contextually relevant text based on given inputs.
- Qwen3 Base: Benefits from the foundational capabilities and efficiency of the Qwen3-1.7B model.
Training Details
The model was trained using SFT, a common method for adapting pre-trained language models to specific tasks by providing examples of desired input-output pairs. The training leveraged specific versions of key frameworks:
- TRL: 0.29.0
- Transformers: 5.8.1
- Pytorch: 2.8.0+cu128
- Datasets: 4.7.0
- Tokenizers: 0.22.2
Use Cases
This model is suitable for general text generation tasks where a smaller, fine-tuned model is preferred for efficiency and specific instruction-following behavior. Its 2 billion parameters make it a good candidate for applications requiring a balance between performance and computational resources.