ayushshah/Qwen3-1.7B-Chat
ayushshah/Qwen3-1.7B-Chat is a 1.7 billion parameter causal language model, a Direct Preference Optimized (DPO) variant of Qwen3-1.7B-UltraChat-SFT, developed by ayushshah. It features a 32,768 token context length and is optimized for improved response quality, reasoning, truthfulness, and conversational naturalness through DPO alignment on human preference datasets. This model excels in generating helpful and coherent conversational responses.
Loading preview...
Model Overview
ayushshah/Qwen3-1.7B-Chat is a 1.7 billion parameter causal language model, developed by ayushshah, that has undergone a three-stage training process: pretraining, supervised fine-tuning (SFT), and Direct Preference Optimization (DPO). It is built upon the Qwen3-1.7B-Base architecture and further fine-tuned from Qwen3-1.7B-UltraChat-SFT.
Key Capabilities & Training
- Direct Preference Optimization (DPO): This model's primary differentiator is its DPO alignment using multiple human preference datasets, which significantly enhances response quality, reasoning, truthfulness, and conversational naturalness compared to its SFT predecessor.
- Context Length: Supports a substantial context window of 32,768 tokens.
- Training Details: The DPO stage utilized LoRA adapters, TRL, and Unsloth frameworks, with specific dataset preprocessing to filter and undersample preference data for optimal alignment.
Performance & Limitations
Evaluation using EleutherAI's Language Model Evaluation Harness shows improvements in benchmarks like MMLU-Pro (+1.95), GSM8K (+1.21), and TruthfulQA (+1.47) compared to the SFT model. Blind A/B evaluations also favored the DPO model. However, it retains limitations of the base Qwen3-1.7B architecture, including potential for incorrect information or struggles with complex multi-step reasoning. A trade-off observed is a slight reduction in strict instruction-following for more elaborate responses and a higher likelihood of hallucinations on complex queries.
Usage
This model is available for use with transformers and is recommended with unsloth for optimized performance. Quantized versions are also available for LLaMA.cpp and Ollama.