PirxTion/full-dataset-instruction
PirxTion/full-dataset-instruction is a 0.8 billion parameter instruction-tuned causal language model, fine-tuned from andresnowak/Qwen3-0.6B-instruction-finetuned_v2. It was trained using Direct Preference Optimization (DPO) with TRL, making it suitable for tasks requiring nuanced instruction following and preference alignment. The model supports a context length of 32768 tokens, offering robust performance for various generative AI applications.
Loading preview...
Model Overview
PirxTion/full-dataset-instruction is a 0.8 billion parameter instruction-tuned language model, building upon the andresnowak/Qwen3-0.6B-instruction-finetuned_v2 base model. It has been specifically trained using Direct Preference Optimization (DPO), a method designed to align language models with human preferences by treating the preference data as implicit rewards.
Key Training Details
- Base Model: Fine-tuned from
andresnowak/Qwen3-0.6B-instruction-finetuned_v2. - Training Method: Utilizes Direct Preference Optimization (DPO), as introduced in the paper "Direct Preference Optimization: Your Language Model is Secretly a Reward Model" (arXiv link).
- Framework: Training was conducted using the TRL library.
- Context Length: The model supports a substantial context length of 32768 tokens.
Use Cases
This model is well-suited for applications requiring a small yet capable instruction-following model, particularly where preference alignment is crucial. Its DPO training makes it effective for generating responses that adhere to specified instructions and exhibit desired behavioral characteristics.