aziz9788/qwen35-4b-t13-opd-rkl-ckpt250
The aziz9788/qwen35-4b-t13-opd-rkl-ckpt250 is a 4.5 billion parameter Qwen3.5-based causal language model, fine-tuned using a hybrid on-policy reverse-KL method. This model integrates a merged parent with an OPD reverse-KL LoRA, making it ready for direct use without separate LoRA loading. It is optimized for general language generation tasks, leveraging its specific fine-tuning approach for improved performance.
Loading preview...
Model Overview
The aziz9788/qwen35-4b-t13-opd-rkl-ckpt250 is a 4.5 billion parameter language model built upon the Qwen3.5 architecture. It represents a full BF16 merge of a merged_t13_ckpt12 parent model with a hybrid On-Policy Distribution (OPD) reverse-KL LoRA, specifically checkpoint-250.
Key Characteristics
- Integrated Fine-tuning: The model is provided as a ready-to-use merged checkpoint, eliminating the need to load a separate LoRA adapter. Users can load it directly using
from_pretrainedlike any standard CausalLM. - Training Method: It utilizes an on-policy reverse-KL (k1) fine-tuning approach, employing hybrid teachers (Qwen3.7-Flash transitioning to Qwen3.8-Max hard-state) on a dataset comprising STEM1500 and IF1000.
- Compatibility: It maintains the same tokenizer, chat template, and generation configuration as its T13 parent model.
- Parameter Count: With 4.5 billion parameters, it offers a balance between performance and computational efficiency.
- Context Length: The model supports a context length of 32768 tokens.
Usage Considerations
This model is suitable for developers seeking a Qwen3.5-based model with specialized fine-tuning for general language tasks. Its integrated nature simplifies deployment, making it a convenient option for various applications where a 4.5B parameter model with a substantial context window is appropriate.