officialmobai/rostam-r1-stage13-2
Rostam R1 Stage 13-2 is a 5.1 billion parameter DPO-aligned language model from the Rostam series, fine-tuned from prog-love/rostam-r1-stage13-1. It demonstrates improved performance on question answering tasks, achieving 42.08% Exact Match and 67.20% F1 Score on the PQuAD Test. This model is optimized for reducing false abstention and eliminating repetition loops, making it suitable for reliable conversational AI applications.
Loading preview...
Rostam R1 Stage 13-2: DPO Aligned Language Model
Rostam R1 Stage 13-2 is a 5.1 billion parameter model, representing the Direct Preference Optimization (DPO) aligned release within the Rostam series. It is a fine-tuned iteration of prog-love/rostam-r1-stage13-1, focusing on enhancing performance and reliability through DPO.
Key Capabilities & Improvements
- Enhanced Question Answering: Achieves a 42.08% Exact Match (EM) and 67.20% F1 Score on the PQuAD Test (8,002 samples), showing significant improvements over previous stages.
- Reduced False Abstention: The false abstention rate has been substantially lowered to 1.23%.
- Repetition Mitigation: Incorporates enforced
<|turn|>stop boundaries to effectively eliminate repetition loops in generated text. - Full-Parameter DPO Training: The model underwent full-parameter BF16 DPO training across all 5.104 billion parameters, utilizing a Native PyTorch Fused AdamW optimizer.
When to Use This Model
This model is particularly well-suited for applications requiring accurate question answering, reduced conversational repetition, and reliable text generation. Its DPO alignment and specific optimizations make it a strong candidate for chatbots, virtual assistants, and other interactive AI systems where precision and coherence are critical.