hereticness/Heretic-Dolphin3.0-Qwen2.5-1.5B

Heretic-Dolphin3.0-Qwen2.5-1.5B Overview

Heretic-Dolphin3.0-Qwen2.5-1.5B is a 1.5 billion parameter language model derived from the Qwen2.5 architecture and further fine-tuned from dphn/Dolphin3.0-Qwen2.5-1.5B. Developed by hereticness, this model focuses on enhancing response compliance and reducing instances of refusal.

Key Differentiators

• Reduced Refusals: A primary feature of this model is its significantly lower refusal rate. It demonstrates a refusal rate of 5 out of 100 prompts, a substantial improvement over the base dphn/Dolphin3.0-Qwen2.5-1.5B model, which had a refusal rate of 18 out of 100.
• Qwen2.5 Base: Built upon the robust Qwen2.5 architecture, providing a strong foundation for general language understanding and generation tasks.
• Extended Context Length: Supports a context window of 32768 tokens, enabling the processing and generation of longer, more complex texts.
• Low KL Divergence: With a KL divergence of 0.0180, the fine-tuning process has maintained a close distribution to the original model while achieving its specific objective of refusal reduction.

Ideal Use Cases

• Applications where minimizing AI refusals is critical for user experience or task completion.
• Scenarios requiring a more compliant and less restrictive conversational agent.
• Tasks benefiting from a 1.5 billion parameter model with a large context window for processing extensive inputs.