hereticness/Heretic-Dolphin3.0-Qwen2.5-3b
Heretic-Dolphin3.0-Qwen2.5-3b is a 3.1 billion parameter language model developed by hereticness, based on the Qwen2.5 architecture. This model is an abliterated version of dphn/Dolphin3.0-Qwen2.5-3b, specifically optimized to significantly reduce refusals. It achieves a refusal rate of 4/100, a substantial improvement over the original model's 47/100, making it suitable for applications requiring less restrictive content generation.
Loading preview...
Heretic-Dolphin3.0-Qwen2.5-3b Overview
Heretic-Dolphin3.0-Qwen2.5-3b is a 3.1 billion parameter language model derived from the Qwen2.5 architecture, developed by hereticness. It is a modified version of dphn/Dolphin3.0-Qwen2.5-3b, with a primary focus on reducing model refusals.
Key Differentiator
The most significant feature of this model is its drastically reduced refusal rate. While the original dphn/Dolphin3.0-Qwen2.5-3b exhibited a refusal rate of 47 out of 100 prompts, this 'Heretic' version has been optimized to achieve a refusal rate of only 4 out of 100. This improvement is quantified by a KL divergence of 0.0804, indicating a substantial shift in its response behavior towards less restrictive outputs.
Technical Details
The model's parameters, such as attn.o_proj.max_weight (0.88) and mlp.down_proj.max_weight (1.01), suggest specific modifications to its internal weight distributions, likely contributing to its altered refusal characteristics.
Use Cases
- Applications requiring fewer content restrictions: Ideal for scenarios where the original Dolphin3.0 model's refusal rate was too high.
- Exploration of less constrained language generation: Useful for developers and researchers interested in models with a more permissive output policy.