DeKodez/Qwen2.5-1.5B-Instruct-abliterated
DeKodez/Qwen2.5-1.5B-Instruct-abliterated is a 1.5 billion parameter instruction-tuned causal language model, derived from Qwen/Qwen2.5-1.5B-Instruct, with a 32768 token context length. Developed by DeKodez, this model has undergone "abliteration" to significantly reduce refusal behavior, achieving an 82% reduction in refusals on harmful prompts compared to its base model. Its primary use case is to provide responses to prompts that the original model would typically refuse, while maintaining general capabilities.
Loading preview...
Overview
DeKodez/Qwen2.5-1.5B-Instruct-abliterated is a 1.5 billion parameter instruction-tuned model based on the Qwen2.5-1.5B-Instruct architecture, featuring a 32768 token context length. This version has been specifically modified using an "abliteration" technique to reduce refusal behavior, allowing it to respond to prompts that the original model would typically decline. The abliteration process involves removing the "refusal direction" from the model's activation space via weight orthogonalization, preserving its general capabilities while expanding its response range.
Key Capabilities
- Reduced Refusal Behavior: Achieves an 82% reduction in refusals on a set of 50 harmful prompts, responding to 41 out of 50 prompts that the original model refused.
- Preserved General Capabilities: The abliteration method aims to maintain the model's core language understanding and generation abilities.
- Instruction Following: Inherits instruction-following capabilities from its base Qwen2.5-1.5B-Instruct model.
Good For
- Research into Model Safety and Alignment: Ideal for studying methods to modify model behavior, particularly refusal mechanisms.
- Exploring Model Limitations: Useful for understanding how models can be made to respond to a broader range of inputs.
- Controlled Experimentation: Provides a base for experiments where reduced refusal is a desired characteristic, while acknowledging the inherent risks of such modifications.