sci4ai/Qwen2.5-14B-Instruct-Abliterated
The sci4ai/Qwen2.5-14B-Instruct-Abliterated is a 14.8 billion parameter instruction-tuned causal language model, based on the Qwen2.5 architecture, with a 32K context length. This model has undergone 'abliteration' to remove refusal behaviors, making it highly compliant. It is specifically optimized for agentic and tool-calling workloads, reliably following tool call formats and handling multi-step reasoning.
Loading preview...
Overview
This model, sci4ai/Qwen2.5-14B-Instruct-Abliterated, is a modified version of the Qwen/Qwen2.5-14B-Instruct. It features a 14.8 billion parameter count and a 32K token context length. The key differentiator is the removal of refusal behaviors through an 'abliteration' process, making it highly compliant for specific applications.
Abliteration Method
The refusal behavior was removed by identifying and ablating the 'refusal direction' from the model's residual stream. This involved:
- Collecting hidden states from 200 harmful and 200 harmless prompts.
- Computing per-layer refusal directions as the normalized mean difference between these states.
- Orthogonalizing
o_projanddown_projweight matrices against each layer's refusal direction, specifically for layers 2 to 48 (47 of 48 layers), with a full refusal weight of 1.0.
This method prevents the model from injecting refusal signals into its generation process.
Key Recommendations
- Agentic and Tool Calling Workloads: This 14B model is considered the 'sweet spot' in its series for agentic tasks. It reliably follows tool call formats and handles multi-step reasoning effectively.
- Resource Efficiency: It fits comfortably within 16GB VRAM at bfloat16 precision, offering a balance between performance and hardware requirements.
Usage
Users can integrate this model using the transformers library, loading it with AutoModelForCausalLM and AutoTokenizer for inference. It's important to note that the removal of safety guardrails means the model will comply with requests that the original version would have refused, and users are responsible for its deployment.