richardyoung/Qwen2.5-14B-Instruct-1M-heretic
richardyoung/Qwen2.5-14B-Instruct-1M-heretic is a 14.7 billion parameter causal language model, based on the Qwen2.5 architecture, with a 1 million token context length. This version is a decensored modification of the original Qwen/Qwen2.5-14B-Instruct-1M, achieved by suppressing refusal behaviors. It is optimized for research into model behavior, red-teaming, and creative applications where safety guardrails are intentionally removed.
Loading preview...
Model Overview
This model, richardyoung/Qwen2.5-14B-Instruct-1M-heretic, is a modified version of the Qwen2.5-14B-Instruct-1M, a 14.7 billion parameter causal language model developed by Qwen. Its primary distinction is the intentional removal of safety guardrails and refusal behaviors through an "abliteration" process using the Heretic tool. The base Qwen2.5-1M model is notable for its extensive context length, supporting up to 1 million tokens, and its architecture includes transformers with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.
Key Characteristics
- Decensored Behavior: Modified to suppress refusal behaviors, resulting in significantly fewer refusals (3/100 compared to 87/100 for the original model).
- Ultra-Long Context: Inherits the base model's capability to process up to 1,010,000 tokens, with optimized inference using a custom vLLM framework that employs sparse attention and length extrapolation.
- Qwen2.5 Architecture: Built on the Qwen2.5 series, known for improved performance in long-context tasks while maintaining short-task capabilities.
Intended Use Cases
- Research and Education: Ideal for studying model behavior, limitations, and the effects of safety guardrail removal.
- Red-Teaming and Safety Research: Useful for probing model vulnerabilities and conducting safety evaluations.
- Creative Applications: Suitable for creative writing and roleplay in contexts where unconstrained generation is desired, with user responsibility for outputs.