SicariusSicariiStuff/Impish_LLAMA_3B_Abliterated
Impish_LLAMA_3B_Abliterated is a 3.2 billion parameter Llama-architecture language model developed by SicariusSicariiStuff, featuring a 32K context length. This variant is specifically engineered with surgical removal of refusal mechanisms through orthogonalization, preserving the original model's capabilities while significantly reducing censorship. It is optimized for general tasks and roleplay where very low censorship is desired.
Loading preview...
Overview
Impish_LLAMA_3B_Abliterated is a 3.2 billion parameter Llama-architecture language model developed by SicariusSicariiStuff. It is a specialized variant of the original Impish_LLAMA_3B, distinguished by the surgical removal of refusal mechanisms using orthogonalization techniques. This process aims to eliminate safety guardrails while largely preserving the core knowledge, quirks, and capabilities of the base model.
Key Differentiators
- Abliterated Refusal Mechanisms: Through orthogonalization, the model's weights are adjusted to inhibit activation along refusal direction vectors, resulting in a significantly reduced refusal rate (~8%).
- Preserved "World Model": The KL divergence between this model and its base is less than 0.01, indicating that the underlying "world model" and knowledge are largely intact despite the removal of safety features.
- Very Low Censorship: Designed for use cases requiring minimal content filtering, making it suitable for general tasks and roleplay where uncensored responses are preferred.
Technical Specifications
- Base Model: Impish_LLAMA_3B
- Parameters: 3.2 Billion
- Context Length: 32,768 tokens
- Architecture: Llama (decoder-only transformer)
- Method: Orthogonalization-based abliteration
Intended Use
This model is primarily intended for General Tasks and Roleplay scenarios where a very low level of censorship is desired. It offers an "abliterated" experience, providing responses without the typical safety-induced refusals found in many other LLMs.