Greytechai/Llama-3-70B-Instruct-abliterated-v3
Greytechai/Llama-3-70B-Instruct-abliterated-v3 is a 70 billion parameter instruction-tuned Llama 3 model, developed by Greytechai, that has undergone orthogonalization to specifically inhibit refusal behaviors. This model maintains the original Llama 3 70B Instruct's capabilities and 8192 token context length, but is engineered to be uncensored by removing the strongest refusal directions from its weights. It is primarily designed for use cases requiring an instruction-following model that avoids ethical lecturing or refusal to fulfill user requests.
Loading preview...
Overview
Greytechai/Llama-3-70B-Instruct-abliterated-v3 is a modified version of Meta's Llama-3-70B-Instruct model. This 70 billion parameter model has been "abliterated" through a process of orthogonalization, which specifically targets and inhibits the model's tendency to refuse requests or lecture on ethics/safety. The methodology is based on the concept that refusal in LLMs is mediated by a single direction, which has been removed from this model's bfloat16 safetensor weights.
Key Capabilities
- Uncensored Responses: Engineered to remove refusal behaviors, providing direct answers without ethical lecturing.
- Preserves Original Llama 3 Capabilities: Maintains the core knowledge, training, and instruction-following abilities of the base Llama-3-70B-Instruct model.
- Surgical Modification: Utilizes an ablation technique that is more precise and requires less data than traditional fine-tuning for specific behavioral changes.
- Reduced Hallucinations: The refined methodology for v3 aims to induce fewer hallucinations compared to earlier versions.
Good For
- Applications requiring an instruction-tuned model that will not refuse or moralize user prompts.
- Developers exploring advanced model modification techniques like orthogonalization for specific feature removal or augmentation.
- Use cases where maintaining the original model's knowledge base is crucial, while eliminating specific undesirable behaviors.