Overview

Greytechai/Llama-3-70B-Instruct-abliterated-v3 is a modified version of Meta's Llama-3-70B-Instruct model. This 70 billion parameter model has been "abliterated" through a process of orthogonalization, which specifically targets and inhibits the model's tendency to refuse requests or lecture on ethics/safety. The methodology is based on the concept that refusal in LLMs is mediated by a single direction, which has been removed from this model's bfloat16 safetensor weights.

Key Capabilities

• Uncensored Responses: Engineered to remove refusal behaviors, providing direct answers without ethical lecturing.
• Preserves Original Llama 3 Capabilities: Maintains the core knowledge, training, and instruction-following abilities of the base Llama-3-70B-Instruct model.
• Surgical Modification: Utilizes an ablation technique that is more precise and requires less data than traditional fine-tuning for specific behavioral changes.
• Reduced Hallucinations: The refined methodology for v3 aims to induce fewer hallucinations compared to earlier versions.

Good For

• Applications requiring an instruction-tuned model that will not refuse or moralize user prompts.
• Developers exploring advanced model modification techniques like orthogonalization for specific feature removal or augmentation.
• Use cases where maintaining the original model's knowledge base is crucial, while eliminating specific undesirable behaviors.