saidutta69/Llama-3.2-1B-Instruct-heretic

Overview

This model, Llama-3.2-1B-Instruct-heretic, is a 1 billion parameter instruction-tuned causal language model developed by saidutta69. It is a modified version of meta-llama/Llama-3.2-1B-Instruct, specifically engineered to suppress refusal behaviors. This "decensoring" is achieved through a technique called directional ablation (abliteration) using Heretic v1.2.0, which involves targeted weight edits to the attention output and MLP down-projections. This method ensures that the base model's core knowledge and instruction-following capabilities remain largely intact, unlike traditional fine-tuning which can alter these aspects.

Key Capabilities

• Decensored Output: Significantly reduces refusal rates from 96/100 to 7/100 on adversarial prompts, allowing it to comply with requests the original model would refuse.
• Preserved Core Functionality: Maintains the original Llama-3.2-1B-Instruct's knowledge and instruction-following due to the targeted nature of the weight edits.
• Efficient Performance: Achieves a low KL divergence of 0.1713 compared to the original model, indicating minimal deviation in overall behavior despite the refusal suppression.
• On-Device Deployment: Its 1 billion parameter size makes it suitable for deployment on edge devices, mobile applications, or other resource-constrained environments.

Good For

• Local Agents & Roleplay: Ideal for applications requiring an uncensored model for interactive agents or role-playing scenarios.
• Alignment Research: Useful for researchers studying alignment, refusal mechanics, and the impact of targeted model modifications.
• Edge & Mobile Use Cases: Designed for scenarios where a responsive, uncensored model is needed on-device or in environments with limited computational resources.
• Developers Seeking Flexibility: Provides Meta's Llama-3.2 architecture without the typical RLHF-era refusal guardrails, offering greater flexibility for specific use cases.