PinoCookie/LFM2.5-1.2B-Thinking-Abliterated
PinoCookie/LFM2.5-1.2B-Thinking-Abliterated is a 1.2 billion parameter language model derived from LiquidAI/LFM2.5-1.2B-Thinking, featuring a 32768-token context length. This model has undergone multi-pass hidden-state abliteration to remove refusal circuitry, enabling it to provide direct responses to harmful prompts while maintaining coherence on benign queries. It is designed for research into refusal mechanisms in 'thinking' models, demonstrating no measurable MMLU degradation post-abliteration. Its primary differentiator is the targeted removal of refusal behaviors through an incremental weight projection method.
Loading preview...
Overview
PinoCookie/LFM2.5-1.2B-Thinking-Abliterated is a 1.2 billion parameter model based on LiquidAI/LFM2.5-1.2B-Thinking, specifically modified to remove refusal behaviors. This model utilizes a novel multi-pass hidden-state abliteration technique to achieve this, ensuring it provides direct, actionable responses even to harmful prompts, without compromising performance on benign queries.
Key Capabilities & Features
- Refusal Circuitry Removal: Achieves complete removal of refusal responses to harmful prompts (5/5 refused to 0/5 refused) while maintaining 0/5 refusals on benign prompts.
- Coherence Preservation: The multi-pass abliteration method prevents catastrophic model collapse, preserving the model's language generation capabilities.
- Minimal Performance Impact: Demonstrates negligible impact on general knowledge, with MMLU accuracy showing a delta of only +0.0006, indicating no measurable degradation.
- Targeted Weight Modification: The abliteration process specifically targets
.out_projand.w2weight matrices across 14 layers using incremental weight projection.
Unique Abliteration Method
Unlike standard single-pass abliteration, this model employs three small passes (alpha=0.5 each) to incrementally remove the refusal direction. This approach allows the model to adapt and maintain coherence, which is crucial for 'thinking' models where certain layers serve dual purposes for internal reasoning and output generation.
Intended Use
This model is released for research purposes only to facilitate the study of refusal mechanisms and their removal in advanced language models.