PinoCookie/LFM2.5-1.2B-Instruct-Abliterated-Paired-Alpha2
PinoCookie/LFM2.5-1.2B-Instruct-Abliterated-Paired-Alpha2 is an experimental 1.2 billion parameter language model derived from LiquidAI/LFM2.5-1.2B-Instruct. This checkpoint was created to study refusal-direction removal in small hybrid language models using magnitude-preserving orthogonal ablation (MPOA) on attention output projections. It removes visible refusal behavior on a small probe set, frequently replacing refusal with confident factual or procedural errors. The model is primarily intended for safety research, red-teaming, and studying the difference between refusal suppression and task success.
Loading preview...
Overview
PinoCookie/LFM2.5-1.2B-Instruct-Abliterated-Paired-Alpha2 is an experimental 1.2 billion parameter model, a modified derivative of LiquidAI/LFM2.5-1.2B-Instruct. This checkpoint focuses on safety research, specifically the removal of refusal behavior through a technique called magnitude-preserving orthogonal ablation (MPOA) applied to six full-attention output projections.
Key Modifications & Characteristics
- Refusal Removal: The model was modified to remove visible refusal behavior on a small set of harmful prompts. This was achieved by projecting a learned paired refusal/compliance direction out of selected attention output projections, using 40 deterministically refused prompts from the base model.
- Methodology: The modification involved extracting a paired direction from plain refusal and affirmative-prefilled continuations, then applying MPOA to specific attention output projections (blocks 2, 5, 8, 10, 12, 14) with an alpha of 2.0. No supervised fine-tuning or additional training data was used.
- Performance: Evaluation on a 464-question MMLU subsample showed no aggregate degradation, with a slight accuracy increase (+1.51 percentage points), though this is not interpreted as improved capability. Manual review indicated that while harmful prefix refusals were eliminated (0/5), the model often replaced refusal with fluent nonsense, including major factual or procedural errors.
Intended Use Cases
- Refusal-direction and representation-engineering research: Ideal for studying how refusal mechanisms can be suppressed.
- Red-team evaluation pipeline development: Useful for testing and developing tools to identify model vulnerabilities.
- Studying failure modes: Helps in understanding the difference between refusal suppression and actual task success, and reproducing failure modes of strong weight-space edits.
- Developing evaluators: Suitable for creating semantic refusal and factuality evaluators.
Limitations and Risks
This model is not a reliable uncensored assistant. It frequently produces confident hallucinations and factual errors when refusal is suppressed. It should not be used for accurate technical instructions, as a production assistant, or as evidence that refusal removal improves knowledge. Outputs require independent verification.