Goekdeniz-Guelmez/Qwen3-0.6B-gabliterated-Dev is a 0.8 billion parameter Qwen3-based language model developed by Goekdeniz-Guelmez, featuring a 40960 token context length. This model introduces 'Gabliteration,' a novel neural weight modification technique that uses adaptive multi-directional projections to selectively alter behavioral patterns. It is designed to address limitations of traditional abliteration methods by improving model quality while modifying specific behaviors, making it suitable for applications requiring fine-grained control over model responses.
Loading preview...
Gabliterated Model Series Overview
This model, part of the Gabliterated series by Goekdeniz-Guelmez, introduces a novel neural weight modification technique called Gabliteration. This method advances beyond traditional abliteration by employing adaptive multi-directional projections with regularized layer selection. The primary goal of Gabliteration is to modify specific behavioral patterns in large language models without compromising overall model quality, a common limitation in existing abliteration techniques.
Key Technical Aspects
- Gabliteration Technique: Extends foundational work on single-direction abliteration to a comprehensive multi-directional framework with theoretical guarantees.
- Refusal Direction Extraction: Utilizes singular value decomposition on difference matrices between harmful and harmless prompt representations to identify multiple refusal directions.
- Scalability: The Gabliterated series includes models ranging from 0.6B to 32B parameters, demonstrating the technique's effectiveness across various model sizes.
Potential Use Cases
This model is particularly relevant for developers and researchers focused on:
- Behavioral Control: Implementing precise and selective modifications to model outputs.
- Safety and Alignment: Enhancing model safety by altering undesirable behaviors without degrading general performance.
- Advanced Fine-tuning: Exploring new methods for fine-tuning that go beyond traditional approaches to achieve specific behavioral outcomes.