arnomatic/gpt-oss-20b-heretic-scannerV1-1

GPT-OSS-20B Heretic (Scanner V1.1) Overview

This model, arnomatic/gpt-oss-20b-heretic-scannerV1-1, is a 20 billion parameter language model derived from OpenAI's gpt-oss-20b. Its primary distinction is its decensored nature, achieved through a specialized process utilizing a version of the Heretic tool.

Key Modifications and Performance

The decensoring process involved a targeted intervention on the attn.o_proj layers, specifically focusing on layers 10 and above, where refusal directions were identified. Notably, mlp.down_proj layers were excluded from modification as they were found to contribute negligibly to divergence. This precise methodology aims to alter the model's refusal behavior while minimizing broader impact.

Trial 142 results indicate a significant reduction in refusals, with the model exhibiting 8 refusals out of 100 prompts, which was the primary goal of this modification. The KL Divergence was measured at 0.94, providing insight into the distributional shift from the base model.

Abliteration Parameters

The specific parameters used for the abliteration process include:

• direction_index: 16.60
• attn.o_proj.max_weight: 1.47
• attn.o_proj.max_weight_position: 9.62
• attn.o_proj.min_weight: 1.37
• attn.o_proj.min_weight_distance: 8.09

Use Cases

This model is suitable for developers and researchers who require a large language model with a reduced propensity for content refusals, particularly in scenarios where the base gpt-oss-20b model's safety alignments might be overly restrictive for specific applications. Its 32768 token context length makes it versatile for various long-form generation tasks.