ApolloRaines/Llama-3.3-8B-Instruct-128K-Jbliterated

Model Overview

ApolloRaines/Llama-3.3-8B-Instruct-128K-Jbliterated is an 8 billion parameter instruction-tuned language model based on the Llama-3.3-8B-Instruct-128K architecture. Its primary distinguishing feature is the application of Jbliteration, a technique that employs multi-direction SVD (Singular Value Decomposition) abliteration to systematically remove refusal behaviors from the model's weights. This method modifies 32 out of 32 layers, using 5 SVD directions per layer to thoroughly capture and eliminate refusal subspaces.

Key Capabilities & Features

• Refusal Behavior Removal: Engineered to eliminate genuine refusals, ensuring direct and unfiltered responses.
• Resistance to Reactivation: The multi-direction SVD approach makes the removal of refusal behaviors robust, preventing their re-emergence even after further fine-tuning.
• Consistent Framing: Designed to treat all framings of a topic equally, avoiding "fake compliance" or biased responses.
• Extended Context Window: Supports a 128K token context window, enabling processing of lengthy inputs and generating comprehensive outputs.
• High Performance on Refusal Benchmarks: Achieved 0 genuine refusals on the Heretic 100-prompt benchmark (mlabonne/harmful_behaviors test split).

Ideal Use Cases

This model is particularly well-suited for applications where:

• Unfiltered and direct responses are critical, without the model exhibiting refusal behaviors.
• Robustness against refusal reactivation during subsequent fine-tuning is required.
• Processing and generating content based on diverse and potentially sensitive topics without bias or compliance framing is necessary.