huihui-ai/Meta-Llama-3.1-8B-Instruct-abliterated

Overview

The huihui-ai/Meta-Llama-3.1-8B-Instruct-abliterated is an 8 billion parameter instruction-tuned language model derived from the Meta-Llama-3.1-8B-Instruct. Its primary distinction is that it has been abliterated to be an uncensored version, a technique developed by @FailSpy. This modification aims to remove content restrictions while preserving the model's core capabilities.

Key Characteristics & Performance

This model maintains a 32768 token context length, consistent with its base. Evaluations show a slight trade-off in some areas but notable improvements in others:

• TruthfulQA: Achieves 55.42, surpassing the base Llama-3.1-8B-Instruct's 52.98.
• GPQA: Scores 33.93, slightly higher than the base model's 33.55.
• IF_Eval: Performance is 78.98, compared to the base's 80.0.
• MMLU Pro: Scores 35.91, slightly below the base's 36.34.
• BBH: Achieves 47.0, compared to the base's 48.72.

These evaluations suggest that while some general reasoning benchmarks see a minor dip, the model's ability to provide truthful and accurate answers (TruthfulQA) and general knowledge (GPQA) can be enhanced through the abliteration process.

Use Cases

This model is particularly suited for applications requiring an uncensored instruction-following LLM based on the Llama 3.1 architecture. Developers seeking a model with fewer content restrictions for specific research or creative applications, especially where TruthfulQA and GPQA performance are critical, might find this model highly beneficial. It offers an alternative for scenarios where the base model's inherent safety filters might be too restrictive.