Model Overview

MuXodious/LongWriter-llama3.1-8B-absolute-heresy is an 8 billion parameter model based on Meta-Llama-3.1-8B, fine-tuned from the THUDM LongWriter project. Its primary distinguishing feature is its exceptional capability for long-context generation, specifically designed to produce over 10,000 words in a single output, leveraging a 32768-token context length.

Key Characteristics

• Long-form Generation: Optimized for generating extensive text, demonstrated by its ability to write 10,000-word articles or guides.
• Heretication Process: This specific variant was created using P-E-W's Heretic (v1.1.0) abliteration engine, merged with a Magnitude-Preserving Orthogonal Ablation PR. It achieves an "Absolute Heresy" classification, indicating low refusal rates (9/100) and a KL Divergence of 0.0743, suggesting a significant alteration from its base model.
• Research Focus: Generated primarily for research purposes by redaihf to test MPOA abliterations against standard abliterations.

Good for

• Extreme Long-form Content Creation: Ideal for applications requiring very lengthy articles, reports, stories, or documentation.
• Research into Model Abliteration: Useful for researchers studying the effects and characteristics of model abliteration techniques.
• High-throughput Long Generation: Can be deployed with tools like vLLM for efficient generation of long texts.