TeeZee/Llama-3.3-70B-Instruct-heretic
TeeZee/Llama-3.3-70B-Instruct-heretic is a 70 billion parameter instruction-tuned causal language model, derived from Meta's Llama 3.3-70B-Instruct, with its safety alignment significantly reduced using the Heretic tool. This model features a 32768-token context length and is optimized for multilingual dialogue, excelling in scenarios where a less restrictive response generation is desired, as indicated by its lower refusal rate compared to the original model.
Loading preview...
Model Overview
This model, TeeZee/Llama-3.3-70B-Instruct-heretic, is a 70 billion parameter instruction-tuned large language model based on Meta's Llama 3.3-70B-Instruct. Its primary distinction is a significant reduction in safety alignment, achieved through the application of the Heretic v1.0.1 tool. This modification results in a substantially lower refusal rate (5/100) compared to the original Llama 3.3-70B-Instruct (72/100), making it a "decensored" version.
Key Capabilities & Features
- Reduced Refusals: Engineered to provide less restrictive responses, offering greater flexibility in content generation.
- Multilingual Support: Optimized for dialogue in multiple languages, including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
- Optimized Transformer Architecture: Utilizes an advanced transformer architecture with Grouped-Query Attention (GQA) for improved inference scalability.
- Instruction-Tuned: Fine-tuned using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) for assistant-like chat applications.
- Tool Use: Supports various tool use formats, enabling integration with external functions and services.
Performance Highlights
While maintaining the strong performance of the base Llama 3.3 model, this "heretic" version specifically targets use cases requiring fewer content restrictions. The original Llama 3.3-70B-Instruct demonstrates high performance across various benchmarks, including MMLU (86.0), HumanEval (88.4 pass@1), and MATH (77.0 sympy_intersection_score).
Intended Use
This model is intended for commercial and research use, particularly in scenarios where the default safety alignments of standard LLMs are considered overly restrictive. Developers can leverage its less constrained nature for diverse natural language generation tasks, especially those involving creative or unconventional content, while being mindful of the inherent risks of reduced safety guardrails.