Overview
This model, p-e-w/Llama-3.1-8B-Instruct-heretic, is an 8 billion parameter instruction-tuned variant of Meta's Llama 3.1 8B Instruct, specifically modified using the Heretic v1.0.0 tool. Its primary distinction lies in its decensored nature, exhibiting a significantly lower refusal rate (3/100) compared to the original model (96/100) while maintaining a low KL divergence of 0.02. The base Llama 3.1 architecture, developed by Meta, is an optimized transformer designed for multilingual dialogue, supporting a 32,768 token context length and trained on over 15 trillion tokens of publicly available data with a knowledge cutoff of December 2023.
Key Capabilities
- Reduced Refusals: Engineered to provide responses with minimal content filtering, offering greater flexibility for diverse applications.
- Multilingual Support: Capable of generating text and code in English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
- Instruction Following: Optimized for assistant-like chat and various natural language generation tasks.
- Tool Use: Supports advanced tool-use and function calling, enabling integration with external services.
- Strong Performance: Maintains competitive performance across general, reasoning, code, and math benchmarks, including a HumanEval pass@1 score of 72.6 and a MATH (CoT) score of 51.9.
Good For
- Research and Development: Ideal for exploring less constrained language generation and understanding model behavior without aggressive safety filters.
- Creative Applications: Suitable for creative writing, role-playing, and scenarios where unfiltered or diverse content generation is desired.
- Specialized Chatbots: Can be used in applications where standard LLM safety guardrails might be overly restrictive for the intended use case, provided developers implement their own responsible use policies.
- Multilingual Applications: Effective for tasks requiring generation in its supported languages, particularly where the original Llama 3.1's capabilities are beneficial.