ArliAI-Llama-3-8B-Cumulus-v0.1 Overview

ArliAI-Llama-3-8B-Cumulus-v0.1 is an 8 billion parameter language model developed by OwenArli, built upon the Meta-Llama-3-8B-Instruct architecture. The primary objective behind its creation was to explore the complete uncensoring of the base Llama 3 model. While this experimental approach has led to a reduction in its general reasoning and knowledge capabilities compared to the original Llama 3, it has concurrently enhanced its performance in specific creative applications.

Key Characteristics

• Base Model: Derived from Meta-Llama-3-8B-Instruct, adhering to the Meta Llama 3 License agreement.
• Context Length: Although trained with a 4096 sequence length, the model effectively maintains performance at the base model's 8192 token context length.
• Training Methodology: Fine-tuned using 4-bit loading and QLoRA with a 64-rank and 128-alpha configuration, resulting in approximately 2% trainable weights. Training was conducted over roughly 3 days on an RTX 4090.
• Instruction Format: Utilizes a specific instruction format for multi-turn conversations, including system, user, and assistant roles delimited by <|begin_of_text|>, <|eot_id|>, and header IDs.

Primary Use Case

This model is particularly well-suited for roleplay (RP) scenarios due to its uncensored nature. Developers seeking a model with fewer content restrictions for creative and interactive narrative generation may find this model beneficial. However, for tasks requiring high accuracy in reasoning or extensive factual knowledge, other Llama 3 variants might be more appropriate.