OwenArli/ArliAI-Llama-3-8B-Dolfin-v0.6-Abliterated

OwenArli/ArliAI-Llama-3-8B-Dolfin-v0.6-Abliterated Overview

This model is an 8 billion parameter instruction-tuned variant of Meta-Llama-3-8B-Instruct, developed by OwenArli. It leverages an improved Dolphin and WizardLM dataset for fine-tuning, primarily aimed at enhancing instruction adherence and minimizing command refusals. A key differentiator for version 0.6 is its foundation on an "abliterated" Llama 3 8B Instruct base, which is specifically engineered to prevent the model from refusing commands.

Key Capabilities

• Enhanced Instruction Following: Fine-tuned to better understand and execute user instructions.
• Reduced Refusals: Built upon an "abliterated" base model designed to eliminate command refusals, offering more compliant responses.
• Llama 3 Architecture: Benefits from the robust architecture of Meta's Llama 3 8B Instruct model.
• Context Length: Supports an 8192 token context length, despite being trained with a shorter sequence length due to dataset characteristics.

Training Details

The model was trained for approximately two days on 2xRTX 3090 GPUs. It utilized 4-bit loading and QLoRA with a 64-rank and 128-alpha configuration, resulting in roughly 2% trainable weights. The training sequence length was 2048, aligning with the average length of the dataset, though the base model's 8192 context length is maintained in performance.

Good For

• Applications requiring a highly compliant language model that strictly follows instructions.
• Use cases where minimizing model refusals is critical.
• Developers seeking an instruction-tuned Llama 3 variant with specific refusal-reduction optimizations.