OwenArli/ArliAI-Llama-3-8B-Dolfin-v0.3 Overview

This model is an 8 billion parameter instruction-tuned variant of Meta-Llama-3-8B-Instruct, developed by OwenArli. It has been fine-tuned using an enhanced combination of Dolphin and WizardLM datasets, specifically aimed at improving the model's ability to follow instructions accurately and minimize instances of refusal.

Key Capabilities

• Enhanced Instruction Following: The primary focus of this fine-tune is to make the model more adept at understanding and executing user instructions.
• Reduced Refusal Rates: Training with improved datasets helps the model provide more direct and helpful responses, reducing unnecessary refusals.
• Llama 3 Architecture: Benefits from the robust architecture of the Meta Llama 3 base model.
• 8192 Token Context Length: Despite being trained with a sequence length of 2048, the model effectively retains the base model's 8192 token context capability.

Training Details

The model was trained for approximately one day on 2xRTX 3090 GPUs, utilizing 4-bit loading and QLoRA with a 64-rank and 128-alpha configuration, resulting in roughly 2% trainable weights.

Good For

• Applications requiring strong instruction adherence.
• General-purpose conversational AI where direct and compliant responses are preferred.
• Scenarios where a balance of performance and resource efficiency (8B parameters) is desired.