Model Overview

iamplus/Llama-2-7b-hf-ChatOrca is a fine-tuned language model based on Meta's Llama-2-7b-hf, designed to improve reasoning and multi-turn conversational abilities. It leverages a diverse training regimen to achieve enhanced interactive performance.

Key Capabilities

• Enhanced Reasoning: Trained with 1 million Orca data points (Gpt-4 Orca data - OpenOrca) to boost logical deduction and problem-solving.
• Multi-turn Conversations: Incorporates 1.7 million chat data points, including OpenAssistant Chat and Ultrachat, enabling it to maintain coherent and extended dialogues.
• Instruction Following: Further refined with 30,000 OpenPlatypus data entries for better adherence to instructions.
• Llama 2 Prompt Format: Adheres to the official Meta Llama 2 chat model prompt format for both single and multi-turn interactions, ensuring compatibility and ease of use.

Training Details

The model was trained for 2 epochs with a batch size of 128 and a sequence length of 4096. It utilized a learning rate of 2e-5 (Cosine) and an Any Precision AdamW Optimizer, with bf16 precision.

Good For

• Applications requiring robust multi-turn conversational AI.
• Tasks that benefit from improved reasoning and instruction following.
• Developers familiar with the Llama 2 ecosystem seeking an enhanced chat-optimized variant.