iamplus/Llama-2-7b-hf-ChatOrca
iamplus/Llama-2-7b-hf-ChatOrca is a 7 billion parameter language model fine-tuned from Meta's Llama-2-7b-hf. It is specifically trained on a blend of Orca data and multi-turn conversation datasets to enhance its reasoning capabilities and proficiency in extended dialogues. This model excels at holding multi-turn conversations and complex reasoning tasks, making it suitable for interactive AI applications.
Loading preview...
Model Overview
iamplus/Llama-2-7b-hf-ChatOrca is a fine-tuned language model based on Meta's Llama-2-7b-hf, designed to improve reasoning and multi-turn conversational abilities. It leverages a diverse training regimen to achieve enhanced interactive performance.
Key Capabilities
- Enhanced Reasoning: Trained with 1 million Orca data points (Gpt-4 Orca data - OpenOrca) to boost logical deduction and problem-solving.
- Multi-turn Conversations: Incorporates 1.7 million chat data points, including OpenAssistant Chat and Ultrachat, enabling it to maintain coherent and extended dialogues.
- Instruction Following: Further refined with 30,000 OpenPlatypus data entries for better adherence to instructions.
- Llama 2 Prompt Format: Adheres to the official Meta Llama 2 chat model prompt format for both single and multi-turn interactions, ensuring compatibility and ease of use.
Training Details
The model was trained for 2 epochs with a batch size of 128 and a sequence length of 4096. It utilized a learning rate of 2e-5 (Cosine) and an Any Precision AdamW Optimizer, with bf16 precision.
Good For
- Applications requiring robust multi-turn conversational AI.
- Tasks that benefit from improved reasoning and instruction following.
- Developers familiar with the Llama 2 ecosystem seeking an enhanced chat-optimized variant.