dphn/dolphin-2.9.3-llama-3-8b
dphn/dolphin-2.9.3-llama-3-8b is an 8 billion parameter language model fine-tuned from Meta-Llama-3-8B, designed for conversational AI and agentic tasks. It leverages a diverse dataset including ShareGPT, SystemChat, Orca-Math, and ToolBench, optimizing it for complex instruction following, coding, and tool use. This model is particularly suited for applications requiring robust dialogue capabilities and problem-solving across various domains.
Loading preview...
dphn/dolphin-2.9.3-llama-3-8b: A Fine-Tuned Llama 3 for Conversational AI
This model is a fine-tuned version of Meta's Llama-3-8B, developed by dphn, focusing on enhancing its capabilities for conversational agents and complex instruction following. It was trained using the Axolotl framework, indicating a structured approach to fine-tuning.
Key Capabilities & Training
The model's training regimen involved a comprehensive and diverse set of datasets, including:
- ShareGPT and SystemChat: For general conversational abilities and instruction adherence.
- dolphin-coder-translate and dolphin-coder-codegen: Specifically targeting code generation and translation tasks.
- m-a-p_Code-Feedback: Enhancing its understanding and application of code-related feedback.
- Orca-Math: Improving mathematical reasoning and problem-solving.
- ToolBench datasets (instruct, negative, react, tflan_cot): Crucial for developing tool-use capabilities and agentic reasoning, allowing the model to interact with external tools effectively.
Training was conducted over 3 epochs with a learning rate of 1e-05, utilizing a total batch size of 128 across 8 GPUs. The model's performance was monitored, showing a final validation loss of 0.5771.
Intended Use Cases
Given its training on a wide array of datasets, dphn/dolphin-2.9.3-llama-3-8b is well-suited for:
- Advanced Chatbots: Capable of handling complex dialogues and following intricate instructions.
- Code Generation & Translation: Assisting developers with programming tasks.
- Agentic Applications: Where the model needs to reason, plan, and potentially use tools to achieve goals.
- Mathematical Problem Solving: For tasks requiring numerical and logical reasoning.