flammenai/Mahou-1.2-llama3-8B is an 8 billion parameter language model based on Meta Llama-3-8B, specifically fine-tuned for conversational and roleplay applications. Developed by flammenai, this model aims to be production-ready for interactive dialogue systems. It utilizes a context length of 8192 tokens and is optimized for engaging in dynamic, character-based interactions.
Loading preview...
Overview
Mahou-1.2-llama3-8B is an 8 billion parameter model developed by flammenai, fine-tuned from Meta Llama-3-8B. Its primary objective is to serve as a production-ready conversational and roleplay LLM, with future iterations planned to incorporate flammen.ai's proprietary conversational data. The model was fine-tuned using an A100 GPU on Google Colab, employing LoRA configuration and Direct Preference Optimization (DPO) for training.
Key Capabilities
- Conversational and Roleplay Focus: Specifically designed and optimized for engaging in dynamic, character-based conversations.
- ChatML Format: Trained to use the ChatML format, ensuring compatibility with common chat interfaces and structured dialogue.
- Llama 3 Base: Leverages the robust architecture and capabilities of the Meta Llama-3-8B model.
- Performance Benchmarks: Achieves an average score of 72.19 on the Open LLM Leaderboard, with specific scores including 69.80 on AI2 Reasoning Challenge, 84.65 on HellaSwag, and 68.43 on MMLU.
Usage and Configuration
Users should configure their environment to use ChatML for the context template and enable Instruct Mode. Specific stopping strings ["<", "|", "<|", "\n"] are recommended for optimal performance. The model is licensed under the META LLAMA 3 COMMUNITY LICENSE AGREEMENT.