Mahou-1.0-llama3-8B Overview

Mahou-1.0-llama3-8B is an 8 billion parameter language model from flammenai, built upon the Meta Llama 3 architecture. This iteration is an experimental attempt to create a production-ready conversational and roleplay LLM. It was fine-tuned using Direct Preference Optimization (DPO) on an A100 GPU within Google Colab.

Key Characteristics

• Base Model: Meta Llama-3-8B.
• Fine-tuning Method: Utilizes Direct Preference Optimization (DPO) with a LoRA configuration (r=16, lora_alpha=16).
• Context Length: Supports a maximum context length of 8192 tokens.
• Development Focus: Specifically aimed at improving conversational and roleplay capabilities, with future versions planned to be iteratively fine-tuned on flammen.ai's conversational data.

Current Status

It is important to note that this model is currently experimental and acknowledged to be significantly flawed by its developers. Users should be aware of its developmental stage and potential limitations.

Licensing

This model operates under the META LLAMA 3 COMMUNITY LICENSE AGREEMENT.