flammenai/Mahou-1.0-llama3-8B
flammenai/Mahou-1.0-llama3-8B is an 8 billion parameter Llama 3-based causal language model developed by flammenai, fine-tuned for conversational and roleplay applications. This model is an experimental iteration focused on building a production-ready LLM for interactive dialogue. It was fine-tuned using Direct Preference Optimization (DPO) with a LoRA configuration, supporting a context length of 8192 tokens.
Loading preview...
Mahou-1.0-llama3-8B Overview
Mahou-1.0-llama3-8B is an 8 billion parameter language model from flammenai, built upon the Meta Llama 3 architecture. This iteration is an experimental attempt to create a production-ready conversational and roleplay LLM. It was fine-tuned using Direct Preference Optimization (DPO) on an A100 GPU within Google Colab.
Key Characteristics
- Base Model: Meta Llama-3-8B.
- Fine-tuning Method: Utilizes Direct Preference Optimization (DPO) with a LoRA configuration (r=16, lora_alpha=16).
- Context Length: Supports a maximum context length of 8192 tokens.
- Development Focus: Specifically aimed at improving conversational and roleplay capabilities, with future versions planned to be iteratively fine-tuned on flammen.ai's conversational data.
Current Status
It is important to note that this model is currently experimental and acknowledged to be significantly flawed by its developers. Users should be aware of its developmental stage and potential limitations.
Licensing
This model operates under the META LLAMA 3 COMMUNITY LICENSE AGREEMENT.
Top 3 parameter combinations used by Featherless users for this model. Click a tab to see each config.