Model Overview

ewof/koishi-7b-qlora is a 7 billion parameter language model, fine-tuned from the mistralai/Mistral-7B-v0.1 base model. It leverages QLoRA (Quantized Low-Rank Adaptation) with a rank 16 LoRA tune applied to all modules, which were subsequently merged. The training was conducted using the axolotl framework on a 6x NVIDIA A40 GPU cluster, generously provided by Arc Compute.

Key Capabilities

• Conversational AI: Optimized for structured dialogue, utilizing specific tokens for system, user, and model roles.
• Multi-turn Interaction: Supports chaining of system, user, and model prompts to form comprehensive conversation histories.
• Efficient Fine-tuning: Built upon QLoRA, enabling efficient adaptation of a powerful base model.

Good For

• Dialogue Systems: Ideal for applications requiring clear role-based conversational structures.
• Chatbots: Suitable for developing interactive agents that distinguish between system instructions, user input, and model responses.
• Research in LoRA Fine-tuning: Provides a practical example of QLoRA application on a Mistral base model.