ewof/koishi-7b-qlora
The ewof/koishi-7b-qlora model is a 7 billion parameter language model, fine-tuned using QLoRA from the Mistral-7B-v0.1 base model. Trained by ewof, it utilizes a rank 16 LoRA adaptation across all modules. This model is specifically designed for conversational AI, employing distinct system, user, and model tokens to structure multi-turn interactions effectively.
Loading preview...
Model Overview
ewof/koishi-7b-qlora is a 7 billion parameter language model, fine-tuned from the mistralai/Mistral-7B-v0.1 base model. It leverages QLoRA (Quantized Low-Rank Adaptation) with a rank 16 LoRA tune applied to all modules, which were subsequently merged. The training was conducted using the axolotl framework on a 6x NVIDIA A40 GPU cluster, generously provided by Arc Compute.
Key Capabilities
- Conversational AI: Optimized for structured dialogue, utilizing specific tokens for system, user, and model roles.
- Multi-turn Interaction: Supports chaining of system, user, and model prompts to form comprehensive conversation histories.
- Efficient Fine-tuning: Built upon QLoRA, enabling efficient adaptation of a powerful base model.
Good For
- Dialogue Systems: Ideal for applications requiring clear role-based conversational structures.
- Chatbots: Suitable for developing interactive agents that distinguish between system instructions, user input, and model responses.
- Research in LoRA Fine-tuning: Provides a practical example of QLoRA application on a Mistral base model.