ewof/koishi-mini-vicuna-mistral-7b
The ewof/koishi-mini-vicuna-mistral-7b is a 7 billion parameter language model, fine-tuned by ewof, based on the Mistral-7B-v0.1 architecture. This model was trained using axolotl on a subset of the Koishi dataset, incorporating data from sources like Dolly, HH-RLHF, and Wizard Evol. It is optimized for general conversational tasks, leveraging a LoRA tune for efficient adaptation.
Loading preview...
Model Overview
The ewof/koishi-mini-vicuna-mistral-7b is a 7 billion parameter language model derived from the mistralai/Mistral-7B-v0.1 base model. It has undergone a rank 8 LoRA (Low-Rank Adaptation) fine-tuning process.
Training Details
This model was trained using the axolotl framework on a GPU cluster provided by Arc Compute. The training utilized a subset of the Koishi dataset (specifically commit 6e675d1) for one epoch. The training data incorporated various datasets, including:
asssdollyhh-rlhfwizard evol
Prompting Format
For optimal interaction, the model expects a specific prompting structure:
### System:
### Instruction:
### Response:Key Characteristics
- Base Model: Mistral-7B-v0.1
- Fine-tuning Method: Rank 8 LoRA
- Training Data: Subset of Koishi dataset, including diverse instruction-following and conversational data.
Use Cases
This model is suitable for general-purpose conversational AI applications, instruction-following tasks, and scenarios where a compact yet capable Mistral-based model is desired.