Kimiko-Mistral-7B-fp16 Overview

This repository provides the fp16 PyTorch format of nRuaif's Kimiko-Mistral-7B model, a fine-tuned version of the original Mistral-7B-v0.1. The model was trained on the Kimiko dataset, distinct from the v3 dataset, and represents a finetuning experiment on a new 7B model architecture.

Key Capabilities

• Roleplay and Assistant Tasks: The model is specifically intended for use in roleplaying scenarios or as a general AI assistant.
• FP16 Format: Provided in float16 PyTorch format, optimized for GPU inference and further conversions.
• Mistral-7B Base: Built upon the robust Mistral-7B-v0.1 architecture.

Training Details

The model was trained with a learning rate of 0.00005, a batch size of 4 (total train batch size 64 with gradient accumulation), and an Adam optimizer. It underwent 2 epochs of training, achieving a final validation loss of 2.1173. The prompt template used during training follows a Vicuna-Short format.

Available Formats

In addition to this fp16 version, TheBloke provides various quantized versions for different inference needs, including AWQ, GPTQ, and GGUF models for CPU+GPU inference.