ludis/tsukasa-limarp-7b

Overview

ludis/tsukasa-limarp-7b is a 7 billion parameter language model built upon the Llama-2-7b-hf architecture. It has undergone a multi-stage fine-tuning process, primarily focusing on enhancing its conversational and role-playing capabilities.

Training Details

The model's training involved several distinct phases:

• Initial Tuning: Started with the llama-2-7b-hf base model.
• First Fine-tune: Tuned for one epoch on commit de693ac of the koishi dataset.
• Second Fine-tune: Further tuned for one epoch on commit 36fc235 of the pippa metharme dataset.
• Primary Fine-tune: The final and most significant tuning involved two epochs on Version 2023-09-03 of the LimaRP dataset. Notably, specific subsets (ponyville, lolicit, all the fallen, and eka's portal) were excluded from the LimaRP training data.

Prompting Structure

This model is designed to interpret prompts using a three-role token system, facilitating structured conversations:

• <|system|>: Used for injecting background or out-of-channel information.
• <|user|>: Designates user input.
• <|model|>: Indicates where the model should generate its response.

These tokens can be chained to maintain conversation history, allowing for complex and multi-turn interactions. Recommended prompting and generation settings are available here.