ZephRP-m7b: Roleplaying Focused Language Model

ZephRP-m7b is a 7 billion parameter model built upon the Mistral-7B-v0.1 architecture. It represents a strategic merge between HuggingFaceH4/zephyr-7b-alpha and a PEFT adapter fine-tuned using the LimaRP dataset.

Key Capabilities & Features

• Enhanced Roleplaying: Specifically engineered to combine the superior knowledge and instruction-following of Zephyr with the message length instruction training and stylistic elements from LimaRPv3.
• Alpaca Instruction Format: Utilizes the Alpaca instruction format from LimaRP v3 for structured roleplaying chats, allowing for detailed character and scenario definitions.
• Granular Message Length Control: Features a unique length modifier system (micro to unlimited) that can be appended to response instructions, directly influencing the verbosity of the model's output. medium is the recommended starting length.

Intended Use Cases

• Interactive Roleplaying: Ideal for applications requiring dynamic, character-driven conversational agents.
• Creative Storytelling: Suitable for generating narrative segments within defined personas and scenarios.

Limitations

• Exhibits biases similar to niche roleplaying forums and the base model.
• Not intended for factual information or advice.

Training Details

The LimaRP PEFT adapter was trained using axolotl as an 8-bit LoRA on the original mistralai/Mistral-7B-v0.1 model over 2 epochs.