Hugging Face Overview
ZephRP-m7b: Roleplaying Focused Language Model
ZephRP-m7b is a 7 billion parameter model built upon the Mistral-7B-v0.1 architecture. It represents a strategic merge between HuggingFaceH4/zephyr-7b-alpha and a PEFT adapter fine-tuned using the LimaRP dataset.
Key Capabilities & Features
- Enhanced Roleplaying: Specifically engineered to combine the superior knowledge and instruction-following of Zephyr with the message length instruction training and stylistic elements from LimaRPv3.
- Alpaca Instruction Format: Utilizes the Alpaca instruction format from LimaRP v3 for structured roleplaying chats, allowing for detailed character and scenario definitions.
- Granular Message Length Control: Features a unique length modifier system (
microtounlimited) that can be appended to response instructions, directly influencing the verbosity of the model's output.mediumis the recommended starting length.
Intended Use Cases
- Interactive Roleplaying: Ideal for applications requiring dynamic, character-driven conversational agents.
- Creative Storytelling: Suitable for generating narrative segments within defined personas and scenarios.
Limitations
- Exhibits biases similar to niche roleplaying forums and the base model.
- Not intended for factual information or advice.
Training Details
The LimaRP PEFT adapter was trained using axolotl as an 8-bit LoRA on the original mistralai/Mistral-7B-v0.1 model over 2 epochs.