BeaverAI/Cream-Phi-3-14B-v1a
BeaverAI/Cream-Phi-3-14B-v1a is a 14.7 billion parameter language model, fine-tuned from Phi-3 Medium, specifically optimized for roleplaying scenarios. This model focuses on generating strong roleplay formatting and straightforward replies, primarily for SFW content. It is designed to provide a specialized experience for interactive narrative generation.
Loading preview...
Model Overview
BeaverAI/Cream-Phi-3-14B-v1a is a 14.7 billion parameter model, fine-tuned from Phi-3 Medium, with a context length of 32768 tokens. Its primary focus is on roleplaying (RP), aiming to produce well-formatted and direct responses within interactive narrative contexts. The model was trained for approximately one hour using 8x H100 80GB SXM GPUs.
Key Capabilities
- Strong Roleplay Formatting: Excels at structuring responses in a manner suitable for roleplaying interactions.
- Direct Replies: Tends to generate concise and straightforward replies to player characters.
- SFW Focus: Optimized for mainly safe-for-work content, with noted limitations when content becomes "moist."
Training Details
The model utilized a lora adapter with lora_r: 128 and lora_alpha: 16. Training involved the Undi95/andrijdavid_roleplay-conversation-sharegpt dataset, with num_epochs: 2 and a learning_rate: 0.0001. It leveraged flash_attention and sample_packing for efficiency.
Intended Use Cases
This model is best suited for applications requiring a specialized roleplaying agent that can maintain consistent formatting and provide direct, SFW-oriented responses. Developers interested in fine-tuning Phi-3 for similar interactive narrative tasks may also find its training details valuable.