zarakiquemparte/pygmalion-lrp-grad-l2-7b
The zarakiquemparte/pygmalion-lrp-grad-l2-7b model is a 7 billion parameter language model based on Pygmalion 2 7B, fine-tuned for fictional writing and roleplay. It integrates a custom LimaRP v1 Lora, optimized for the Metharme prompt format. This model is specifically designed for entertainment purposes, generating character-driven conversational text.
Loading preview...
Overview
This model, pygmalion-lrp-grad-l2-7b, is a 7 billion parameter language model built upon the Pygmalion 2 7B base. It incorporates a custom LimaRP v1 Lora, which was merged using a specific script to enhance its capabilities. The fine-tuning process utilized the LimaRP dataset, converted to the Metharme prompt format, and was trained with axolotl.
Key Capabilities
- Roleplay Optimization: Specifically fine-tuned for fictional writing and character-driven roleplay scenarios.
- Metharme Prompt Format: Designed to work effectively with the Metharme prompt structure, utilizing
<|system|>,<|user|>, and<|model|>tokens for conversational flow. - Custom Lora Integration: Merges a customized LimaRP v1 Lora to specialize its output for interactive storytelling.
Intended Use and Limitations
This model's primary and sole intended use-case is fictional writing for entertainment purposes. It is explicitly not fine-tuned for safety or harmlessness and may produce socially unacceptable or offensive text due to its training data, which includes profanity and lewd content. Users should be aware that outputs may also be factually incorrect or misleading. Any use outside of fictional entertainment is considered out of scope.
Training Details
The Lora was fine-tuned on a single NVIDIA L40 GPU, taking approximately 1 hour and 45 minutes. Key hyperparameters included lora_r: 8, lora_alpha: 16, and a learning rate of 0.000065 over 3 epochs.