IceLemonTeaRP-32k-7b: A Merged Model for Roleplay
IceLemonTeaRP-32k-7b is a 7 billion parameter language model developed by icefog72, specifically engineered to mitigate repetition issues prevalent in its predecessor, IceTeaRP-7b. This model was constructed using the SLERP merge method via mergekit, combining several base models including icefog72/Kunokukulemonchini-32k-7b and icefog72/Mixtral_AI_Cyber_3.m1-BigL.
Key Capabilities & Features
- Repetition Mitigation: Primary focus on improving text generation quality by reducing repetitive outputs, particularly beneficial for conversational and creative applications.
- Merged Architecture: Leverages the strengths of multiple foundational models to enhance overall performance and coherence.
- Context Length: Supports an 8192-token context window, allowing for more extended and detailed interactions.
- Prompt Template Flexibility: Designed to work with Alpaca and potentially ChatML prompt templates.
- Quantization Support: Includes
measurement.json for EXL2 quantization, offering optimized versions for various bit-per-weight configurations (e.g., 4.0bpw, 4.2bpw, 6.5bpw, 8.0bpw).
Performance Highlights
Evaluations on the Open LLM Leaderboard indicate an average score of 70.43 across various benchmarks, including:
- HellaSwag (10-Shot): 86.53
- Winogrande (5-shot): 79.72
- AI2 Reasoning Challenge (25-Shot): 67.66
- MMLU (5-Shot): 64.51
- GSM8k (5-shot): 62.40
Good For
- Roleplay and Creative Writing: Its primary design goal of reducing repetition makes it well-suited for generating engaging and varied narrative content.
- Conversational AI: Can be used in applications requiring more natural and less repetitive dialogue.
- Experimentation with Merged Models: Provides a practical example of a model created through advanced merging techniques.