icefog72/IceLemonTeaRP-32k-7b

IceLemonTeaRP-32k-7b: A Merged Model for Roleplay

IceLemonTeaRP-32k-7b is a 7 billion parameter language model developed by icefog72, specifically engineered to mitigate repetition issues prevalent in its predecessor, IceTeaRP-7b. This model was constructed using the SLERP merge method via mergekit, combining several base models including icefog72/Kunokukulemonchini-32k-7b and icefog72/Mixtral_AI_Cyber_3.m1-BigL.

Key Capabilities & Features

• Repetition Mitigation: Primary focus on improving text generation quality by reducing repetitive outputs, particularly beneficial for conversational and creative applications.
• Merged Architecture: Leverages the strengths of multiple foundational models to enhance overall performance and coherence.
• Context Length: Supports an 8192-token context window, allowing for more extended and detailed interactions.
• Prompt Template Flexibility: Designed to work with Alpaca and potentially ChatML prompt templates.
• Quantization Support: Includes measurement.json for EXL2 quantization, offering optimized versions for various bit-per-weight configurations (e.g., 4.0bpw, 4.2bpw, 6.5bpw, 8.0bpw).

Performance Highlights

Evaluations on the Open LLM Leaderboard indicate an average score of 70.43 across various benchmarks, including:

• HellaSwag (10-Shot): 86.53
• Winogrande (5-shot): 79.72
• AI2 Reasoning Challenge (25-Shot): 67.66
• MMLU (5-Shot): 64.51
• GSM8k (5-shot): 62.40

Good For

• Roleplay and Creative Writing: Its primary design goal of reducing repetition makes it well-suited for generating engaging and varied narrative content.
• Conversational AI: Can be used in applications requiring more natural and less repetitive dialogue.
• Experimentation with Merged Models: Provides a practical example of a model created through advanced merging techniques.