icefog72/Kunokukulemonchini-7b
icefog72/Kunokukulemonchini-7b is a 7 billion parameter language model created by icefog72 through a SLERP merge of grimjim/kukulemon-7B and Nitral-AI/Kunocchini-7b-128k-test. This model is configured to support an extended context window of approximately 32k tokens, making it suitable for tasks requiring longer input sequences. It achieves an average score of 69.61 on the Open LLM Leaderboard, demonstrating solid performance across various reasoning and language understanding benchmarks.
Loading preview...
Overview
Kunokukulemonchini-7b is a 7 billion parameter language model developed by icefog72. It is a merged model, created using the SLERP (Spherical Linear Interpolation) method, combining the strengths of two base models: grimjim/kukulemon-7B and Nitral-AI/Kunocchini-7b-128k-test.
Key Capabilities & Features
- Extended Context Window: The model's configuration was edited to achieve an approximate 32k context window, enhancing its ability to process and generate longer texts.
- Quantized Versions Available: Several EXL2 quantized versions are provided by Natkituwu, ranging from 3.5bpw to 8.0bpw, catering to users with varying VRAM constraints, including a 4.1bpw version for 6GB VRAM.
Performance
Evaluated on the Open LLM Leaderboard, Kunokukulemonchini-7b demonstrates competitive performance for its size:
- Average Score: 69.61
- HellaSwag (10-Shot): 86.31
- MMLU (5-Shot): 65.31
- GSM8k (5-Shot): 60.20
Use Cases
This model is suitable for applications requiring a 7B parameter model with an extended context window, particularly for tasks where a balance between performance and resource efficiency is desired. Its benchmark scores suggest proficiency in reasoning, common sense, and language understanding tasks.