fhai50032/xLakeChat
fhai50032/xLakeChat is a 7 billion parameter language model created by fhai50032, formed by merging xDAN-AI/xDAN-L1-Chat-RL-v1 and fhai50032/BeagleLake-7B-Toxic with senseable/WestLake-7B-v2 as its base. Utilizing a DARE TIES merge method, it achieves an average score of 63.72 on the Open LLM Leaderboard. This model is designed for general chat applications, offering a 4096-token context length and demonstrating balanced performance across various reasoning and language understanding benchmarks.
Loading preview...
Overview
fhai50032/xLakeChat is a 7 billion parameter language model developed by fhai50032. It is a merged model, combining the strengths of xDAN-AI/xDAN-L1-Chat-RL-v1 and fhai50032/BeagleLake-7B-Toxic using senseable/WestLake-7B-v2 as its base model. The merge was performed using the dare_ties method, with specific weight and density parameters applied to the constituent models.
Performance Highlights
Evaluated on the Open LLM Leaderboard, xLakeChat demonstrates a solid average performance of 63.72. Key benchmark scores include:
- AI2 Reasoning Challenge (25-Shot): 62.37
- HellaSwag (10-Shot): 82.64
- MMLU (5-Shot): 59.32
- TruthfulQA (0-shot): 52.96
- Winogrande (5-shot): 74.74
- GSM8k (5-shot): 50.27
These results indicate a balanced capability across various tasks, including reasoning, common sense, and language understanding.
Usage and Configuration
The model is configured with float16 dtype and includes normalize and int8_mask parameters for the merge. It can be easily integrated into Python applications using the transformers library, with a provided example demonstrating how to set up a text generation pipeline for chat-based interactions. The model supports a context length of 4096 tokens.