Overview

fhai50032/xLakeChat is a 7 billion parameter language model developed by fhai50032. It is a merged model, combining the strengths of xDAN-AI/xDAN-L1-Chat-RL-v1 and fhai50032/BeagleLake-7B-Toxic using senseable/WestLake-7B-v2 as its base model. The merge was performed using the dare_ties method, with specific weight and density parameters applied to the constituent models.

Performance Highlights

Evaluated on the Open LLM Leaderboard, xLakeChat demonstrates a solid average performance of 63.72. Key benchmark scores include:

• AI2 Reasoning Challenge (25-Shot): 62.37
• HellaSwag (10-Shot): 82.64
• MMLU (5-Shot): 59.32
• TruthfulQA (0-shot): 52.96
• Winogrande (5-shot): 74.74
• GSM8k (5-shot): 50.27

These results indicate a balanced capability across various tasks, including reasoning, common sense, and language understanding.

Usage and Configuration

The model is configured with float16 dtype and includes normalize and int8_mask parameters for the merge. It can be easily integrated into Python applications using the transformers library, with a provided example demonstrating how to set up a text generation pipeline for chat-based interactions. The model supports a context length of 4096 tokens.