rishiraj/CatPPT-base
rishiraj/CatPPT-base is a 7 billion parameter chat model created by rishiraj through merging openchat and neuralchat models using Gradient SLERP. This model is notable for being the top-performing 7B model on the Open LLM Leaderboard that is certified free from evaluation data contamination. It excels in general chat applications, offering strong performance across various benchmarks including ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, and GSM8K.
Loading preview...
CatPPT: A Contamination-Free 7B Chat Model
CatPPT is a 7 billion parameter chat model developed by rishiraj, created by merging the openchat and neuralchat models using the Gradient SLERP method. This model stands out as the highest-ranked 7B chat model on the Open LLM Leaderboard that has been verified to be free from evaluation data contamination, ensuring reliable and unbiased performance metrics.
Key Capabilities & Performance
- Top-tier 7B Performance: Achieves an average score of 72.32 on the Open LLM Leaderboard, outperforming other 7B models like Intel/neural-chat-7b-v3-3 and openchat/openchat-3.5-1210.
- Robust Benchmark Scores: Demonstrates strong results across diverse tasks:
- ARC: 68.09
- HellaSwag: 86.69
- MMLU: 65.16
- TruthfulQA: 61.55
- Winogrande: 81.61
- GSM8K: 70.81
- Contamination-Free: A primary differentiator is its verified absence of evaluation data contamination, providing a trustworthy assessment of its capabilities.
Training Details
- Trained between December 15th and 17th, 2023.
- Utilized a learning rate of 2e-05, a batch size of 4, and 128 gradient accumulation steps over 1 epoch.
Ideal Use Cases
- General Chat Applications: Suitable for conversational AI where high performance and reliability are crucial.
- Benchmarking & Research: An excellent choice for researchers and developers seeking a strong 7B baseline model with verified contamination-free evaluation.