CatPPT: A Contamination-Free 7B Chat Model
CatPPT is a 7 billion parameter chat model developed by rishiraj, created by merging the openchat and neuralchat models using the Gradient SLERP method. This model stands out as the highest-ranked 7B chat model on the Open LLM Leaderboard that has been verified to be free from evaluation data contamination, ensuring reliable and unbiased performance metrics.
Key Capabilities & Performance
- Top-tier 7B Performance: Achieves an average score of 72.32 on the Open LLM Leaderboard, outperforming other 7B models like Intel/neural-chat-7b-v3-3 and openchat/openchat-3.5-1210.
- Robust Benchmark Scores: Demonstrates strong results across diverse tasks:
- ARC: 68.09
- HellaSwag: 86.69
- MMLU: 65.16
- TruthfulQA: 61.55
- Winogrande: 81.61
- GSM8K: 70.81
- Contamination-Free: A primary differentiator is its verified absence of evaluation data contamination, providing a trustworthy assessment of its capabilities.
Training Details
- Trained between December 15th and 17th, 2023.
- Utilized a learning rate of 2e-05, a batch size of 4, and 128 gradient accumulation steps over 1 epoch.
Ideal Use Cases
- General Chat Applications: Suitable for conversational AI where high performance and reliability are crucial.
- Benchmarking & Research: An excellent choice for researchers and developers seeking a strong 7B baseline model with verified contamination-free evaluation.