Pearl-7B-slerp: A Math-Optimized 7B Model
Pearl-7B-slerp is a 7.24 billion parameter language model developed by louisbrulenaudet, created through a Spherical Linear Interpolation (SLERP) merge of two base models: mlabonne/OmniBeagle-7B and WizardLM/WizardMath-7B-V1.1. This merging technique, SLERP, is chosen over traditional linear interpolation to preserve directional information and mitigate issues in high-dimensional spaces.
Key Capabilities
- Mathematical Proficiency: Achieves a strong 73.62 on the GSM8K benchmark, indicating a high capability for mathematical problem-solving.
- Balanced Performance: Demonstrates competitive scores across various benchmarks on the HuggingFace Open LLM Leaderboard, including ARC (68.00), HellaSwag (87.16), MMLU (64.04), TruthfulQA (62.35), and Winogrande (81.29).
- Efficient Interpolation: Utilizes SLERP to combine model weights, ensuring a constant rate of change and maintaining geometric properties, which is crucial for effective model merging.
Good for
- Mathematical Applications: Ideal for tasks and applications that require strong mathematical reasoning and problem-solving abilities.
- General-Purpose Use: Its balanced performance across multiple benchmarks makes it suitable for a variety of general language understanding and generation tasks where mathematical aptitude is a plus.
- Resource-Efficient Deployment: As a 7B parameter model, it offers a good balance between performance and computational resource requirements compared to larger models.