arcee-ai/raspberry-3B
arcee-ai/raspberry-3B is an experimental 3 billion parameter model developed by arcee-ai, heavily optimized for reasoning tasks. It utilizes the ChatML prompt format and is not intended for production use. The model demonstrates specific performance metrics on reasoning benchmarks, including 19.53 on BBH (3-Shot) and 7.63 on MATH Lvl 5 (4-Shot). It is best suited for research and development in advanced reasoning capabilities.
Loading preview...
Model Overview
arcee-ai/raspberry-3B is an experimental language model developed by arcee-ai, specifically optimized for reasoning tasks. This model is designed for research and development purposes rather than production environments, focusing on pushing the boundaries of reasoning capabilities within a smaller parameter count.
Key Characteristics
- Reasoning Optimization: Heavily optimized for complex reasoning tasks, making it a suitable candidate for exploring advanced AI logic.
- ChatML Format: Utilizes the ChatML prompt format for interaction.
- Experimental Nature: Positioned as an experimental model, indicating ongoing development and a focus on specific research goals.
Performance Highlights
Evaluated on the Open LLM Leaderboard, raspberry-3B shows specific performance on reasoning benchmarks:
- Average Score: 15.40
- IFEval (0-Shot): 31.54
- BBH (3-Shot): 19.53
- MATH Lvl 5 (4-Shot): 7.63
- GPQA (0-shot): 3.69
- MuSR (0-shot): 9.41
- MMLU-PRO (5-shot): 20.60
Detailed evaluation results are available on the Open LLM Leaderboard.
Use Cases
This model is ideal for:
- Research: Investigating and developing new approaches to AI reasoning.
- Experimentation: Testing hypotheses related to model architecture and training for reasoning tasks.
It is explicitly noted as "not meant for production-use" due to its experimental focus.