Model Overview

arcee-ai/raspberry-3B is an experimental language model developed by arcee-ai, specifically optimized for reasoning tasks. This model is designed for research and development purposes rather than production environments, focusing on pushing the boundaries of reasoning capabilities within a smaller parameter count.

Key Characteristics

• Reasoning Optimization: Heavily optimized for complex reasoning tasks, making it a suitable candidate for exploring advanced AI logic.
• ChatML Format: Utilizes the ChatML prompt format for interaction.
• Experimental Nature: Positioned as an experimental model, indicating ongoing development and a focus on specific research goals.

Performance Highlights

Evaluated on the Open LLM Leaderboard, raspberry-3B shows specific performance on reasoning benchmarks:

• Average Score: 15.40
• IFEval (0-Shot): 31.54
• BBH (3-Shot): 19.53
• MATH Lvl 5 (4-Shot): 7.63
• GPQA (0-shot): 3.69
• MuSR (0-shot): 9.41
• MMLU-PRO (5-shot): 20.60

Detailed evaluation results are available on the Open LLM Leaderboard.

Use Cases

This model is ideal for:

• Research: Investigating and developing new approaches to AI reasoning.
• Experimentation: Testing hypotheses related to model architecture and training for reasoning tasks.

It is explicitly noted as "not meant for production-use" due to its experimental focus.