ArcOffical/PiCo-1B
PiCo-1B by ArcOffical is a 1.46 billion parameter dense transformer model optimized for reasoning and knowledge tasks. Despite its compact size, it achieves competitive performance across various benchmarks, notably excelling in science reasoning with top ranks on ARC-Challenge and ARC-Easy. The model features a 32768 token context length and is designed for efficient deployment in applications requiring strong analytical capabilities.
Loading preview...
PiCo-1B: A Compact Reasoning-Optimized Language Model
PiCo-1B, developed by ArcOffical, is a 1.46 billion parameter dense transformer model trained from scratch, utilizing the Qwen 2 1.5B tokenizer. It stands out for its strong performance in reasoning and knowledge tasks, particularly within the 1B-2B parameter range.
Key Capabilities & Performance Highlights
- Science Reasoning: Achieves best-in-class performance, ranking #1 on both ARC-Challenge (69.2%) and ARC-Easy (85.56%) among 1B-2B models.
- General Knowledge: Ranks in the top 3 on MMLU with a score of 53.93%, outperforming many larger models in its size class.
- Coding Ability: Demonstrates strong HumanEval performance (39.63%), placing it in the top 4 for its size.
- Truthfulness: Ranks in the top 5 on TruthfulQA (39.3%), indicating reliable factual output.
- Compact Size: Offers competitive performance despite its small parameter count, making it efficient for deployment.
Areas for Improvement
- Commonsense Reasoning: HellaSwag score (49.4%) indicates room for improvement compared to modern 1.5B+ models.
- Mathematical Reasoning: GSM8K performance (29.33%) is solid but not top-tier.
When to Use PiCo-1B
This model is ideal for applications requiring strong analytical and knowledge-based reasoning, especially in scientific domains, where a compact and efficient model is preferred. Its competitive coding and general knowledge capabilities also make it suitable for a range of general-purpose tasks where larger models might be overkill.