arcee-ai/AFM-4.5B-Base
AFM-4.5B-Base is a 4.5 billion parameter instruction-tuned causal language model developed by Arcee.ai. It was trained on 8 trillion tokens, with a specific focus on mathematical reasoning and code generation during midtraining. This model incorporates grouped query attention and ReLU^2 activation functions for enhanced performance and efficiency, making it suitable for enterprise-grade deployment from cloud to edge.
Loading preview...
AFM-4.5B-Base Overview
AFM-4.5B-Base is a 4.5 billion parameter instruction-tuned model from Arcee.ai, designed for robust performance across various deployment environments. The model's development prioritized high-quality data, leveraging DatologyAI's curation pipeline which includes model-based quality filtering, embedding-based curation, and synthetic data generation.
Key Capabilities & Features
- Optimized Training: Trained on an extensive 8 trillion token dataset, with 1.5 trillion tokens specifically focused on mathematical reasoning and code generation during midtraining.
- Instruction-Tuned: Underwent supervised fine-tuning on high-quality instruction datasets, followed by reinforcement learning using verifiable rewards and human preference.
- Efficient Architecture: Utilizes a modified transformer decoder-only design, incorporating grouped query attention for improved inference efficiency and ReLU^2 activation functions to enable sparsification while maintaining performance.
- Data Quality Focus: Developed in collaboration with DatologyAI to ensure a highly curated dataset, supporting strong real-world performance.
Good For
- Enterprise Applications: Designed for enterprise-grade performance, suitable for deployment in diverse environments from cloud to edge.
- Mathematical Reasoning: Enhanced focus during midtraining makes it well-suited for tasks requiring mathematical reasoning.
- Code Generation: Specific training on code generation data positions it as a strong candidate for coding-related tasks.