CollectionStudio/AFM-4.5B-Base
AFM-4.5B-Base is a 4.5 billion parameter instruction-tuned model developed by Arcee.ai, trained on 8 trillion tokens with a focus on mathematical reasoning and code generation. It features a transformer decoder-only architecture with grouped query attention and ReLU^2 activation functions for efficiency and performance. This model is designed for enterprise-grade applications across various deployment environments, excelling in tasks requiring robust reasoning and code generation capabilities.
Loading preview...
AFM-4.5B-Base Overview
AFM-4.5B-Base is a 4.5 billion parameter instruction-tuned model developed by Arcee.ai, designed for enterprise-grade performance from cloud to edge. The model's base version was trained on an extensive 8 trillion token dataset, which included 6.5 trillion tokens of general pretraining data and an additional 1.5 trillion tokens specifically focused on enhancing mathematical reasoning and code generation. Following pretraining, it underwent supervised fine-tuning on high-quality instruction datasets and further refinement through reinforcement learning based on verifiable rewards and human preferences.
Key Capabilities
- Robust Performance: Engineered for enterprise-grade applications across diverse deployment environments.
- Enhanced Reasoning: Specialized midtraining data focused on mathematical reasoning.
- Strong Code Generation: Dedicated training for improved code generation capabilities.
- Efficient Architecture: Incorporates grouped query attention and ReLU^2 activation functions for performance and efficiency.
- High-Quality Data: Developed in collaboration with DatologyAI, utilizing proprietary algorithms for data curation to ensure strong real-world performance.
Good For
- Applications requiring strong mathematical reasoning.
- Code generation tasks.
- Enterprise deployments where robust and efficient performance is critical.
- Use cases benefiting from a model trained on a vast, high-quality dataset with specific domain focus.