IFM/K2-Horizon-MoVA-36B-A4B
K2-Horizon-MoVA-36B-A4B is a 36 billion parameter Mixture-of-Experts (MoE) model developed by IFM, featuring Mixture-of-Values Attention (MoVA) and running with 4 billion active parameters per token. It achieves frontier-class performance on agentic and reasoning benchmarks, outperforming many larger dense and MoE models. The model supports a native context length of 512K tokens and is fully open, with training data, code, and intermediate checkpoints publicly available. Its primary use case is advanced agentic tasks, reasoning, and coding, offering high performance with efficient active parameter usage.
Loading preview...
K2-Horizon-MoVA-36B-A4B Overview
K2-Horizon-MoVA-36B-A4B is a 36 billion parameter Mixture-of-Experts (MoE) model from IFM, distinguished by its Mixture-of-Values Attention (MoVA) architecture. It operates with only 4 billion active parameters per token, offering a highly efficient yet powerful solution. The model provides frontier-class results on agentic and reasoning benchmarks, often surpassing open-weight dense and MoE models up to 15 times its size, and competes effectively with closed frontier models.
Key Capabilities
- Exceptional Efficiency: Achieves high performance with a sparse activation of only 4B parameters per token, making it resource-efficient for its capabilities.
- Extended Context Window: Features a native context length of 512,000 tokens, enabling processing of very long inputs and complex reasoning tasks.
- Strong Agentic and Reasoning Performance: Excels in benchmarks requiring agentic tool use (e.g., tau3-Banking: 26.8%) and agentic terminal use (Terminal-Bench 2.1: 58.6%), demonstrating robust reasoning abilities.
- Fully Open-Source: The model, its training data (IFM/TxT360-v2), and training code (GitHub) are publicly available, including intermediate checkpoints for research into capability evolution.
Good For
- Agentic Applications: Ideal for developing AI agents that require advanced tool use and complex decision-making.
- Long-Context Reasoning: Suitable for tasks demanding the processing and understanding of extensive documents or conversations due to its 512K token context.
- Research and Development: Its fully open nature, including intermediate checkpoints and training recipes, makes it an excellent choice for researchers exploring MoE architectures and training dynamics.
- Efficient High-Performance Tasks: When seeking high performance in reasoning and coding tasks without the computational overhead of larger dense models, K2-Horizon-MoVA-36B-A4B offers a compelling balance.