LGAI-EXAONE/EXAONE-4.0-32B
LGAI-EXAONE/EXAONE-4.0-32B is a 32 billion parameter language model developed by LG AI Research, integrating both Non-reasoning and Reasoning modes for diverse applications. It features a hybrid attention scheme and QK-Reorder-Norm for enhanced performance, supporting agentic tool use and multilingual capabilities across English, Korean, and Spanish. This model is optimized for complex problem-solving and high-performance scenarios, offering a 131,072 token context length.
Loading preview...
EXAONE 4.0-32B Overview
EXAONE 4.0-32B is a 32 billion parameter large language model developed by LG AI Research, designed to bridge the gap between general usability and advanced reasoning. It uniquely integrates a Non-reasoning mode for general tasks and a Reasoning mode for complex problem-solving, building upon previous EXAONE versions. The model also incorporates essential features for the agentic AI era, including agentic tool use capabilities.
Key Architectural Innovations
- Hybrid Attention: The 32B model utilizes a hybrid attention scheme, combining local (sliding window) and global (full) attention in a 3:1 ratio, without RoPE for global attention to improve context understanding.
- QK-Reorder-Norm: This new normalization approach applies LayerNorm directly to attention and MLP outputs and RMS normalization after Q and K projection, leading to better performance on downstream tasks.
Capabilities and Performance
EXAONE 4.0-32B demonstrates strong performance across various benchmarks in both reasoning and non-reasoning modes. It excels in:
- Reasoning: Achieves high scores on MMLU-Redux (92.3), AIME 2025 (85.3), and HMMT Feb 2025 (72.9) in reasoning mode, outperforming many models in its size class.
- Multilinguality: Extends support to English, Korean (KMMLU-Pro 67.7, KMMLU-Redux 72.7), and Spanish (MMMLU 85.6, MATH500 95.8), making it suitable for diverse linguistic applications.
- Agentic Tool Use: Shows competitive results on BFCL-v3 (63.9) and Tau-Bench, indicating robust tool-calling capabilities.
- Long Context: Features an impressive context length of 131,072 tokens, enabling processing of extensive inputs.
Usage Guidelines
- Non-reasoning mode: Recommended to use with
temperature<0.6for optimal performance. - Reasoning mode: Best used with
temperature=0.6andtop_p=0.95.presence_penalty=1.5can mitigate degeneration.
This model is a versatile choice for applications requiring both general language understanding and advanced problem-solving with tool integration, particularly in multilingual contexts.