openpangu/openPangu-Embedded-7B-V1.1
openPangu-Embedded-7B-V1.1 is a 7 billion parameter (excluding embedding) large language model developed by openpangu, trained from scratch on Ascend NPUs with approximately 25T tokens. It features a dense architecture with GQA attention and a 32K context length. This model is distinguished by its integrated fast and slow thinking capabilities with adaptive switching, optimizing for both accuracy and inference efficiency across various tasks.
Loading preview...
Overview
openPangu-Embedded-7B-V1.1 is a 7 billion parameter (non-embedding) large language model developed by openpangu, trained from scratch on Ascend NPUs. It was pre-trained on approximately 25 trillion tokens and features a dense architecture with 34 layers, a hidden dimension of 12800, and Grouped-Query Attention (GQA).
Key Capabilities
- Adaptive Fast and Slow Thinking: The model integrates and adaptively switches between fast and slow thinking modes. This allows it to significantly reduce average output chain-of-thought length for simpler tasks while maintaining accuracy comparable to pure slow-thinking models for complex tasks.
- Extended Context Length: Supports a native context length of 32,000 tokens.
- Multilingual Performance: Achieves strong results on Chinese benchmarks like CMMLU (72.18% Acc) and C-Eval (83.33% Acc) in adaptive mode.
- Mathematical Reasoning: Demonstrates high performance on mathematical benchmarks such as MATH-500 (96.00% Avg@1) and AIME24 (79.02% Avg@16) in adaptive mode.
- Code Generation: Shows solid capabilities in code tasks, scoring 58.27% Avg@2 on LiveCodeBench and 75.66% Avg@2 on MBPP+.
When to Use This Model
This model is particularly well-suited for applications requiring a balance between inference speed and accuracy, especially in scenarios where task complexity varies. Its adaptive thinking mechanism makes it efficient for general-purpose language tasks while retaining strong performance in specialized domains like mathematics and coding. It is designed for deployment on Ascend NPU hardware, with specific environment requirements including CANN 8.1.RC1 and Python 3.10.