FreedomIntelligence/openPangu-Embedded-7B-V1.1
The FreedomIntelligence openPangu-Embedded-7B-V1.1 is a 7 billion parameter dense language model, natively trained from scratch on Ascend NPU with approximately 25T tokens. It features a 32K context length and incorporates a fast-slow thinking fusion mechanism with adaptive switching capabilities. This model is optimized for general, mathematical, and code-related tasks, demonstrating strong performance across various benchmarks.
Loading preview...
Overview
FreedomIntelligence/openPangu-Embedded-7B-V1.1 is a 7 billion parameter (excluding embedding) dense large language model, uniquely trained from scratch on Ascend NPUs. It has been pre-trained on an extensive dataset of approximately 25 trillion tokens and features a native context length of 32,768 tokens. A key innovation of this model is its fast-slow thinking fusion mechanism, which allows for adaptive switching between different reasoning modes.
Key Capabilities & Features
- Adaptive Thinking: The model can dynamically switch between "slow thinking" for complex tasks and "fast thinking" for simpler ones, optimizing output length without significantly impacting accuracy. This is achieved by appending
/auto_thinkor/no_thinkto user input. - Robust Architecture: Utilizes a dense architecture with 34 layers, a hidden dimension of 12800, and Grouped Query Attention (GQA) with 32 Q heads and 8 KV heads.
- Extensive Vocabulary: Features a vocabulary size of 153k.
- Strong Performance: Achieves competitive results across various benchmarks, including:
- General Abilities: MMLU-Pro (72.81), CMMLU (72.18), C-Eval (83.33), GPQA-Diamond (73.74).
- Mathematical Reasoning: MATH-500 (96.00), AIME24 (79.02), AIME25 (70.21).
- Code Generation: LiveCodeBench (58.27), MBPP+ (75.66).
When to Use This Model
This model is particularly well-suited for applications requiring efficient and adaptable reasoning, especially in environments leveraging Ascend NPUs. Its adaptive thinking mechanism makes it ideal for scenarios where both high accuracy on complex problems and concise outputs for simpler queries are desired. It performs strongly in general language understanding, mathematical problem-solving, and code generation tasks.