师承万象基础教育大模型（MuduoLLM）

传承木铎金声，智启教育未来
Inheriting Wisdom, Inspiring Future Education

[![GitHub](https://img.shields.io/badge/GitHub-MuduoLLM-blue)](https://github.com/ERC-ITEA/MuduoLLM)

# 简介 | Introduction 师承万象基础教育大模型（MuduoLLM）是北京师范大学和北京世纪好未来教育科技有限公司共同研发的首个紧扣新课标知识体系的基础教育语言大模型，确保所学知识内容与基础教育课程标准高度契合，精准对接学生核心素养培育与教师专业成长需求。在应用层面，基础教育大模型深度融合新课标核心知识和育人理念，具备知识理解型智能解题、启发引导式智能答疑、情境创设型智能出题和素养导向型教案生成等教育能力，从知识传授转向核心素养培育，助力培养全面发展时代新人。同时，师承万象基础教育大模型是当前性能表现较为突出的开源基础教育大模型之一，为开发者提供了可进一步优化的空间。 MuduoLLM is the first foundational educational large language model jointly developed by Beijing Normal University and TAL Education Group, specifically aligned with China's new curriculum standards. It ensures high fidelity to curriculum content, precisely addressing the dual needs of cultivating students' core competencies and supporting teachers’ professional growth. At the application level, MuduoLLM deeply embeds the core knowledge and educational philosophy of the new curriculum, enabling capabilities such as knowledge-based intelligent problem-solving, inquiry-guided Q&A, context-driven question generation, and competency-oriented lesson planning. By shifting the focus from rote knowledge transmission to holistic competency cultivation, the model supports the development of well-rounded talents for the new era. Furthermore, MuduoLLM stands out as one of the most capable open-source foundational models in the field of basic education, offering substantial potential for further optimization and secondary development by researchers and developers. ## 模型概述 | Model Overview - **Base Architecture**: [Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) - **Parameters**: 14 billion (14B) - **Training Data**: Approximately 1TB of general and educational domain text data, including question generation, Q&A, and lesson plans - **Training Methods**: - Domain-specific Pretraining: Injecting educational domain-specific corpora to enhance semantic understanding - Supervised Fine-Tuning (SFT): Targeted optimization for educational scenarios (question generation/Q&A/lesson plan generation) - Direct Preference Optimization (DPO): Improving generation accuracy and educational ethics compliance through expert-annotated preference data ## 训练环境 | Training Environment - **Hardware Configuration**: - Number of Servers: 4 - GPU Configuration: 8 NVIDIA A800-SXM4-80GB per server (32 total) - Single GPU Memory: 80GB - Interconnection: NVLink 4.0 (9.6TB/s bandwidth) - Parallel Strategy: Data Parallel + Tensor Parallel - **Software**: - Base Framework: - CUDA: 12.4 - PyTorch: 2.5.1+cu124 - Optimization Tools: - DeepSpeed: 0.15.4 (ZeRO-3 optimizer) - FlashAttention - Training Precision: bfloat16 mixed precision - Runtime Environment: Conda virtual environment + Weights & Biases monitoring - **Training Duration**: 10 days # 快速开始 | Quick Start ## 环境要求 | Requirements - Python 3.10 - PyTorch - transformers >= 4.37.0 ## 安装 | Installation ```bash # 克隆仓库 | Clone repository huggingface-cli download --resume-download ERC-ITEA/MuduoLLM --local-dir ./muduo-llm/ # 创建环境 | Create environment conda create --name muduo python=3.10 conda activate muduo # 安装依赖 | Install dependencies pip install transformers ``` ## 使用示例 | Usage Example ```python from transformers import AutoModelForCausalLM, AutoTokenizer # 加载模型和分词器 | Load model and tokenizer model_name = "MuduoLLM" model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained(model_name) # 准备输入 | Prepare input prompt = "Give me a short introduction to large language model." messages = [ {"role": "system", "content": "你是北京师范大学和好未来开发的人工智能语言模型，名为师承万象。可以回答问题、提供信息、进行对话并帮助解决问题。"}, {"role": "user", "content": prompt} ] # 生成回复 | Generate response text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) generated_ids = model.generate( **model_inputs, max_new_tokens=512 ) generated_ids = [ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) ] response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] ``` # 许可证 | License This project is licensed under the [Apache 2.0](https://opensource.org/licenses/Apache-2.0) License. This project is for research purposes only. The project developers are not responsible for any harm or loss caused by using this project (including but not limited to data, models, code, etc.). # 引用 | Citation ```bibtex @misc{muduollm2025, title={MuduoLLM: A High-Performance LLM for Intelligent Education Solutions}, author={MuduoLLM Contributors from BNU and TAL}, year={2025}, howpublished={\url{https://huggingface.co/ERC-ITEA/MuduoLLM}}, } ```