twnlp/ChineseErrorCorrector4-4B

ChineseErrorCorrector4-4B (CSRP) Overview

The twnlp/ChineseErrorCorrector4-4B model, developed by TW-NLP, is a 4-billion parameter model specifically designed for high-precision Chinese Grammatical Error Correction (CGEC) and Chinese Spelling Check (CSC). It introduces the CSRP (CPT → SFT → RL) three-stage training framework to overcome the common "over-correction bias" found in traditional LLM-based correction systems, where models tend to unnecessarily paraphrase correct text.

Key Capabilities & Differentiators

• Over-Correction Bias Mitigation: CSRP employs a structured curriculum to calibrate decision boundaries, ensuring the model only corrects actual errors.
  • Phase I: Balanced Continued Pre-training (CPT): Internalizes linguistic priors using 5.9M samples with an 8:2 mixture of general and correction-specific data.
  • Phase II: Rationale-Augmented SFT: Distills Chain-of-Thought reasoning paths to guide error diagnosis before correction.
  • Phase III: Efficiency-Aware Policy Alignment: Utilizes GRPO with a novel Efficiency-Aware Reward (EAR) to penalize unnecessary edits and reward precise corrections.
• State-of-the-Art Performance: Achieves new SOTA on both NACGEC (CGEC) and CSCD (CSC) benchmarks.
  • CGEC: Scores an $F_{0.5}$ of 50.99 on NACGEC, surpassing previous 14B models with significantly fewer parameters.
  • CSC: Achieves a Correction F1 of 59.61 on CSCD, outperforming GPT-4 in character-level correction.
• High Precision: Demonstrates a precision of 57.17% on CGEC, effectively minimizing false-positive rewrites.

Ideal Use Cases

This model is particularly well-suited for applications requiring highly accurate and precise Chinese text correction, where avoiding unnecessary changes to correct text is critical. It can be used for:

• Automated proofreading of Chinese text.
• Enhancing the quality of written Chinese content.
• Educational tools for Chinese language learners.