NextCoder-32B: Advanced Code Editing LLM

NextCoder-32B is a 32.5 billion parameter causal language model from Microsoft, part of the NextCoder series, specifically designed for robust code editing. Built upon the Qwen2.5-Coder Instruct variants, this model leverages a novel Selective Knowledge Transfer (SeleKT) finetuning methodology to achieve its specialized capabilities.

Key Capabilities & Features

• Exceptional Code Editing: Achieves significant performance gains in code editing benchmarks, showing up to a 44% improvement over its base model on Aider-Polyglot and performing on par with GPT-4o.
• Maintained Generalizability: The SeleKT finetuning method ensures that the model's general language understanding and generation capabilities are not compromised while specializing in code editing.
• Long Context Support: Supports a context window of up to 32,768 tokens, enabling it to handle large codebases and complex editing scenarios.
• Robust Architecture: Utilizes a transformer architecture with RoPE, SwiGLU, RMSNorm, and Attention QKV bias.

Performance Highlights

NextCoder-32B demonstrates strong performance across various code editing benchmarks:

• HUMANEVALFIX: 88.9
• CANITEDIT: 62.4
• AIDER: 74.7
• POLYGLOT: 23.6

These results highlight its superior performance in code editing compared to its base QwenCoder-2.5-32B model and other variants. For detailed evaluation, refer to the official paper.

Ideal Use Cases

• Automated Code Refactoring: Efficiently refactor and improve existing code.
• Bug Fixing: Identify and correct errors in code with high accuracy.
• Code Modernization: Adapt code to new standards or frameworks.
• Developer Tooling: Integrate into IDEs or CI/CD pipelines for intelligent code suggestions and modifications.