Shuibai12138/Open-Dcoder-0.5B-CDLM-OpenCodeInstruct

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 28, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

Shuibai12138/Open-Dcoder-0.5B-CDLM-OpenCodeInstruct is a 0.5 billion parameter code generation model developed by Shuibai12138, based on the fredzzp/open-dcoder-0.5B architecture. It was fine-tuned for 2,000 steps on the nvidia/OpenCodeInstruct dataset using a Corrective Diffusion Language Model (CDLM) objective, which involves absorbing corruption and uniform token replacement. This model excels at code generation and correction tasks, demonstrating improved performance over its MDLM control counterpart on CRB and HumanEval benchmarks.

Loading preview...

Open-Dcoder-0.5B-CDLM-OpenCodeInstruct Overview

This model is a 0.5 billion parameter code generation model developed by Shuibai12138, fine-tuned from the fredzzp/open-dcoder-0.5B base model. It utilizes a Corrective Diffusion Language Model (CDLM) objective, which involves both absorbing corruption and a 10% uniform replacement of visible tokens during training, with a cross-entropy term on replaced positions.

Key Capabilities & Training

  • Corrective Objective: Trained with a CDLM objective on the nvidia/OpenCodeInstruct dataset for 2,000 steps, focusing on code correction and generation.
  • Performance: Achieves a Pass@1 of 0.2196 on CRB (T=1) and 0.2165 on HumanEval (vanilla decoding), outperforming its matched MDLM control model (Pass@1 T=1 CRB: 0.1401).
  • Context Length: Supports a context length of 32768 tokens.
  • Research Context: While not a direct model from the Corrective Diffusion Language Models (NeurIPS 2026) paper, it uses the same code and hyperparameters on a public dataset to allow for reproducible research and comparison.

Usage and Limitations

This model is a masked diffusion language model with bidirectional attention. It should not be loaded with AutoModelForCausalLM as this will lead to incorrect outputs. Evaluation and usage are intended via the provided launchers in the CDLM code repository, which correctly handle its diffusion Qwen2 implementation.

Licensing

The model weights are licensed under MIT. The base model, fredzzp/open-dcoder-0.5B, is under Apache License 2.0, and the training data, nvidia/OpenCodeInstruct, is CC BY 4.0.

Finetunes
1 models