Shuibai12138/Open-Dcoder-0.5B-baseline-mdm-step2000

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Nov 19, 2025License:mitArchitecture:Transformer Open Weights Featherless Exclusive Warm

The Shuibai12138/Open-Dcoder-0.5B-baseline-mdm-step2000 is a 0.5 billion parameter masked diffusion language model, based on the Open-Dcoder architecture, and continued for 2,000 steps using an absorbing-only masked-diffusion objective. It is trained on a subset of the nvidia/Nemotron-Pretraining-SFT-v1 dataset, specifically Nemotron-SFT-Code. This model is designed for tasks requiring bidirectional attention and is intended for use with a specialized diffusion Qwen2 implementation rather than standard causal language model loading.

Loading preview...

Open-Dcoder-0.5B-baseline-mdm-step2000 Overview

This model is a 0.5 billion parameter masked diffusion language model (MDLM) derived from the fredzzp/open-dcoder-0.5B base. It represents a baseline version from the Corrective Diffusion Language Models research (NeurIPS 2026) and was further trained for 2,000 steps using a standard absorbing-only masked-diffusion objective.

Key Characteristics

  • Architecture: Based on the Open-Dcoder framework, utilizing a Qwen2 implementation with bidirectional attention.
  • Training Data: Continued training on the Nemotron-SFT-Code subset of the nvidia/Nemotron-Pretraining-SFT-v1 dataset.
  • Objective: Trained with a masked-diffusion objective, differing from mixture-based approaches.
  • Specialized Usage: Requires a specific evaluation pipeline from the associated code repository for correct loading and inference, as standard AutoModelForCausalLM loading will produce incorrect outputs due to its bidirectional nature.

Intended Use Cases

This model is suitable for research and development in:

  • Exploring masked diffusion language models and their performance characteristics.
  • Tasks that benefit from bidirectional context, such as code completion, infilling, or correction, when used with the appropriate diffusion-specific inference setup.
  • Comparative studies against other diffusion language models, particularly those with different training objectives like Shuibai12138/Open-Dcoder-0.5B-mixture-mdm-step2000.

Finetunes
1 models