interpretable-finetuning/qwen2.5-32b-unaliased

interpretable-finetuning/qwen2.5-32b-unaliased: Enhanced ChatML Termination

This model is a modified version of the Qwen/Qwen2.5-32B base, specifically engineered to resolve a critical issue with the reliable emission of ChatML turn markers, particularly <|im_end|>. The original base model exhibited a significant defect where it could not consistently generate this crucial token, leading to incomplete and rambling responses in conversational contexts.

Key Modifications and Improvements

Unlike the published Qwen/Qwen2.5-32B checkpoint, this version replaces the lm_head and embed_tokens rows corresponding to the <|im_start|> and <|im_end|> ChatML markers with those from Qwen/Qwen2.5-32B-Instruct. This targeted change, while keeping 769 of 771 tensors bit-identical to the original, dramatically improves the model's ability to terminate turns correctly. The tokenizer, vocabulary, token IDs, configuration, and architecture remain otherwise unchanged.

Measured Impact

Before this fix, models fine-tuned on the unpatched base would fail to end turns in 98.4-100% of cases. After swapping just the lm_head <|im_end|> row, the probability of emitting <|im_end|> at a turn boundary increased from 0.00033 to 0.131, and its rank improved from 567 to 1. This resulted in 26 out of 40 greedy turns ending within 256 tokens, compared to 0 out of 40 previously, and eliminated payload leakage.

Important Caveats

• The fix makes the token reachable, but not as reliable as a model explicitly trained against a repaired row (which can reach ~0.99 probability). This version achieves 0.13-0.35 probability at rank 1.
• Static geometry tests failed to predict this issue; behavioral verification (generating turns and counting completions) is crucial.
• Only the two ChatML markers were addressed; other special tokens were not examined.

Ideal Use Case

This model is particularly suited for developers who need a Qwen2.5-32B base that reliably handles ChatML turn termination, especially for fine-tuning conversational agents where proper turn-taking and response completion are essential. It provides a more functional foundation for building robust dialogue systems compared to the original Qwen/Qwen2.5-32B.