johnbean393/chiboard-1-t2-preview-0712
Chiboard-1 T2 Preview is a 1.2 billion parameter Chinese pinyin input-method teacher model developed by johnbean393. This model is a preference-tuned iteration of Chiboard-1 T1, specifically designed to improve handling of hard ambiguity and revision behavior in pinyin input. While it shows strong gains in these targeted areas, it exhibits slight regressions in general plain-input guardrails compared to its predecessor. It is released as a preview for evaluation and research purposes, not as a production replacement for T1.
Loading preview...
Chiboard-1 T2 Preview: A Specialized Chinese Pinyin Input Model
This repository introduces johnbean393/chiboard-1-t2-preview-0712, a 1.2 billion parameter model specifically designed as a Chinese pinyin input-method teacher. It is a preference-tuned iteration of the chiboard-1-t1 base model, trained on 150,000 preference pairs using DPO (Direct Preference Optimization).
Key Characteristics & Performance
Chiboard-1 T2 was developed to address specific challenges in pinyin input:
- Improved Hard Ambiguity Handling: Achieved a significant +7.453 pp gain in exact match accuracy and a -0.013983 reduction in Character Error Rate (CER) on hard-ambiguity cases compared to the S1 baseline.
- Enhanced Revision Behavior: Showed modest improvements in revision exact match and CER, passing the acceptance gate for this metric.
- Preference Tuning: Utilizes DPO with a chosen-answer NLL anchor, resulting in a reward accuracy of 61.5% and a preference gain of 0.2227.
Important Considerations
- Preview Release: This is a research and evaluation checkpoint and did not pass the full production acceptance gate due to regressions in general plain-input quality.
- Regression in Plain Input: Compared to T1, T2 exhibited a -0.341 pp decrease in plain exact match and a slight increase in plain CER, indicating a tradeoff between specialized ambiguity handling and general input performance.
- Intended Domain: The model is strictly intended for Chinese pinyin IME behavior. Its performance on general-purpose text generation has not been validated.
When to Use This Model
This model is particularly useful for researchers and developers interested in:
- Exploring tradeoffs between specialized ambiguity resolution and general input quality in pinyin IME systems.
- Evaluating advanced preference tuning techniques like DPO for specific linguistic tasks.
- Developing or testing input methods where handling complex pinyin ambiguities is a primary concern, even if it means a slight compromise on general input accuracy.