surrey-nlp/diallm-qwen-dpo-ind
The surrey-nlp/diallm-qwen-dpo-ind is an 8 billion parameter Qwen 3-8B model developed by surrey-nlp, specifically adapted for Indian English (en-IN). It was continually pretrained on the International Corpus of English and then fine-tuned using Direct Preference Optimization (DPO) on dialect-specific preference data. This model excels at generating responses aligned with Indian English linguistic nuances, making it suitable for applications requiring dialect-aware language generation.
Loading preview...
DiaLLM: Qwen 3-8B for Indian English (en-IN)
This model, developed by surrey-nlp, is an 8 billion parameter variant of the Qwen 3-8B architecture, specifically adapted for Indian English (en-IN). It is part of the DiaLLM project, which investigates the robustness-generation gap in English dialect adaptation.
Key Adaptation Details
- Base Model: Qwen 3-8B, a powerful causal language model.
- Target Variety: Explicitly adapted for Indian English (en-IN).
- Training Process: The model underwent continual pretraining on the International Corpus of English (~20M tokens across 18 varieties). Following this, it was fine-tuned using a dialect-specific Supervised Fine-Tuning (SFT) approach on Multi-VALUE-transformed en-IN preference data. The final alignment was achieved through Direct Preference Optimization (DPO) using target-variety preference pairs.
- Alignment Method: Utilizes DPO for enhanced alignment with target dialect preferences.
Use Cases
- Dialect-Aware Generation: Ideal for applications requiring text generation that reflects the linguistic characteristics and nuances of Indian English.
- Research: Useful for researchers studying dialect adaptation, robustness, and generation in large language models.
Further details, including code, checkpoints, preference datasets, and a linguistic-analysis toolkit, are available on the surrey-nlp/diallm GitHub repository. The underlying research paper, "DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation," provides comprehensive insights into its development and methodology.