Mecellem-Qwen3-1.7B-TR: A Specialized Turkish Legal LLM
Mecellem-Qwen3-1.7B-TR is a 1.7 billion parameter language model developed by newmindai, specifically adapted for the Turkish legal domain. It is built upon the Qwen3 decoder architecture and has undergone extensive Continual Pre-training (CPT) on a massive dataset of Turkish legal and official texts, totaling approximately 225 billion tokens.
Key Capabilities
- Domain-Specific Expertise: Achieves a 36.2% perplexity reduction on Turkish legal text compared to the base Qwen3-1.7B model, indicating superior understanding and generation of legal content.
- Curriculum Learning: Utilizes a unique four-phase curriculum learning strategy, progressively transitioning from general-purpose Turkish texts to complex, domain-specific legal documents, ensuring both linguistic stability and deep legal knowledge injection.
- Robust Training: Trained on a diverse dataset including Turkish legal sources (Yargıtay, Danıştay, YÖKTEZ) and general Turkish web data (FineWeb2, CulturaX), preserving general language capabilities while specializing in law.
- Performance: Consistently outperforms the base Qwen3-1.7B model across various legal quality objectives, including depth of coverage, statute reference usage, and legal accuracy, as evaluated by the Muhakim reward model.
Good for
- Turkish Legal Text Generation: Creating accurate and contextually relevant legal documents.
- Legal Document Summarization: Condensing lengthy legal texts into concise summaries.
- Legal Question Answering: Providing informed responses to legal queries in Turkish.
- Domain-Specific Language Modeling: Applications requiring deep understanding and generation within the Turkish legal framework.
- Retrieval-Augmented Generation (RAG): Enhancing RAG systems with specialized Turkish legal knowledge.