md-nishat-008/TigerLLM-1B-it
TigerLLM-1B-it is a 1 billion parameter instruction-tuned causal language model developed by Nishat Raihan and Marcos Zampieri, specifically optimized for the Bangla language. It is part of the TigerLLM family, which leverages a high-quality 10M-token Bangla-TextBook corpus and a 100K-pair Bangla-Instruct dataset. This model surpasses existing open-source and some larger proprietary models like GPT-3.5 on standard Bangla benchmarks, establishing a new baseline for Bangla language modeling.
Loading preview...
TigerLLM-1B-it: A Specialized Bangla Language Model
TigerLLM-1B-it is a 1 billion parameter instruction-tuned model from the TigerLLM family, developed by Nishat Raihan and Marcos Zampieri. This model addresses the significant linguistic disparity in LLM development by focusing on Bangla, the 5th most spoken language, which has historically been underserved. It aims to provide a highly reproducible and performant open-source solution for Bangla NLP.
Key Capabilities and Differentiators
- Bangla-Optimized Performance: TigerLLM-1B-it demonstrates superior performance compared to all open-source alternatives and even outperforms larger proprietary models like GPT-3.5 on various Bangla-specific benchmarks, including MMLU-bn, PangBench-bn, and BanglaQuaD.
- High-Quality Training Data: The model is built upon two meticulously curated datasets:
- Bangla-TextBook Corpus: A 10 million-token corpus derived from 163 high-quality educational textbooks (Grades 6-12) from Bangladesh, capturing authentic academic language.
- Bangla-Instruct Dataset: 100,000 native Bangla instruction-response pairs generated using a self-instruct framework with GPT-4 and Claude-3.5-Sonnet, ensuring high linguistic quality and cultural sensitivity through multi-stage filtering.
- Robust Training Methodology: The training involves continual pretraining on the Bangla-TextBook corpus to capture language nuances, followed by full fine-tuning (without LoRA) using Flash Attention for efficient convergence.
Ideal Use Cases
TigerLLM-1B-it is particularly well-suited for applications requiring high-accuracy natural language understanding and generation in Bangla. This includes:
- Educational Tools: Developing intelligent tutoring systems or content generation for Bangla educational materials.
- Information Retrieval: Enhancing search engines or question-answering systems for Bangla content.
- Content Creation: Generating high-quality, culturally sensitive text in Bangla for various purposes.
- Research and Development: Serving as a strong baseline for further research into low-resource language modeling and Bangla NLP.