md-nishat-008/TigerLLM-9B-it
TigerLLM-9B-it is a 9 billion parameter instruction-tuned large language model developed by Nishat Raihan and Marcos Zampieri, based on the Gemma-2 architecture. It is specifically designed for the Bangla language, leveraging a high-quality 10M-token Bangla-TextBook corpus and a 100K-pair Bangla-Instruct dataset. This model sets new benchmarks for Bangla language modeling, outperforming existing open-source alternatives and even larger proprietary models like GPT-3.5 on various Bangla-specific tasks.
Loading preview...
TigerLLM-9B-it: A Leading Bangla Language Model
TigerLLM-9B-it is a 9 billion parameter instruction-tuned model developed by Nishat Raihan and Marcos Zampieri, specifically optimized for the Bangla language. It addresses the significant disparity in LLM development for low-resource languages by providing a high-performance, reproducible solution for Bangla NLP.
Key Capabilities & Features
- Superior Bangla Performance: Outperforms all open-source Bangla LLMs and proprietary models like GPT-3.5 on standard Bangla benchmarks (MMLU-bn, PangBench-bn, BanglaQuaD, mHumanEval-bn, BEnQA, BanglaRQA).
- High-Quality Training Data: Leverages two meticulously curated datasets:
- Bangla-TextBook Corpus: A 10 million-token corpus derived from 163 educational textbooks (Grades 6-12) from Bangladesh, capturing authentic academic language.
- Bangla-Instruct Dataset: 100,000 native Bangla instruction-response pairs generated via a self-instruct framework using GPT-4 and Claude-3.5-Sonnet, with multi-stage filtering for quality and cultural sensitivity.
- Robust Training Methodology: Involves continual pretraining on the Bangla-TextBook corpus to capture language nuances, followed by full fine-tuning (without LoRA) using Flash Attention.
- Reproducible Research: Developed with a focus on reproducibility, providing clear methodologies and datasets to advance Bangla NLP research.
Good for
- Bangla-specific NLP applications: Ideal for tasks requiring deep understanding and generation in Bangla.
- Research and Development: Serves as a new baseline for future Bangla language modeling research.
- Educational Technology: Can be applied in tools and platforms for Bangla-speaking students and educators.
- Content Generation: Generating high-quality, culturally sensitive text in Bangla.