AvinabhDutta-Dev/bangla-paraphrase-tigerllm-qlora
AvinabhDutta-Dev/bangla-paraphrase-tigerllm-qlora is a 1 billion parameter causal decoder-only language model, based on the Gemma architecture, fine-tuned by AvinabhDutta-Dev using QLoRA for Bengali paraphrase generation. This model excels at creating semantically equivalent but lexically diverse Bengali sentences, achieving strong performance on metrics like BLEU and BERTScore while training only 5% of its parameters. It is specifically designed for tasks such as data augmentation in Bengali NLP or research into low-resource paraphrase generation.
Loading preview...
Overview
This model, AvinabhDutta-Dev/bangla-paraphrase-tigerllm-qlora, is a 1 billion parameter Gemma-based causal language model. It has been fine-tuned using QLoRA on a 100,000-pair subset of the BanglaParaphrase dataset specifically for Bengali paraphrase generation. The LoRA adapter has been merged into the base weights, making it a complete, standalone model.
Key Capabilities
- Bengali Paraphrase Generation: Generates alternative phrasings for Bengali sentences.
- Parameter-Efficient Fine-Tuning (PEFT): Achieves performance comparable to full fine-tuning while updating only ~13 million parameters (approximately 5% of the model's total parameters).
- Strong Performance: Matches or exceeds full-fine-tuned models like BanglaT5 on metrics such as BLEU (18.69) and BERTScore (0.942 F1) for Bengali paraphrasing.
- LLM-as-a-Judge Validation: Achieved a mean meaning-preservation score of 4.47/5 and fluency score of 4.77/5 in an LLM-judge evaluation.
Intended Use Cases
- Data Augmentation: Useful for expanding datasets in downstream Bengali NLP tasks.
- Research: Supports research into low-resource paraphrase generation and parameter-efficient fine-tuning techniques.
Limitations
- Language Specificity: Intended only for Bengali; not suitable for other languages or code-mixed text.
- Task Specificity: Fine-tuned exclusively for sentence-level paraphrasing; not a general-purpose Bengali instruction-following or chat model.
- Semantic Accuracy: While performing well, the model can exhibit hallucination or meaning reversal in some cases, so outputs should be reviewed for high-stakes applications.