paumkim/zomi-qlora-v1

TEXT GENERATIONConcurrent Unit Cost:1Model Size:3.1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 14, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

paumkim/zomi-qlora-v1 is a 3.1 billion parameter Qwen 2.5 model fine-tuned using QLoRA for the Zomi (Zou) language. This model is the first publicly available language model specifically adapted to generate and understand Zomi text, a language spoken by approximately 500,000 people. It excels at Zomi language generation and comprehension, serving as a foundational model for further Zomi-specific AI development.

Loading preview...

Zomi QLoRA v1: A Foundational Zomi Language Model

This model, paumkim/zomi-qlora-v1, is a Qwen 2.5 3B base model that has been fine-tuned using QLoRA (rank 128) specifically for the Zomi (Zou) language. It represents the first publicly available language model dedicated to Zomi, a language spoken by around 500,000 individuals across Myanmar, India, and Bangladesh.

Key Capabilities

  • Zomi Text Generation: Capable of producing coherent text in the Zomi language.
  • Language Understanding: Demonstrates an understanding of Zomi grammar, vocabulary, and structural nuances.
  • Continued Pretraining: Functions as a language adaptation model, speaking Zomi but not yet following complex instructions.

Training Details

The model was trained on approximately 54,000 chunks of Zomi text sourced from public domain literature, community content, and religious texts. The training utilized QLoRA with 239 million trainable parameters (7.2% of the base model) on an NVIDIA A100 80GB GPU for about 20 hours.

Limitations (Current)

Currently, this model is a language adaptation model. It can generate and understand Zomi but does not yet:

  • Follow instructions or commands.
  • Perform on-demand translation between Zomi and English.
  • Engage in conversational dialogue.

Instruction fine-tuning (Phase 2) is actively in progress to address these limitations and enable command-following abilities.

Why This Model Exists

Developed by a Zomi speaker, this model aims to provide AI support for the Zomi language, which is largely unrepresented in commercial AI, empowering the Zomi community with language technology.

Finetunes
1 models