URajinda/ShweYon-V3-Base
URajinda/ShweYon-V3-Base is a 1.5 billion parameter base language model built upon the Qwen 2.5 architecture, specifically fine-tuned for the Myanmar language. It integrates a custom tokenizer with over 9,000 Myanmar morphemes and an extended vocabulary of 160,746 tokens directly into its embedding, enhancing efficiency for Myanmar text processing. This model serves as a foundational base for developing Myanmar-centric NLP applications like chatbots and question-answering systems.
Loading preview...
ShweYon-V3-Base: A Myanmar-Centric Foundation Model
URajinda/ShweYon-V3-Base is a 1.5 billion parameter base language model derived from the Qwen 2.5 architecture, meticulously optimized for the Myanmar language. It represents a significant advancement in the "ShweYon" project by directly integrating Myanmar tokenization into the model's embedding, eliminating the need for a separate tokenizer.
Key Technical Highlights
- Integrated Custom Tokenizer: Incorporates over 9,000 Myanmar morphemes and word combinations directly, streamlining Myanmar language processing.
- Extended Vocabulary: Features an expanded vocabulary size of 160,746, enabling more compact and efficient computation of Myanmar texts.
- Myanmar-Specific Base Training: The model has been extensively trained on a large corpus of Myanmar literary texts to enhance its fundamental understanding and knowledge of the language.
Purpose and Use Cases
This model is designed as a Foundation Base Model for the Myanmar language. It provides a robust starting point for further fine-tuning (SFT/RLHF) to develop various downstream NLP applications, including:
- Chatbots
- Question Answering systems
- Other Myanmar-specific NLP tasks
Note: As a base model, ShweYon-V3-Base requires additional chat fine-tuning for instruction following and human-like conversational capabilities.