nassimjp/LFM2.5-1.2B-Base-Pashto
The nassimjp/LFM2.5-1.2B-Base-Pashto model is a 1.2 billion parameter language model, likely based on the LiquidAI/LFM2.5-1.2B-Base architecture, specifically adapted for the Pashto language. It features a context length of 32768 tokens and incorporates a specialized tokenizer for Pashto linguistic atoms. This model is designed for applications requiring Pashto language understanding and generation, with potential for continued pretraining and supervised fine-tuning using Pashto corpora and instruction datasets.
Loading preview...
nassimjp/LFM2.5-1.2B-Base-Pashto: Pashto Language Model
This model, nassimjp/LFM2.5-1.2B-Base-Pashto, is a 1.2 billion parameter base model specifically tailored for the Pashto language. It is built upon the LiquidAI/LFM2.5-1.2B-Base architecture and features an extended context length of 32768 tokens.
Key Features & Adaptations
- Pashto-Specific Tokenizer: The model incorporates a specialized tokenizer designed to handle unique Pashto linguistic atoms, ensuring accurate processing of the language.
- Base Model: This is a base model, indicating it is suitable for further adaptation and fine-tuning for specific Pashto NLP tasks.
Training Considerations
- Continued Pretraining (CPT): Requires extensive, deduplicated, and domain-balanced Pashto corpora for effective continued pretraining.
- Supervised Fine-Tuning (SFT): Benefits from high-quality Pashto instruction datasets formatted for chat applications to achieve optimal performance in instruction-following tasks.
License
The model operates under the same license terms as its base model, LiquidAI/LFM2.5-1.2B-Base.