mariklolik/AraToken-Qwen3-1.7B-CPT

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 23, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

mariklolik/AraToken-Qwen3-1.7B-CPT is a 1.7 billion parameter Qwen3-based language model developed by mariklolik, specifically optimized for Arabic language processing. This model is a continued-pretraining baseline that utilizes the AraToken normalizer for efficient Arabic tokenization, including NFKC, Tatweel removal, and diacritics removal. It achieves a lower Arabic Bits Per Character (BPC) score of 1.2672 compared to the base Qwen3-1.7B, indicating improved compression and representation of Arabic text. This model is ideal for applications requiring high-quality Arabic language understanding and generation.

Loading preview...

AraToken-Qwen3-1.7B-CPT: Optimized Arabic Language Model

This model, developed by mariklolik, is a 1.7 billion parameter Qwen3-based language model focused on enhancing Arabic language processing through continued pre-training. It builds upon the original Qwen3 architecture, specifically fine-tuning layers 24-27 over 2,000 steps using the same data as its LEP counterpart.

Key Capabilities

  • Efficient Arabic Tokenization: Integrates the AraToken normalizer, which processes raw Arabic text by applying NFKC normalization, removing Tatweel, Western digits, Latin punctuation, and diacritics. This ensures cleaner and more consistent input for the model.
  • Improved Arabic Compression: Demonstrates superior performance in Arabic text compression, achieving a Bits Per Character (BPC) score of 1.2672. This is a significant improvement over the base Qwen3-1.7B's 1.3313 BPC, indicating more efficient encoding of Arabic characters.
  • Research-Backed Development: The model's methodology and performance are detailed in the paper "AraToken: Optimizing Arabic Tokenization with Normalization Pipeline and Language Extension for Qwen3" (arXiv:2512.18399).

Good For

  • Applications requiring robust and efficient processing of Arabic text.
  • Research and development in Arabic Natural Language Processing (NLP).
  • Tasks where optimized tokenization and character representation for Arabic are crucial.

Finetunes
1 models