Ismantic/Qwen3-1.7B-Base-ReTok

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 16, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

Ismantic/Qwen3-1.7B-Base-ReTok is a 1.7 billion parameter language model developed by Ismantic, derived from Qwen/Qwen3-1.7B-Base. This model features a custom 81,903-token Piece tokenizer, replacing the original Qwen tokenizer, and has undergone continued pretraining. It is specifically optimized for tasks requiring a custom tokenizer, such as machine translation, and is a base model rather than an instruction-tuned assistant.

Loading preview...

Overview

Ismantic/Qwen3-1.7B-Base-ReTok is a specialized variant of the Qwen3-1.7B-Base model, developed by Ismantic as part of the Summer project. Its primary distinction lies in the replacement of the original Qwen tokenizer with a custom 81,903-token Piece tokenizer, followed by continued pretraining to recover and adapt the model. This model is the final tie-preserving v18 checkpoint from the Summer project.

Key Characteristics

  • Custom Tokenizer: Utilizes a unique Piece tokenizer, requiring a specific PieceTokenizerWrapper for proper use, especially for Chinese segmentation. Standard AutoTokenizer will not work.
  • Continued Pretraining: The model underwent a two-phase pretraining process: initially training new embeddings with a frozen transformer, then annealing with LoRA q/v adapters and tied embedding/head preservation.
  • vLLM Compatibility: While vLLM loads the model weights, its tokenizer cannot be used directly; users must pass token IDs manually with skip_tokenizer_init=True.
  • Performance: Evaluation on WMT22 and WMT23 translation tasks shows competitive results compared to the original Qwen3-1.7B-Base, particularly in COMET scores. General benchmarks (LAMBADA, PIQA, ARC-C, HellaSwag, CEVAL) are also provided.
  • Known Limitations: The tokenizer replacement leads to a significant, permanent loss in GSM8K performance due to altered numeric tokenization. It is a base model, not instruction-tuned, and generic Hugging Face inference may not support the custom tokenizer out-of-the-box.

Intended Use

This model is best suited for research and development where a custom tokenizer is beneficial or required, particularly in machine translation tasks. Developers should be prepared to integrate the custom tokenizer wrapper and manage tokenization manually, especially when using frameworks like vLLM.

Finetunes
1 models