Overview

This model, shaohongwu/qwen2.5-0.5b-special-tokens, is a 0.5 billion parameter derivative of the Qwen/Qwen2.5-0.5B base model. Its core distinction lies in an extended tokenizer vocabulary that incorporates specific schema and control special tokens. These additions are designed to facilitate more precise and structured interactions with the model.

Key Capabilities

• Enhanced Tokenization: Includes special tokens like <|domain_start|>, <|intent_start|>, <|slot_type_start|>, <|slot_span_start|>, and <|canonical_start|> (and their corresponding _end tokens) to enable schema-aware processing.
• Structured Output Focus: Optimized for tasks requiring the extraction or generation of structured information.
• Base Model for Fine-tuning: Intended as a foundational model for further fine-tuning, particularly with LoRA adapters, which must be trained using the same extended tokenizer.

Intended Usage

This model is particularly well-suited for:

• Schema-aware prompting: Guiding the model to understand and generate content based on predefined schemas.
• Structured information extraction: Accurately pulling out specific data points like slots, intents, and domains from text.
• Compatibility: Designed for efficient serving with vLLM and TensorRT-LLM, supporting multi-LoRA dynamic loading. Users should note that vocabulary shapes are fixed and should not be altered at runtime.