shaohongwu/qwen2.5-0.5b-special-tokens
The shaohongwu/qwen2.5-0.5b-special-tokens model is a 0.5 billion parameter derivative base model of Qwen/Qwen2.5-0.5B, developed by shaohongwu. It extends the original Qwen2.5-0.5B tokenizer with specific schema/control special tokens, increasing its vocabulary size. This model is primarily intended for schema-aware prompting and structured information extraction tasks, such as slot, intent, and domain prediction, serving as a base for LoRA adapters.
Loading preview...
Overview
This model, shaohongwu/qwen2.5-0.5b-special-tokens, is a 0.5 billion parameter derivative of the Qwen/Qwen2.5-0.5B base model. Its core distinction lies in an extended tokenizer vocabulary that incorporates specific schema and control special tokens. These additions are designed to facilitate more precise and structured interactions with the model.
Key Capabilities
- Enhanced Tokenization: Includes special tokens like
<|domain_start|>,<|intent_start|>,<|slot_type_start|>,<|slot_span_start|>, and<|canonical_start|>(and their corresponding_endtokens) to enable schema-aware processing. - Structured Output Focus: Optimized for tasks requiring the extraction or generation of structured information.
- Base Model for Fine-tuning: Intended as a foundational model for further fine-tuning, particularly with LoRA adapters, which must be trained using the same extended tokenizer.
Intended Usage
This model is particularly well-suited for:
- Schema-aware prompting: Guiding the model to understand and generate content based on predefined schemas.
- Structured information extraction: Accurately pulling out specific data points like slots, intents, and domains from text.
- Compatibility: Designed for efficient serving with vLLM and TensorRT-LLM, supporting multi-LoRA dynamic loading. Users should note that vocabulary shapes are fixed and should not be altered at runtime.