RESMPDEV/Qwen2-Wukong-0.5B

Hugging Face
TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:0.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 29, 2024License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Warm

RESMPDEV/Qwen2-Wukong-0.5B is a 0.5 billion parameter instruction-tuned language model, finetuned by RESMPDEV from the Qwen2-0.5B base model. It was trained for three epochs on the Teknium OpenHermes-2.5 dataset and supplementary datasets from Cognitive Computations. This model is specifically designed for chat applications, offering a dealigned finetune of the original Qwen2 architecture with a 32768 token context length, making it suitable for conversational AI tasks.

Loading preview...

Qwen2-Wukong-0.5B: A Dealigned Chat Finetune

Qwen2-Wukong-0.5B is a 0.5 billion parameter instruction-tuned language model developed by RESMPDEV, based on the Qwen2-0.5B architecture from the Qwen team. This model is specifically finetuned for chat applications, distinguishing itself as a "dealigned" version of the original Qwen2-0.5B. It leverages a substantial 32768 token context length, enabling it to handle longer conversational turns and more complex interactions.

Key Capabilities

  • Instruction Following: Optimized for understanding and responding to user instructions in a conversational context.
  • Chat-Oriented Performance: Designed for effective dialogue generation and interactive applications.
  • Extended Context Window: Supports a 32768 token context, allowing for more coherent and context-aware responses over longer conversations.
  • Efficient Size: At 0.5 billion parameters, it offers a balance between performance and computational efficiency, making it suitable for deployment in resource-constrained environments.

Training Details

The model underwent three epochs of training on a combination of the Teknium OpenHermes-2.5 dataset and additional datasets from Cognitive Computations. This targeted training regimen aims to enhance its conversational abilities and alignment for chat-based use cases.

Good For

  • Building lightweight, efficient chatbots and conversational agents.
  • Applications requiring a model with a good balance of performance and a smaller footprint.
  • Scenarios where an extended context window is beneficial for maintaining conversation flow and understanding.

Finetunes
3 models