Abhaykoul/Qwen1.5-0.5B-vortex: Dealigned Chat Finetune

This model, developed by Abhaykoul, is a 0.6 billion parameter chat-finetuned variant of the Qwen1.5-0.5B base model. It has been specifically 'dealigned' and trained on the Vortex mini dataset over 5 epochs using axolotl, aiming for specialized chat interactions.

Key Characteristics & Performance

• Base Model: Derived from the robust Qwen1.5-0.5B architecture.
• Training: Fine-tuned for chat on the Vortex mini dataset.
• Parameter Count: A compact 0.6 billion parameters, making it efficient for deployment.
• Context Length: Supports a substantial context window of 32768 tokens.
• Benchmark Performance: While being a dealigned version, it maintains competitive average scores compared to its base model and other 0.5B variants across benchmarks like ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, and GSM8k.

Good For

• Resource-constrained chat applications: Its small size makes it suitable for environments where computational resources are limited.
• Experimental chat deployments: Ideal for exploring dealigned chat model behaviors or specific conversational styles.
• Rapid prototyping: Enables quick iteration for chat-based features due to its efficiency.