artificialguybr/QWEN-2.5-0.5B-Synthia-I
artificialguybr/QWEN-2.5-0.5B-Synthia-I is a 0.49 billion parameter causal language model, fine-tuned by artificialguybr from the Qwen2.5-0.5B series. This model is specifically enhanced for instruction following and task completion, leveraging the base model's 32,768 token context length and multilingual support. It excels at generating structured outputs and is intended for conversational AI applications.
Loading preview...
Model Overview
artificialguybr/QWEN-2.5-0.5B-Synthia-I is a fine-tuned version of the Qwen2.5-0.5B base model, developed by artificialguybr. This model, with 0.49 billion parameters, is part of the latest Qwen2.5 series and inherits its robust capabilities in instruction following, long text generation, and structured data understanding. It supports a substantial context length of 32,768 tokens and over 29 languages.
Key Enhancements & Capabilities
This specific iteration has been significantly enhanced through fine-tuning on the Synthia v1.5-I dataset, which comprises over 20.7k instruction-following examples. This training specifically boosts its performance in:
- Instruction following and task completion: Optimized to accurately understand and execute given instructions.
- Text generation and completion: Capable of producing coherent and relevant text based on prompts.
- Conversational AI applications: Well-suited for interactive dialogue systems.
- Multilingual support: Retains the base model's ability to process and generate text in numerous languages.
Training Details
The model was trained using a learning rate of 1e-05, a batch size of 5 (total 40 with gradient accumulation), and an Adam optimizer over 3 epochs. The training utilized a sequence length of 4096 with sample packing enabled, ensuring efficient use of the long context window.