artificialguybr/QWEN-2-1.5B-Synthia-I
The artificialguybr/QWEN-2-1.5B-Synthia-I is a 1.5 billion parameter causal language model, fine-tuned by artificialguybr from the Qwen2-1.5B base model. It has been specifically trained on the Synthia v1.5-I dataset, comprising over 20.7k instruction-following examples. This model enhances the base Qwen2's language understanding, generation, and multi-language support, making it optimized for instruction following and conversational AI applications.
Loading preview...
Model Overview
This model, artificialguybr/QWEN-2-1.5B-Synthia-I, is a fine-tuned version of the Qwen2-1.5B base model, developed by artificialguybr. It leverages the robust architecture of the Qwen2 series, which is known for its advancements in language understanding, generation, structured data processing, and multi-language support. The model has 1.5 billion parameters and was trained using Transformers 4.45.0.dev0.
Key Enhancements & Capabilities
The primary differentiator of this model is its fine-tuning on the Synthia v1.5-I dataset, which includes over 20.7k instruction-following examples. This specialized training significantly boosts its ability in:
- Instruction following and task completion: Excelling at understanding and executing user commands.
- Text generation and completion: Producing coherent and contextually relevant text.
- Conversational AI applications: Engaging in more natural and effective dialogues.
Training Details
The fine-tuning process involved a learning rate of 1e-05, a total batch size of 40 (with gradient accumulation), and 3 epochs of training. A sequence length of 4096 was used with sample packing enabled, ensuring efficient use of context. The model inherits the base Qwen2's capabilities while being specifically optimized for instruction-driven tasks, making it a strong candidate for applications requiring precise command execution and interactive text generation.