artificialguybr/QWEN-2-1.5B-Synthia-II-Redmond
artificialguybr/QWEN-2-1.5B-Synthia-II-Redmond is a 1.5 billion parameter causal language model, fine-tuned from Qwen/Qwen2-1.5B on the Synthia v1.5-II dataset. This model enhances instruction-following capabilities, building upon the base Qwen2 series' improvements in language understanding, structured data processing, and multi-language support. With a context length of 32768 tokens, it is optimized for instruction following and conversational AI applications.
Loading preview...
Model Overview
artificialguybr/QWEN-2-1.5B-Synthia-II-Redmond is a 1.5 billion parameter causal language model, fine-tuned from the base Qwen/Qwen2-1.5B model. This fine-tuned version specifically enhances instruction-following capabilities by training on the Synthia v1.5-II dataset, which comprises over 20.7k instruction-following examples. The fine-tuning process was sponsored by Redmond.ai, providing the necessary GPU resources.
Key Capabilities
- Enhanced Instruction Following: Specialized training on the Synthia v1.5-II dataset significantly improves the model's ability to understand and execute instructions.
- Strong Base Model Performance: Inherits the Qwen2 series' advancements in:
- Language understanding and generation.
- Structured data processing.
- Support for multiple languages.
- Long context handling (up to 32768 tokens).
- Causal Language Modeling: Designed for text generation and completion tasks.
Training Details
The model was trained using Axolotl, with hyperparameters including a learning rate of 1e-05, 3 epochs, and a sequence length of 4096. The training utilized an Adam optimizer and a cosine LR scheduler.
Intended Uses
This model is well-suited for:
- Instruction following and task completion.
- General text generation and completion.
- Conversational AI applications requiring robust instruction adherence.