starcoding/day1-train-model
The starcoding/day1-train-model is a 0.5 billion parameter Qwen2.5-based instruction-tuned causal language model developed by starcoding. Finetuned from unsloth/Qwen2.5-0.5B-Instruct-unsloth-bnb-4bit, this model was trained using Unsloth and Huggingface's TRL library, achieving 2x faster training. With a 32768 token context length, it is optimized for efficient instruction-following tasks.
Loading preview...
Model Overview
The starcoding/day1-train-model is a compact 0.5 billion parameter instruction-tuned language model developed by starcoding. It is based on the Qwen2.5 architecture and was finetuned from unsloth/Qwen2.5-0.5B-Instruct-unsloth-bnb-4bit.
Key Characteristics
- Architecture: Qwen2.5-based causal language model.
- Parameter Count: 0.5 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Training Efficiency: Finetuned using Unsloth and Huggingface's TRL library, resulting in a 2x speedup during the training process.
Use Cases
This model is suitable for applications requiring efficient instruction-following capabilities, particularly where a smaller model size and optimized training are beneficial. Its substantial context length allows for processing longer prompts and maintaining conversational coherence over extended interactions.