Model Overview

The starcoding/day1-train-model is a compact 0.5 billion parameter instruction-tuned language model developed by starcoding. It is based on the Qwen2.5 architecture and was finetuned from unsloth/Qwen2.5-0.5B-Instruct-unsloth-bnb-4bit.

Key Characteristics

• Architecture: Qwen2.5-based causal language model.
• Parameter Count: 0.5 billion parameters.
• Context Length: Supports a substantial context window of 32768 tokens.
• Training Efficiency: Finetuned using Unsloth and Huggingface's TRL library, resulting in a 2x speedup during the training process.

Use Cases

This model is suitable for applications requiring efficient instruction-following capabilities, particularly where a smaller model size and optimized training are beneficial. Its substantial context length allows for processing longer prompts and maintaining conversational coherence over extended interactions.