dibakar12b/DeepSeek-R1-Distill-1.5B-Indic
The dibakar12b/DeepSeek-R1-Distill-1.5B-Indic is a 1.5 billion parameter Qwen2-based causal language model, developed by dibakar12b and fine-tuned using Unsloth and Huggingface's TRL library. This model is optimized for faster training, leveraging Unsloth's efficiency. It is designed for general language tasks, with a notable context length of 32768 tokens.
Loading preview...
Model Overview
The dibakar12b/DeepSeek-R1-Distill-1.5B-Indic is a 1.5 billion parameter language model based on the Qwen2 architecture. It was developed by dibakar12b and fine-tuned from the unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit base model.
Key Characteristics
- Architecture: Qwen2-based causal language model.
- Parameter Count: 1.5 billion parameters.
- Context Length: Supports a substantial context window of 32768 tokens.
- Training Efficiency: Fine-tuned using Unsloth and Huggingface's TRL library, enabling 2x faster training compared to standard methods.
Intended Use Cases
This model is suitable for applications requiring a compact yet capable language model with efficient training characteristics. Its large context window makes it potentially useful for tasks involving longer texts. The use of Unsloth for fine-tuning suggests an emphasis on resource-efficient deployment and iteration.