burgasdotpro/bgGPT-DeepSeek-R1-Distill-Qwen-7B
The burgasdotpro/bgGPT-DeepSeek-R1-Distill-Qwen-7B is a 7.6 billion parameter language model, based on the DeepSeek-R1-Distill-Qwen-7B architecture, developed by burgasdotpro. This model is specifically optimized for Bulgarian language processing, demonstrating significantly improved perplexity on both short and long Bulgarian texts compared to its base model. It excels in tasks requiring logical reasoning and step-by-step problem-solving in Bulgarian, making it suitable for applications needing robust Bulgarian language understanding and generation.
Loading preview...
bgGPT-DeepSeek-R1-Distill-Qwen-7B Overview
This model, developed by burgasdotpro, is a 7.6 billion parameter language model built upon the DeepSeek-R1-Distill-Qwen-7B architecture. It has undergone continued pretraining with a focus on Bulgarian language data, specifically utilizing Wikipedia content (50% and 100% in different pretraining phases).
Key Capabilities & Performance
- Bulgarian Language Optimization: The model shows substantial improvements in perplexity (PPL) for Bulgarian text. For short texts, PPL improved from 179.76 (base model) to 72.63, and for long texts, from 258.56 to 83.96.
- Reasoning and Problem Solving: Demonstrated capability in logical reasoning and step-by-step problem-solving, as shown in the example of solving algebraic equations with detailed thought processes.
- Efficient Training: The model was trained using Unsloth and Huggingface's TRL library, enabling faster training times (2x faster).
Use Cases
This model is particularly well-suited for applications requiring strong Bulgarian language understanding, generation, and logical reasoning. It can serve as an effective Bulgarian-language automated assistant for tasks involving text comprehension, mathematical problem-solving, and general conversational AI in Bulgarian.