Hugging Face Overview
Model Overview
DeepSeek-R1-Distill-Llama-3B is a 3.2 billion parameter language model, created by suayptalha. It is a distilled version of DeepSeek-R1, built upon the Llama-3.2-3B architecture and fine-tuned using the R1-Distill-SFT dataset. This model leverages the Llama3 chat template for instruction following and includes a suggested system prompt format for structured reasoning.
Key Capabilities
- Instruction Following: Designed to respond effectively to user instructions, utilizing a Llama3-style chat template.
- Reasoning: The suggested prompt format encourages a "think-then-answer" approach, as demonstrated in the example usage for logical problem-solving.
- Compact Size: At 3.2 billion parameters, it offers a balance between performance and computational efficiency.
Performance Highlights
Evaluated on the Open LLM Leaderboard, the model achieved an average score of 23.27. Specific scores include 70.93 on IFEval (0-Shot) and 21.45 on BBH (3-Shot), indicating its ability to handle various reasoning and instruction-based tasks.
Good For
- Instruction-tuned applications: Suitable for tasks requiring the model to follow specific commands or formats.
- Reasoning tasks: Benefits from the structured prompting approach to tackle problems requiring logical thought processes.
- Resource-constrained environments: Its 3.2B parameter count makes it a viable option where larger models might be impractical.