Sjnu/DeepSeek-R1-Distill-Llama-8B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 16, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

Sjnu/DeepSeek-R1-Distill-Llama-8B is an 8 billion parameter language model distilled from DeepSeek-R1, a 671B parameter reasoning model developed by DeepSeek AI. This model leverages reasoning patterns from the larger DeepSeek-R1, which was trained using large-scale reinforcement learning without initial supervised fine-tuning. It is specifically fine-tuned on a Llama-3.1-8B base model using DeepSeek-R1 generated data, making it particularly effective for complex reasoning tasks in math, code, and general problem-solving with a 32768 token context length.

Loading preview...

DeepSeek-R1-Distill-Llama-8B Overview

This model is an 8 billion parameter distilled version of DeepSeek AI's DeepSeek-R1, a powerful reasoning model. DeepSeek-R1 itself was developed using a novel approach of large-scale reinforcement learning (RL) directly on a base model, without initial supervised fine-tuning (SFT), to foster advanced reasoning capabilities like self-verification and long chain-of-thought generation.

Key Capabilities & Distinguishing Features

  • Reasoning Distillation: It effectively transfers the complex reasoning patterns discovered by the much larger DeepSeek-R1 (671B parameters) into a smaller, more efficient 8B parameter Llama-3.1-8B base model.
  • RL-Driven Reasoning: Benefits from the core DeepSeek-R1 methodology, which demonstrated that reasoning capabilities can be incentivized purely through RL.
  • Performance: Achieves strong performance on reasoning benchmarks, including math (AIME 2024 pass@1: 50.4, MATH-500 pass@1: 89.1), code (LiveCodeBench pass@1: 39.6, CodeForces rating: 1205), and general reasoning tasks, often outperforming other models in its size class.
  • Context Length: Supports a substantial context length of 32,768 tokens.

When to Use This Model

  • Complex Reasoning Tasks: Ideal for applications requiring robust reasoning in domains like mathematics, programming, and logical problem-solving.
  • Resource-Efficient Reasoning: Provides high reasoning performance in a smaller package compared to its larger DeepSeek-R1 counterpart, making it suitable for environments with computational constraints.
  • Research and Development: Useful for researchers exploring distillation techniques and the transfer of advanced reasoning capabilities to smaller models.

Finetunes
1 models