amao0o0/InfoDensity-DeepSeek-R1-Distill-Llama-8B

TEXT GENERATIONPricing:Input $0.2 / Cached $0.028 / Output $0.32Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 2, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

amao0o0/InfoDensity-DeepSeek-R1-Distill-Llama-8B is an 8 billion parameter language model, fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Llama-8B, that leverages an InfoDensity reinforcement learning reward to generate more information-dense and efficient reasoning traces. This model excels in complex reasoning tasks, achieving higher accuracy with significantly shorter response lengths across benchmarks like AMC23, AIME24, MATH500, and GPQA-Diamond. It is particularly optimized for efficient problem-solving where concise yet accurate reasoning is critical, featuring a 32768 token context length.

Loading preview...

Overview

amao0o0/InfoDensity-DeepSeek-R1-Distill-Llama-8B is an 8 billion parameter language model derived from deepseek-ai/DeepSeek-R1-Distill-Llama-8B. Its core innovation lies in its training with InfoDensity, a reinforcement learning reward designed to favor information-dense reasoning traces. This reward mechanism combines an entropy-trajectory quality term with a group-relative length scaling term, applied exclusively to traces that lead to correct answers. The result is a model that achieves correct solutions with notably less deliberation and shorter output lengths.

Key Capabilities & Performance

This model demonstrates superior performance in complex reasoning tasks compared to its base model, as evidenced by its results across multiple benchmarks:

  • Improved Accuracy: Achieves a 58.4% accuracy (pass@1) averaged over AMC23, AIME24, MATH500, and GPQA-Diamond, a significant increase from the base model's 47.6%.
  • Reduced Length: Generates responses with a mean length of 6.5k tokens, substantially shorter than the base model's 9.4k tokens, indicating more efficient reasoning.
  • Enhanced Efficiency: Boasts an Accuracy–Efficiency Score (AES) of +0.99, highlighting its ability to deliver both accuracy and conciseness.

Good For

  • Applications requiring efficient and concise reasoning for complex problems.
  • Scenarios where reduced token generation is beneficial for speed or cost, without sacrificing accuracy.
  • Tasks involving mathematical problem-solving and general question answering where information density is key.

Finetunes
1 models