SirSahOl/DeepSeek-R1-Distill-Qwen-7B-chat-mlx-16bit

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Sep 14, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

SirSahOl/DeepSeek-R1-Distill-Qwen-7B-chat-mlx-16bit is a 7.6 billion parameter language model, converted by SirSahOl from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B, specifically optimized for native GPU inference on Apple Silicon (M1/M2/M3/M4) using the MLX framework. This 16-bit bfloat16 precision variant offers full unquantized quality with zero perplexity loss, making it ideal for research, benchmarking, and high-fidelity reference output. It supports a context length of 131,072 tokens and requires a minimum of 24GB unified memory.

Loading preview...

Overview

SirSahOl/DeepSeek-R1-Distill-Qwen-7B-chat-mlx-16bit is a specialized conversion of the deepseek-ai/DeepSeek-R1-Distill-Qwen-7B model, optimized for Apple Silicon GPUs (M1/M2/M3/M4 series) using Apple's MLX framework. This particular variant retains the full 16-bit bfloat16 precision of the original model, ensuring zero quality degradation from quantization. It features a Qwen2ForCausalLM architecture with 7 billion parameters and an extensive context length of 131,072 tokens, making it suitable for demanding tasks requiring high fidelity.

Key Capabilities

  • Full Precision Inference: Provides unquantized bfloat16 precision for maximum accuracy and zero perplexity loss.
  • Apple Silicon Optimization: Engineered for native GPU acceleration on Apple M-series chips, leveraging the MLX framework.
  • High Context Handling: Supports a substantial context window of 131,072 tokens, enabling processing of long inputs.
  • Reference Quality Output: Ideal for scenarios where the highest possible output quality and fidelity are paramount.

Good For

  • Research and Benchmarking: Provides a full-precision baseline for evaluating model performance without quantization artifacts.
  • High-End Workstation Deployment: Suitable for users with M2/M3/M4 Max/Ultra chips (32GB+ Unified Memory) seeking uncompromised performance.
  • Precision-Critical Applications: Recommended for tasks requiring the utmost accuracy, such as deep mathematical derivations or sensitive reasoning.
  • Development and Ground-Truth Comparison: Useful for prompt engineering and establishing ground-truth model comparisons.

Finetunes
1 models