Model Overview

This model, reaperdoesntknow/DistilQwen3-1.7B-uncensored, is a 1.7 billion parameter language model developed by Convergent Intelligence LLC: Research Division. It is a key component of the DistilQwen Collection, a series of models utilizing proof-weighted knowledge distillation based on Discrepancy Calculus (DISC).

Key Distinguishing Features

• Discrepancy Calculus (DISC): The model's distillation chain is founded on DISC, a measure-theoretic framework that decomposes the teacher's output distribution to quantify local structural mismatch, aiming for deeper structural understanding.
• Proof-Weighted Distillation: Employs a unique methodology with 55% cross-entropy (decaying proof weights) and 45% KL divergence, where proof weights amplify loss on reasoning-critical tokens. This forces the student model to prioritize structural understanding over superficial pattern matching.
• BF16 Training on H100: Unlike other models in the broader Convergent Intelligence portfolio, the DistilQwen series was trained on premium H100 hardware at BF16 precision, leveraging significant computational resources.
• Teacher Model: Distilled from a powerful Qwen3-30B-A3B (Instruct) teacher, inheriting its capabilities.

Primary Strengths

• Instruction Following: Optimized to accurately follow complex instructions.
• Structured Output: Excels at generating output in defined formats.
• Legal Reasoning: Specifically noted for its proficiency in legal reasoning tasks.

Methodology Details

The distillation process uses a 30B-parameter teacher model. The proof weights decay from 2.5x to 1.5x, and KL divergence is applied at a temperature of T=2.0. This approach is detailed in the paper "On the Formal Analysis of Discrepancy Calculus" (Colca, 2026) and the methodology document Structure Over Scale (DOI: 10.57967/hf/8165).