reaperdoesntknow/DistilQwen3-1.7B-uncensored

DistilQwen3-1.7B-uncensored Overview

This model, developed by Convergent Intelligence LLC: Research Division, is a 1.7 billion parameter language model from the DistilQwen3 Series. It leverages Discrepancy Calculus (DISC), a unique measure-theoretic framework for knowledge distillation, which quantifies local structural mismatch that standard KL divergence typically averages away. The model was trained on H100 GPUs at BF16 precision, using a 30B-parameter teacher model, a significant upgrade in compute compared to other models in the broader Convergent Intelligence portfolio.

Key Capabilities

• Advanced Knowledge Distillation: Utilizes proof-weighted knowledge distillation (55% cross-entropy with decaying proof weights, 45% KL divergence at T=2.0) to emphasize reasoning-critical tokens.
• Structural Understanding: The proof weighting mechanism forces the student model to prioritize structural understanding over surface-level pattern matching.
• Efficient Inference: As a 1.7B parameter model, it offers a balance between performance and computational efficiency, benefiting from distillation from a larger 30B teacher.
• Uncensored Variant: Designed without explicit content filtering, offering broader applicability for research and development where unconstrained output is required.

Good For

• Research in Knowledge Distillation: Ideal for exploring the effects of Discrepancy Calculus and proof-weighted training on model performance.
• Applications Requiring Logical Inference: Suitable for tasks where robust structural understanding and reasoning are paramount.
• Resource-Constrained Environments: Provides strong capabilities in a smaller parameter footprint, making it efficient for deployment.
• Unfiltered Content Generation: Useful for use cases where censorship might hinder desired outputs, such as creative writing or specific research scenarios.