Overview

This model, sourcepirate/Qwen2.5-Coder-1.5B-Instruct-heretic, is a 1.5 billion parameter instruction-tuned causal language model derived from the Qwen2.5-Coder series by Qwen. It is a decensored version of the original Qwen/Qwen2.5-Coder-1.5B-Instruct, processed with Heretic v1.2.0. The Qwen2.5-Coder family, formerly known as CodeQwen, focuses on code-specific applications.

Key Capabilities & Features

• Decensored Variant: This model exhibits a significantly lower refusal rate (3/100) compared to the original (95/100), making it less prone to content restrictions.
• Code-Specific Optimization: Built upon the strong Qwen2.5 foundation, it is specifically enhanced for code generation, code reasoning, and code fixing.
• Architecture: Utilizes a transformer architecture with RoPE, SwiGLU, RMSNorm, Attention QKV bias, and tied word embeddings.
• Context Length: Supports a substantial context window of 32,768 tokens.
• Parameter Count: Features 1.54 billion parameters, with 1.31 billion non-embedding parameters.

Good For

• Code Generation: Ideal for tasks requiring the creation of programming code.
• Code Reasoning: Suitable for understanding and analyzing code logic.
• Code Fixing: Useful for identifying and correcting errors in code.
• Applications Requiring Fewer Refusals: Developers needing a model with less content filtering for specific coding or general instruction-following tasks.