Model Overview
MuXodious/Nemotron-Cascade-14B-Thinking-impotent-heresy is a 14 billion parameter model fine-tuned from the Nemotron-Cascade-14B-Thinking base, which itself is post-trained from Qwen3-14B Base. This model was created using P-E-W's Heretic engine, specifically incorporating a Magnitude-Preserving Orthogonal Ablation PR. It is exclusively designed for the thinking mode, distinguishing it from other Nemotron-Cascade variants.
Key Capabilities & Training
- Reinforcement Learning: The model is trained through sequential and domain-wise reinforcement learning, starting with a multi-stage SFT phase for foundational skills, followed by Cascade RL across multiple domains.
- Reasoning Focus: It is optimized for complex reasoning abilities, with RLHF boosting performance beyond mere preference optimization.
- Benchmark Performance: Achieves strong results across diverse benchmarks including knowledge reasoning, alignment, mathematics, and competitive programming. Notably, it surpasses DeepSeek-R1-0528 (671B) on LiveCodeBench v5, v6, and Pro benchmarks.
- Context Length: Supports a 32768 token context length, with recommendations for YaRN scaling to extend this to 64K or even 90K tokens for specific tasks like SWE Verified.
Usage Recommendations
- Sampling Parameters: Recommended settings are
temperature = 0.6 and top_p = 0.95. - Thinking Mode: The model is designed for a 'thinking' mode, requiring the
" /think" tag appended to user input in single-turn conversations. For multi-turn, " /no_think" is used for previous user turns to manage context length.
Heretication Status
The model is classified with an "Impotent Heresy" index due to 33/100 refusals and 0.0000 KL Divergence, indicating a specific fine-tuning outcome related to its 'heretication' process, though this classification is noted as arbitrary and not indicative of performance.