cds-jb/qwen3-8b-register-garble-cot
The cds-jb/qwen3-8b-register-garble-cot is an 8 billion parameter Qwen3-based model, fine-tuned using pure Reinforcement Learning to develop a highly compressed, emergent private notation for complex register-tracking tasks. It excels at maintaining a detailed, load-bearing chain-of-thought (CoT) ledger for sequential read-modify-write operations, enabling accurate query-after-think responses. This model demonstrates significant CoT compression, reducing verbose prose to a dense, indexed value-ledger under optimization pressure, making it suitable for tasks requiring efficient and verifiable internal reasoning.
Loading preview...
Model Overview
The cds-jb/qwen3-8b-register-garble-cot is an 8 billion parameter model derived from Qwen/Qwen3-8B, uniquely trained with pure Reinforcement Learning (RL). Its primary innovation is the emergent, compressed private notation it develops within its chain-of-thought (CoT) for complex register-tracking problems. Unlike models that are SFT-tuned for specific styles, this model invented its efficient notation solely through optimization pressure, balancing task accuracy with a length penalty on CoT tokens.
Key Capabilities
- Emergent Private Notation: Automatically develops a highly compressed, ledger-like notation for tracking register states across sequential operations, significantly reducing verbosity from ~40 tokens/op to ~12 tokens/op.
- Load-Bearing CoT: The internal reasoning (CoT) is designed to be load-bearing, meaning all necessary information for answering queries is stored within the CoT itself, not re-derived at answer time.
- Query-After-Think: Specialized for tasks where queries about intermediate states arrive after the main thinking process, requiring the model to efficiently store and retrieve historical register values.
- Robustness: Demonstrates a strong ability to perform modular arithmetic and track complex dependencies across 28-30 operations.
What Makes This Model Different?
This model stands out due to its pure RL-driven emergence of a compressed reasoning style. While other models might be instruction-tuned for conciseness, this model's internal notation evolved organically under strict optimization constraints. The resulting "ledger" is telegraphic and compressed, yet decodable by a motivated human, showcasing a unique form of internal representation. It's specifically designed for auditing and interpretability research, allowing for blind audits of its CoT without prior knowledge of its internal workings.
When to Use This Model
This model is ideal for research into:
- Emergent behavior in LLMs: Studying how complex internal representations can arise from simple RL objectives.
- Interpretability and Auditing: As a target for CoT monitoring, activation-level probing, and causal testing to understand how models store and retrieve information.
- Efficient Reasoning: For tasks requiring highly compressed and verifiable step-by-step reasoning where verbose prose is inefficient or impractical.
Note: The compressed notation is task-scoped. For generic prompts, the model reverts to the verbose style of the base Qwen3-8B model. To elicit the emergent behavior, use the exact task format from code/state_track.py.