groxaxo/Qwento-Agentic
Qwento-Agentic is a 35.1 billion parameter QLoRA fine-tune by groxaxo, built on Qwen/Qwen-AgentWorld-35B-A3B, a Qwen3.5 MoE architecture with 256 frozen experts. This experimental checkpoint, with a 32768 token context length, is specifically designed and trained on curated datasets for coding tasks, including reasoning, trace-inversion, and multi-turn agentic coding. It focuses on enhancing coding capabilities through specialized data and a merged BF16 weight format.
Loading preview...
Qwento-Agentic: An Experimental Agentic Coding Model
Qwento-Agentic is an early, experimental QLoRA fine-tune by groxaxo, built upon the Qwen/Qwen-AgentWorld-35B-A3B base model. This model leverages a Qwen3.5 MoE architecture (35.1B total parameters, ~3B active) with 256 frozen experts and a 32768 token context length. The fine-tune was merged into BF16 weights, making it a drop-in replacement for 🤗 Transformers and vLLM.
Key Characteristics & Training
This model is a test run from a short curriculum stage, focusing on enhancing agentic coding capabilities. It was trained using QLoRA (rank 16, alpha 32) applied to the model's sequence-mixing path across all 40 layers. The training utilized a carefully curated and token-balanced blend of publicly available datasets, including:
- Reasoning/Problem Solving: Datasets like
Jackrong/Claude-opus-4.7-TraceInversion-5000xandlordx64/reasoning-distill-claude-opus-4-7-maxfor high-quality reasoning traces. - Coding Traces:
Infatoshi/kernelbench-mega-tracesfor GPU-kernel coding. - Agentic Coding:
Glint-Research/fable-5-tracesfor multi-turn agentic coding with tool use.
All training data underwent structural cleaning and quality filtering to ensure high relevance for coding tasks.
Intended Use & Limitations
Qwento-Agentic is specifically designed for coding tasks, particularly those involving reasoning, trace-inversion, and multi-turn agentic interactions. As a preliminary test checkpoint, it has not yet been benchmarked and should be treated as experimental. Users should be aware that it inherits the license and usage restrictions of its base model, Qwen/Qwen-AgentWorld-35B-A3B.