natzx94/shiori-router-270m

TEXT GENERATIONConcurrent Unit Cost:1Model Size:0.3BQuant:BF16Context Size:32kPublished:Jul 17, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

natzx94/shiori-router-270m is a 270M-parameter model, fine-tuned from google/gemma-3-270m, designed for model-routing policy decisions. It maps a task and a registry of available models to a JSON output specifying an execution mode (DIRECT, TOOL_CALL, PLAN) and the minimal-cost model_id. This model excels at intelligently routing tasks to appropriate backend models based on their capabilities and cost, making it ideal for local inference environments requiring dynamic model selection.

Loading preview...

Overview

natzx94/shiori-router-270m is a 270M-parameter model, a full fine-tune of google/gemma-3-270m, specifically engineered to act as a model-routing policy. Its core function is to analyze a given task and a registry of available models, then output a JSON object containing a mode (DIRECT, TOOL_CALL, or PLAN) and a model_id. This model_id corresponds to the minimal-cost model from the registry whose capabilities are sufficient for the task.

Key Capabilities

  • Intelligent Task Routing: Determines the most suitable model and execution mode for a given task from a dynamic registry.
  • Fixed Output Contract: Generates a structured JSON output {"mode": "...", "model_id": "..."} for consistent parsing.
  • Constrained Decoding: Optimized for local runtimes (llama.cpp, Ollama, LM Studio) with constrained decoding to ensure valid output modes and model IDs.
  • Configurable Planning Recall: Features a plan_prior parameter to adjust PLAN mode recall at inference time, trading off against DIRECT/TOOL_CALL recall and mean cost.
  • Efficient Performance: Achieves low latency, with Q8_0 and Q4_K_M quantized versions processing requests in ~500ms on CPU and ~35ms on GPU.

Good For

  • Dynamic Model Selection: Applications requiring an on-device, intelligent agent to choose the best backend model for diverse tasks.
  • Cost Optimization: Identifying the minimal-cost model capable of handling a specific task.
  • Local Inference Environments: Designed for efficient operation within local runtimes like Ollama and LM Studio.
  • Tool-Calling and Planning Orchestration: Facilitating decisions for tool-use and multi-step planning workflows.

Finetunes
1 models