# decisio-gemma-4-12b Built by [Tachara AI Lab](https://huggingface.co/tachara-ai). ![How Decisio works: your text and your questions go in, a probability for every answer comes out](assets/decisio_article_corrected.png) *How it works (illustration; the probabilities are examples, real recorded responses are on the 31B card).* **Typed questions about any text in, a calibrated probability for every option out: one forward pass, no generated text.** ## Teach it your task **Teach it your task in minutes. No retraining.** Your labels, your rules, and it tells you which answers it is sure of. - **BANKING77: 0.747 to 0.832 after 10 labelled examples per intent (mean of 6 draws, 150 test items).** Intent accuracy on the Gemma 4 12B, the test items held out. - **CLINC150: 0.850 to 0.908 after 10 labelled examples per intent (mean of 6 draws, 100 test items).** The same, on 150 options, the test items held out. - **169 of 200 held-out tickets certified, 17 wrong.** On a ticket question (the Gemma 4 12B, on a Mac) with 400 labelled tickets, asking for at most 15% wrong. Every answer says `certified: true` or `certified: false`. How it works: you send labelled examples of your own question to `POST /v1/tasks`. The server fits a small correction to the frozen model's answers, checks it against your examples, and keeps it only if it helps. The tradeoff: you need labelled examples (10 per option for the intent head, and 59 held out to certify at the defaults, so 118 in all), and a certificate holds for one task on one serving setup. It is a rate over many answers, not a promise about one. **Under the hood.** `POST /v1/tasks` fits, in seconds to minutes and from a handful of your examples, a calibration, an intent head for questions with 10 or more options, a per-task temperature and, if you ask, certified answers. The weights are not touched, and each correction is kept only if cross-validation on your examples shows it helps. The per-task temperature and certified answers need decisio 0.12.0 or later. Certified means this: ask for an error rate `alpha` (and a confidence `delta`), and the server finds the confidence above which at most that share of the answers are wrong. In the recorded example (the Gemma 4 12B on a Mac, `alpha` 0.15) the threshold 0.986 kept 169 of 200 held-out tickets with 17 wrong, and the promise is that with probability at least 90% at most 15% of the answers above it are wrong. Below 59 held-out examples, at the defaults, the server refuses to certify. The intent numbers above are from the [record](https://github.com/aminry/decisio/tree/main/runs/2026-10-04_gemma-base) (six draws). They use labels, so they compare the taught server with the untaught one and not with systems that see no labels. How to register a task end to end, and where the promise stops: [`docs/tasks.md`](https://github.com/aminry/decisio/blob/main/docs/tasks.md). A dot plot of JevBench v1.6.1 composite scores with 95% intervals for ten open-weights systems in alphabetical order, with Jev 1.13.0 below as a reference row. Decisio on Gemma 4 31B scored 71.69 (interval 69.86 to 72.81) and Jev 1.13.0 71.49 (69.14 to 72.40); the two intervals overlap.

A dot plot of JevBench v1.6.1 composite scores with 95% intervals for ten open-weights systems in alphabetical order, with Jev 1.13.0 below as a reference row. Decisio on Gemma 4 31B scored 71.69 (interval 69.86 to 72.81) and Jev 1.13.0 71.49 (69.14 to 72.40); the two intervals overlap.

Gemma 4 12B for [decisio](https://github.com/aminry/decisio), an open-source server that takes a text state and typed questions (yes/no, a choice among named options, a score on a scale) and returns a probability for each option. The 12B is the base for smaller machines: on a 16 GB laptop through Ollama, and on a Mac from its MLX conversion. This repository holds the weights, byte for byte the source's, and `decision_config.json`, the settings decisio serves this base with. It is versioned to decisio releases: `decision_config.json` names the release it was made for (decisio 0.11.1; the revision decisio 0.11.1 pins is tagged `v0.11.1`), and with it this base is served with the registration before the question by default (`--register-boundary after` still selects the earlier order); the revision decisio 0.11.0 pins, tagged `v0.11.0`, has the same weights and names `after`. | | | | --- | --- | | JevBench v1.6.1 | 67.76 (decisio 0.9.0; 95% interval 54.38 to 69.56) | | Calibration | ECE 0.033 standard, 0.085 hard | | New 300-token state | 53.6 ms | | Second question | 35.8 ms | | Memory | 22.8 GiB (bf16) | | Licence | Apache 2.0 | ## Quickstart ### On a laptop, with Ollama ```bash ollama pull aminroudaki/decisio-gemma # Gemma 4 12B; 8.6 GB in Ollama at its smallest tag, so a 16 GB laptop curl http://localhost:11434/v1/systemone -d '{ "model": "aminroudaki/decisio-gemma", "state": "Hi, since this morning none of our 40 staff can log in to the dashboard. We get \"session expired\" right after entering the password. Payroll is due today.", "questions": { "urgent": {"type": "noul", "instructions": "Does this ticket need a response within the hour?"}, "category": {"type": "choice", "instructions": "Which team should handle this ticket?", "criteria": {"billing": "Invoices, payments, refunds", "access": "Login, passwords, permissions", "bug": "The product behaves wrongly", "other": null}} } }' ``` Requires Ollama 0.35.1 or later; the request was checked against Ollama 0.35.1 on a Mac, where the smallest tag takes 8.6 GB in Ollama, so it should fit a 16 GB laptop. Ollama builds its own prompt and applies no calibration, so its numbers are the ones on its model page, not the ones on this card. ### On a GPU, with pip One Linux host with an NVIDIA GPU, Python 3.12: ```bash python3.12 -m venv .venv && . .venv/bin/activate pip install "decisio[serve]" python -m decisio.serve.vllm_engine --base tachara-ai/decisio-gemma-4-12b curl -s http://127.0.0.1:8000/health # answers once it is ready ``` The first start downloads this repository's weights and warms the engine. Measured on 2026-10-09 on decisio 0.11.0 from PyPI (one RTX PRO 6000 at 600 W, an AMD EPYC 9534, one session's network): `pip install` 124 s, and the first start of `--base gemma-4-12b` ready after 4 min 12 s, including its download. The measured environment is decisio's lockfile (`uv sync --extra serve --frozen`); pip resolves its own versions around vLLM 0.30.0. `--base tachara-ai/decisio-gemma-4-12b@` pins a commit. The server refuses to start if this repository's `decision_config.json` differs from the installed decisio's profile of the base, naming the differing keys. ### With Docker ```bash docker run -d --gpus all --shm-size 8g -p 127.0.0.1:8000:8000 -v decisio-data:/data \ -e DECISIO_BASE=tachara-ai/decisio-gemma-4-12b ghcr.io/aminry/decisio:0.11.1 ``` Then ask a question about any text: ```bash curl -s http://127.0.0.1:8000/v1/systemone -H 'Content-Type: application/json' -d '{ "state": "I was charged twice for October. Both charges are 49.00 EUR, same day. Please refund one.", "questions": { "queue": {"type": "choice", "instructions": "Which team should handle this ticket?", "criteria": {"billing": "Invoices, payments, refunds", "access": "Login, passwords, permissions", "bug": "The product behaves wrongly", "other": null}} } }' ``` ## Worked decisions The three requests of decisio's README (a support ticket, a message gate, an assistant's next step), sent to a freshly started server on this base, `--base gemma-4-12b`: it is served from `google/gemma-4-12B-it`, the checkpoint this repository copies, with the registration after the response (`--register-boundary after`), which was this base's default on decisio 0.11.0. Since decisio 0.11.1 the default is `before`, with `after` still selectable (`--register-boundary after`); a first read then goes through the prefix cache, so a question asked again returns its first answer. Probabilities are rounded to three decimals; the files with the full-precision responses are in decisio's [`runs/2026-10-09_readme-pip-path/`](https://github.com/aminry/decisio/tree/main/runs/2026-10-09_readme-pip-path).

1. Route a support ticket: three questions, one pass (server time 108.8 ms on the first send, 83.0 ms on its repeat)

Recorded 2026-10-09 on decisio 0.11.0, vLLM 0.30.0, google/gemma-4-12B-it at 707f0a3b, one NVIDIA RTX PRO 6000 Blackwell Workstation Edition at 600 W, AMD EPYC 9534; served from `google/gemma-4-12B-it` with the registration after the response (`--register-boundary after`), this base's default on 0.11.0. Request: ```json {"state": "I was charged twice for October. Both charges are 49.00 EUR, same day. Please refund one.", "questions": {"urgent": {"type": "noul", "instructions": "Does this ticket need a response within the hour?"}, "queue": {"type": "choice", "instructions": "Which team should handle this ticket?", "criteria": {"billing": "Invoices, payments, refunds, prices, VAT", "access": "Login, passwords, two-factor, SSO, permissions, invitations, account security", "bug": "The product behaves wrongly: errors, crashes, wrong numbers, lost data", "feature": "A request for something the product does not do yet", "cancellation": "Cancelling, pausing or closing an account, deleting data", "other": "Press, partners, jobs, sales pitches, wrong address, thanks, general questions"}}, "impact": {"type": "score", "instructions": "Rate the business impact using only the reported facts.", "criteria": ["No function impaired", "One user impaired, with a workaround", "Many users blocked from a core function", "Data loss or legal exposure"]}}} ``` Response: ```json { "model": "decisio-gemma-4-12b-it-letters", "answers": { "urgent": { "type": "noul", "noul": 0.052 }, "queue": { "type": "choice", "choice": "billing", "confidence": 0.981, "probabilities": { "billing": 0.984, "access": 0.007, "bug": 0.003, "feature": 0.002, "cancellation": 0.002, "other": 0.002 } }, "impact": { "type": "score", "score": 0.714, "confidence": 0.555, "legend": { "0": "No function impaired", "1": "One user impaired, with a workaround", "2": "Many users blocked from a core function", "3": "Data loss or legal exposure" }, "probabilities": { "0": 0.365, "1": 0.574, "2": 0.042, "3": 0.019 } } }, "usage": { "input_tokens": 240, "output_tokens": 3 } } ``` It routes the ticket to billing with 98% on the queue and puts the chance it needs a response within the hour at 0.05; the impact is most likely "one user impaired, with a workaround" at 57%.

2. Gate a message: a yes/no with a threshold you choose (server time 34.0 ms on the first send, 44.6 ms on its repeat)

3. Decide what an assistant does next (server time 131.2 ms on the first send, 43.5 ms on its repeat)

## Question types | Type | What you send | What comes back | | --- | --- | --- | | `noul` (yes or no) | an `instructions` question, optionally a `criteria` for true and false | `noul`: the probability of yes | | `choice` | `instructions` and `criteria`, named options (up to 255) | `choice` (the most likely option), `confidence`, and `probabilities` for every option | | `score` | `instructions` and `criteria`, an ordered list of levels | `score` (the expected level, counting from 0), `confidence`, a `legend` of the levels and `probabilities` for every level | Many questions go in one request: the state is read once and each question is answered as if it had been sent alone. Nothing is generated: `usage.output_tokens` is 0 for the text route. ## Check your setup A correct setup scores **200 of 231** on JevBench's public items: 231 published questions, scored through decisio's server on Gemma 4 12B at its served defaults. From a decisio checkout, with the server on port 8000 (the harnesses are the public ones, fetched by the first script): ```bash bash benchmarks/fetch_harnesses.sh JEVBENCH= bash benchmarks/jevbench.sh http://127.0.0.1:8000 ``` If your count is lower, something in the setup differs from the one measured: the checkpoint, the prompt, the temperatures or the precision. `GET /health` shows what the server is serving. The figure is our own reading through the public harness, not a board score. ## Results ### JevBench On [JevBench v1.6.1](https://benchmarkheaven.com/api/jevbench/v1.6.1) the board's row for this base served by decisio 0.9.0 scores 67.76 on the headline composite (95% interval 54.38 to 69.56), measured on one RTX 6000 Ada. Its interval is wide and overlaps the 31B row's and Jev's. On the v1.6.1 file the 31B base's row (decisio 0.8.0) scores 71.69 on the composite and this base's row (decisio 0.9.0) 67.76. A dot plot of the intelligence axis of JevBench v1.6.1 for eleven systems in alphabetical order, each with its sealed-set score and its open-set score. Decisio on Gemma 4 31B scored 73.3 on the sealed set and 67.6 on the open set.

A dot plot of the intelligence axis of JevBench v1.6.1 for eleven systems in alphabetical order, each with its sealed-set score and its open-set score. Decisio on Gemma 4 31B scored 73.3 on the sealed set and 67.6 on the open set.

Decisio scores higher on the sealed items than on the open ones. ### Latency and hardware A second, different question on a 3,000-token state already read takes 35.8 ms after a 260 ms first read, with the state read from the cache (server time; one RTX PRO 6000 Blackwell at 585 W on an AMD Ryzen Threadripper 9960X host). One question on a new state takes 53.6 ms at 300 tokens and 258.6 ms at 3,000, and a question on a 1,000-token state from the prefix cache takes 24.8 ms (server time, same card and host). Memory: on the 96 GB card, vLLM's 0.90 share leaves 57.4 GiB for the KV cache after the 22.8 GiB of bf16 weights. On vLLM the 12B did not start on a card limited to 32 GB, and cards between 32 and 96 GB were not measured (decisio's EVAL_CARD section 6.6). We did not measure a separate peak at a typical context. ## Intended use - Classifying, routing and triaging text against options you define in each request, on smaller hardware than the 31B needs. - Many questions about one text in a single request. - Rating on an ordered scale, with the uncertainty attached. ## Out of scope - The sole basis for a decision about a person's access to credit, work, housing, healthcare, education, legal status or benefits. - Any text generation. - Images: the image route was not measured with this base. - Languages other than English were not evaluated. ## Limitations - On the same questions it is less accurate than the 31B on every accuracy measure we have (EVAL_CARD sections 6 and 7). - A cached read and a fresh read of the same question can differ, by up to 0.113 in our measurement, so with `--register-boundary after`, a question asked again can differ from its first answer by that much; since decisio 0.11.1 the default is `before`, with which the repeat equals the first answer (EVAL_CARD 6.5). - One card and one session per measurement. - A prompt over the server's `max_model_len` (at least 32,768 and at most 65,536 on this base; `/health` reports it and why) is refused with HTTP 422 (decisio 0.14.0). - A probability is not an explanation. ## The weights are unmodified They are `google/gemma-4-12B-it` at revision `707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7`, byte for byte (the Hub's own sha256 for each of the source files equals this repository's). `LICENSE`, `NOTICE`, `SHA256SUMS` and `decision_config.json` were added by the decisio project and are not part of Google's release; this card replaces the source's. ## What the weights do and what the server does The weights load in vLLM as the source checkpoint does. The decision readout, the prompt, the temperatures and the letter forms are in `decision_config.json` and are applied by the server. No reference code is in this repository yet; a plain-transformers implementation is planned and will be added once it passes decisio's conformance gate against the server. ## Files `LICENSE`, `NOTICE`, `SHA256SUMS` (the model files' hashes), `decision_config.json` (the settings the server serves this base with), the model files, `assets/` (the charts on this card), and this card. ## Licence Apache License 2.0, in `LICENSE`. `NOTICE` names the source repository and revision. ## Citation ```bibtex @software{tachara_ai_lab_decisio_gemma_4_12b, author = {{Tachara AI Lab}}, title = {decisio-gemma-4-12b: Gemma 4 12B for decisio}, year = {2026}, version = {0.11.1}, url = {https://huggingface.co/tachara-ai/decisio-gemma-4-12b} } ```

Full Model Card (README)