Moonshot released Kimi K3 as open weights on 29 July 2026: a 2.8 trillion parameter mixture-of-experts model with 104B parameters active per token, 896 experts of which 16 fire at a time, and a 1M token context window. The weights ship in MXFP4 with MXFP8 activations, quantisation-aware from the SFT stage onward, which is what makes broad hardware compatibility possible at all.
It also means about 1.56TB of weight files. That is the number that decides this article.
This guide covers what running K3 on your own hardware actually requires: the quantisations available, how many accelerators each one needs, what that hardware costs to buy or rent, and the CPU-offload route people reach for when the GPU math stops working. It closes with what to do instead, because for most readers the hardware is the wrong place to spend the money. Featherless has served K3 at moonshotai/Kimi-K3 since launch day, with a 256K context window on the Developer plan and the full 1M on dedicated, and no logging of prompts or completions.
Why the active parameter count doesn’t save you
Mixture-of-experts models are often described as cheap to run because only a fraction of the weights fire per token. For K3 that fraction is 104B of 2.8T, roughly 3.7%. It is a real saving, and it applies to compute rather than memory.
Routing happens per token, and the router can select any 16 of the 896 experts. There is no way to predict which ones the next token needs, so every expert has to be resident and reachable at low latency. If a required expert lives on disk or in system RAM behind a PCIe link, generation stalls on every token that routes to it. So the memory you need is set by total parameters, and the compute you need is set by active parameters. K3’s compute profile is that of a 104B model. Its memory profile is that of a 2.8T one.
What each quantisation costs in accelerators
The community quantisations from Unsloth’s K3 GGUF repository are where most local attempts start. Sizes below are the published file sizes. GPU counts are the minimum to hold the weights alone, so add headroom for KV cache, activations and framework overhead before you plan a build.
QuantisationFile sizeH200s (141GB)MI325Xs (256GB)What you give upUD-Q8_K_XL1.56 TB127Nothing, this is losslessUD-Q4_K_XL1.51 TB116Very little, 50GB smaller than Q8UD-Q2_K_XL861 GB74Measurable degradation on long reasoning chainsUD-IQ1_S594 GB53Enough that benchmark comparisons stop being meaningfulUD-Q1_0466 GB42The smallest published build, and the least faithful
Two things follow from that table. Accelerators come in nodes of eight, so the counts round up in practice: Q4 on H200s is two full nodes rather than eleven cards, and Q2 fits a single 8×H200 node with about 267GB left over for KV cache. And the quantisation that fits comfortably on hardware a well-funded team might already own is the one where the model stops being the model you read the benchmarks for.
What that hardware costs
An 8-GPU HGX H200 system runs $320,000 to $420,000 in 2026, around $370,000 typical from the major OEMs, with the GPUs themselves at $32,000 to $40,000 each. Serving Q4 needs two of those, so roughly $740,000 in capital before power, cooling, networking and the engineer who keeps it running.
Renting is not the escape hatch people expect. The median on-demand H200 rate across providers is $4.29 per GPU-hour, with the cheapest verified in-stock listing at $3.00. Sixteen H200s at the median, running continuously, is about $50,100 a month. At the cheapest rate it is about $35,000. Running Q2 on a single node instead brings it to roughly $25,000 a month, and you are now serving a degraded model.
What to run instead
On Featherless, the Developer plan is $50 in credits a month, billed per token from prepaid credits that roll over, with context up to 256K and one agent environment included. The Chat plan at $25 a month is for human-driven use and excludes background automation, so it is not the plan for an agent. Start on Developer if you are writing code against the API.
Test with curl before you configure anything:
curl https://api.featherless.ai/v1/chat/completions \
-H "Authorization: Bearer $FEATHERLESS_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"moonshotai/Kimi-K3","messages":[{"role":"user","content":"hello"}]}'
If that returns a completion, every OpenAI-compatible client will work. The API lives at https://api.featherless.ai/v1, so most SDKs need only the base URL and key swapped:
from openai import OpenAI
client = OpenAI(
base_url="https://api.featherless.ai/v1",
api_key="YOUR_API_KEY",
)
response = client.chat.completions.create(
model="moonshotai/Kimi-K3",
messages=[{"role": "user", "content": "Refactor this module and explain the tradeoffs."}],
)
Our guide to K3 in opencode, Claude Code and your coding agent covers the tool-by-tool configuration from there.
When the agents run all day
The shape of it is in our GLM 5.2 against Opus writeup: a dedicated deployment on four MI325X GPUs serving an agent workload that would have cost about $150K a month in Opus API bills, for $7,592 a month flat. Dedicated also lifts K3 to its full 1M context. Against $740,000 of hardware and an engineer to run it, a flat monthly figure with the engineering team included is a different kind of decision. If your per-token bill is growing faster than your team, talk to us about a dedicated K3 deployment.
Kimi K3 local deployment FAQ
Can you run Kimi K3 on a consumer GPU? No. The smallest published quantisation is 466GB, against 32GB on the largest consumer cards. Eight of those cards still leave you short by a factor of nearly two, and that build is the least faithful of the set.
What is the smallest quantisation worth running? UD-Q2_K_XL at 861GB is the floor where K3 still behaves like K3, and it needs a full 8×H200 node. Below that, the one-bit builds degrade enough on code and tool calling that benchmark comparisons against the original stop being meaningful.
Can you run it on CPU RAM instead of GPUs? Yes, and it will be slow. Streaming roughly 52GB of active experts per token over about 576 GB/s of DDR5 bandwidth caps you near 11 tokens per second in theory and well below that in practice. It is fine for batch jobs you can leave overnight, and it is not fine for an agent.
What context window do you get? K3’s native window is 1M tokens. Featherless serves 256K on the Developer plan and the full 1M on dedicated deployments. Locally you get whatever your leftover VRAM affords after the weights, which at Q4 on two H200 nodes is not much.
Is Kimi K3 open source? The weights and code are published under Moonshot’s Kimi K3 licence on the GitHub repository. Read the licence terms before shipping on it. Open weights mean you may download and modify the model, and they say nothing about whether you can afford to serve it.
Moonshot built an open-weight model that competes at the frontier, then published 1.56 terabytes of weights that almost nobody can serve themselves. We can. Point your tools at moonshotai/Kimi-K3 on the Developer plan, and when your agents outgrow serverless, talk to us about dedicated.
Last updated: 4 September 2026. GPU prices, cloud rates and quantisation file sizes move; re-verify with each provider before committing.
Related articles
Start building under 3 minutes
