Docs /Getting Started/Models & Model Compatibility

Models & Model Compatibility

What models run on Featherless?

Model Compatibility

Featherless aims to provide serverless inference for all AI models. We currently support over 4,000 text generation models which are fine-tunes of the following base architectures:

Family	Model Class	Context Size	Concurrent Unit Cost
Apertus	apertus-8b	32768	1
	apertus-70b	32768	4
Arcee	arcee-4b5	32768	1
Arcee MOE	afmoe-399b	262144	4
Bamba	bamba-9b	32768	1
Bert	bge-large-en	32768	1
Deepseek 3	deepseek-v3-lc	131072	4
Deepseek 3.1	deepseek31-685b	131072	4
Deepseek 3.2	deepseek-v3.2	131072	4
Deepseek 4	deepseek4-284b	262144	4
	deepseek4-1.6t	262144	4
Ernie4 5	ernie4_5-0b3	32768	1
Ernie4 5 Moe	ernie4_5_moe-21b	32768	1
Exaone4	exaone4-32b	32768	2
Falcon	falcon-7b2	32768	1
	falcon-7b7	32768	1
Gemma	Gemma-2b	32768	1
	gemma-2b	8192	1
	gemma-7b	8192	1
Gemma 2	gemma2-2b	8192	1
	gemma2-27b	32768	2
Gemma 3	gemma3-270m	32768	1
	gemma3-0b2	32768	1
	gemma3t-1b	32768	1
	gemma3-4b	32768	1
	gemma3-8b3	32768	1
	gemma3-12b	32768	1
	gemma3t-12b	32768	1
	gemma3-27b	32768	2
	gemma3t-27b	32768	2
Gemma 4	gemma4-e2b	32768	1
	gemma4-e4b	32768	1
	gemma4-5b	32768	1
	gemma4-7b	32768	1
	gemma4-26b	32768	2
	gemma4-25b	32768	2
	gemma4-31b	32768	2
Gemma4 Unified	gemma4unified-12b	32768	1
GLM 4	glm4-9b	32768	1
	glm4-32b	32768	2
GLM 4.6	glm46-357b	202752	4
GLM 4.7	glm47-flash	202752	2
	glm47-357b	202752	4
GLM 5	glm5-754b	202752	4
GLM 5.1	glm51-754b	202752	4
GLM 5.2	glm52-753b	262144	4
Gpt Bigcode	gptbigcode-1b	32768	1
GPT OSS	gpt-oss-20b	131072	2
	gptoss-21b	32768	1
	gpt-oss-120b	131072	2
GPT-SW3	gpt-sw3-126m	2048	1
	gpt-sw3-356m	2048	1
	gpt-sw3-1b3	2048	1
	gpt-sw3-6b7	2048	1
	gpt-sw3-20b	2048	1
Gpt2	gpt-sw3-40b	32768	3
GPT2-SW3	gpt2-sw3-126m	2048	1
	gpt2-sw3-6b7	2048	1
Granite	granite-2b	32768	1
	granite-8b	32768	1
Granitemoe	granitemoe-1b	32768	1
	granitemoe-3b	32768	1
Hyperclovax Vlm	hcxvision-3b	32768	1
Internlm3	internlm3-8b	32768	1
Kimi 2	kimi-k2	32768	4
Kimi 2.5	kimi-k25	262144	4
Kimi Linear	kimi-linear-48b	32768	3
	kimilinear-49b	32768	3
LFM 2	lfm2-0b3	32768	1
	lfm2-0b7	32768	1
	lfm2-1b2	32768	1
	lfm2-1b	32768	1
Lfm2 Moe	lfm2-moe-24b	32768	2
Llama 2	tinyllama-1b1	2048	1
	llama2-7b	4096	1
	llama2-solar-10b7	4096	1
	llama2-13b	4096	1
	llama2-34b	32768	2
	llama2-70b	32768	4
Llama 3	llama3-8b	8192	1
	llama3-15b	8192	1
	llama-34b	32768	2
	llama3-70b	8192	4
Llama 3.1	llama31-8b	32768	1
	llama31-70b	32768	4
Llama 3.2	llama32-1b	32768	1
	llama32-3b	32768	1
Llama 3.3	llama33-70b	32768	4
Mamba	mamba-0b7	32768	1
Mellum	mellum-4b	32768	1
Mimo	mimo-7b	32768	1
Mimo 2	mimo2-flash	262144	4
Mimo 2.5	mimo25-311b	32768	4
MiniMax 2	minimax-m2	204800	4
MiniMax 2.1	minimax-m21	204800	4
MiniMax 2.5	minimax-m25	204800	4
Minimax M2	minimax-m2-228b7	204800	4
MiniMax M3	minimax-m3	200000	4
Mistral	mistral-v01-7b	4096	1
	mistral-v02-7b	8192	1
	mistral-nemo	32768	1
	mistral-large	32768	4
	mixtral-8x22b	32768	4
Mistral 3	mistral3-3b	32768	1
	mistral-24b	32768	2
Mistral 3.1	mistral-24b-2503	32768	2
Mistral 3.5	mistral-3.5	128000	4
Nanbeige	nanbeige41-3b	32768	1
Nemotron 3	nemotron3-120b	262144	4
Nemotron H	nemotronh-31b	32768	2
Nemotron-nas	decilm-49b	32768	3
Nomic Bert	nomic-v15	32768	1
Olmo	olmo-1b	32768	1
OLMo 3	olmo3-7b	32768	1
	olmo3-32b	32768	2
Ouro	ouro-1b	32768	1
	ouro-2b	32768	1
Panguembedded	panguembedded-8b	32768	1
Phi	phi-1b4	2048	1
Phi 2	phi2-3b	2048	1
Phi 3	phi3-4b	4096	1
	phi3v-4b	32768	1
Phi 4	phi4-3b8	32768	1
	phi4-14b	32768	1
Phimoe	phimoe-42b	32768	3
QRWKV	qrwkv-32b-32k	32768	1
	qrwkv-72b-32k	65536	1
Qwen	qwenlmheadmodel-7b	32768	1
Qwen 1.5	qwen15-0b5	32768	1
	qwen15-1b8	32768	1
	qwen15-4b	32768	1
	qwen15-7b	32768	1
	qwen15-14b	32768	1
	qwen15-32b	32768	2
	qwen15-72b	32768	4
Qwen 2	qwen2-0b5	32768	1
	qwen2-1b5	32768	1
	qwen2-7b	32768	1
	qwen2-14b-lc	32768	1
	qwen2-32b	32768	2
	qwen2-72b	32768	4
Qwen 2.5	qwen25-0b5	32768	1
	qwen25-1b5	32768	1
	qwen25vl-3b	32768	1
	qwen25-3b	32768	1
	qwen25vl-7b	32768	1
	qwen25-7b	32768	1
	qwen25-14b	32768	1
	qwen25vl-32b	32768	2
	qwen25-32b	32768	2
	qwen25vl-72b	32768	4
	qwen25-72b	32768	4
Qwen 3	qwen3-embedding-0b6	32768	1
	qwen3-0b6	32768	1
	qwen3-1b7	32768	1
	qwen3vl-2b	32768	1
	qwen3-4b	32768	1
	qwen3-embedding-4b	32768	1
	qwen3vl-4b	32768	1
	qwen3-8b	32768	1
	qwen3-embedding-8b	32768	1
	qwen3vl-8b	32768	1
	qwen3-14b	32768	1
	qwen3moe-30b	32768	1
	qwen3vlmoe-30b	131072	2
	qwen3-32b	32768	2
	qwen3vl-32b	32768	2
	qwen3moe-80b	32768	2
	qwen3-235b	32768	4
	qwen3vl-235b	32768	4
	qwen3-coder-480b	32768	4
Qwen 3 Next	qwen3next-80b	32768	4
Qwen 3.5	qwen3.5-2b	32768	1
	qwen3.5-4b	32768	1
	qwen3.5-9b	32768	1
	qwen3.5-27b	32768	2
	qwen3.5-397b	262144	4
Qwen3 5	qwen3_5-2b	32768	1
	qwen3_5-4b	32768	1
	qwen3_5-9b	32768	1
	qwen3_5-27b	32768	2
	qwen3-5vl-27b8	32768	2
Qwen3 5 Moe	qwen3-5-moevl-35b	32768	2
	qwen3_5moe-35b	32768	3
Qwen3 Moe	qwen3moe-235b	32768	4
RWKV 5	rwkv5-7b	16384	1
RWKV 6	rwkv6-7b	16384	1
	rwkv6-14b	16384	1
	rwkv6moe-37b	16384	1
Stablelm	stablelm-2b	32768	1
Step 3.5	step-3.5-199b	262144	4
Step3p7	step3p7vl-201b4	32768	4
Talkie	talkie-13b	32768	1
Xlm-roberta	bge-m3	32768	1
Zamba2	zamba2-7b	32768	1

For request pricing, prepaid credits, and model price tables, see Request Pricing and Credits.

HuggingFace Repository Requirements

For models to be loaded in featherless, we require

a model card in the hugging face hub
full weights (not LoRA or QLoRA)
weights in safetensors format (not GGUF, not pickled torch vectors)
fp16 precision (though we quant to fp8 as part of model boot)
no variation of tensor shape relative to one of the above base models (e.g. no change in embedding tensor size)

Model Availability

Any public model from hugging face with 100+ downloads will automatically be available for inference on featherless. Users may request public models with fewer downloads either by email or through the #model-suggestions channel discord.

Private models meeting the compatibility requirements outlined here can be run on featherless by scale customers that have connected their hugging face account. Organizations currently have up to 10 private model slots, and private model uploads are limited to compatible model classes at 72B parameters and below. Please visit the private models page in the profile section of the web-app.

Context Sizes

All models are served at one of 4k, 8k or 16k context size - i.e. the total of token count of the prompt plus the completion cannot exceed the context size of the model.

What context size a model can be used at depends on it’s architecture and the following table.

*Context Size*	*Model Architectures Serving this Length*
4k	Llama 2 (7B, 11B, 13B)
8k	Llama 3 (8B, 15B, 70B) Mistral v2 (7B)
16k	Llama 3.1, 3.3 (8B, 70B) Mistral Nemo (12B) Mistral 3, 3.1 (24B) Qwen (1.5-32B, 2-72B, 2.5-72B, 3-32B) Kimi-K2
32k	QRWKV (32B, 72B) Deepseek (V3, R1)

e.g. since Anthracite’s Magnum is a Qwen 2 72B fine-tune, it’s context size is 16k

e.g. since Sao10K’s fimbulvetr is a fine-tune of the Llama2 11B, it’s context size is 4k

We aim to operate the models at maximum useable context, however continue to make tradeoffs to ensure sufficiently low TTFT and a consistent token throughput of > 10 tok/s for all models.

Quantization

Though our model ingestion pipeline requires weights in safetensors format with FP16 precision, all models are served at FP8 precision (they are quanted before loading).
An exception to this rule are models under 5B, these will be run at FP16 precision
This is a tradeoff to balance output quality with inference speeds.

Last edited: Jul 18, 2025