Models & Model Compatibility

Model Compatibility

Featherless serves 50,000+ open-source models. Browse the model catalog for the current selection. Supported text-generation model architectures are listed below.

For request pricing, prepaid credits, and model price tables, see Developer and Credits.

HuggingFace Repository Requirements

For models to be loaded in featherless, we require

• a model card in the hugging face hub

• full weights (not LoRA or QLoRA)

• weights in safetensors format (not GGUF, not pickled torch vectors)

• fp16 precision (though we quant to fp8 as part of model boot)

• no variation of tensor shape relative to one of the above base models (e.g. no change in embedding tensor size)

Model Availability

Any public model from hugging face with 100+ downloads will automatically be available for inference on featherless. Users may request public models with fewer downloads either by email or through the #model-suggestions channel discord.

Private models meeting the compatibility requirements outlined here can be run on featherless by scale customers that have connected their hugging face account. Organizations currently have up to 10 private model slots, and private model uploads are limited to compatible model classes at 72B parameters and below. Please visit the private models page in the profile section of the web-app.

Chat Plan Context Sizes

On the Chat plan, models are served with 4K, 8K, or 32K context windows, depending on their architecture. The combined token count of your prompt and completion cannot exceed the model's context window.

The following table applies specifically to the Chat plan. See Developer Plan Context Sizes below for models with higher limits.

e.g. since Anthracite’s Magnum is a Qwen 2 72B fine-tune, its context size on the Chat plan is 32K

e.g. since Sao10K’s fimbulvetr is a fine-tune of the Llama2 11B, it’s context size is 4k

We aim to operate the models at maximum useable context, however continue to make tradeoffs to ensure sufficiently low TTFT and a consistent token throughput of > 10 tok/s for all models.

Developer Plan Context Sizes

Modern model architecture finetunes and abliterations are generally served with a 32K context window on the Developer plan. Selected original models support larger context windows, as listed below.

These limits describe the context served by Featherless, which may be lower than a model’s published maximum. Your prompt and completion must fit within the context window; maximum output length is a separate limit.

Quantization

Though our standard model ingestion pipeline requires weights in safetensors format with FP16 precision, most models are served at FP8 precision (they are quantized before loading).
Models under 5B are served at FP16 precision. Model-specific exceptions also apply: Kimi models that use quantization-aware training (QAT) are served at MXFP4 precision. QAT accounts for reduced precision during training, rather than relying solely on quantization after training. Unless otherwise stated on a model’s catalog page, serving precision is FP8 or FP16.
This is a tradeoff to balance output quality with inference speeds.