Docs /Getting Started/Completions Endpoints

Completions Endpoints

/v1/completions and /v1/chat/completions

/v1/chat/completions

The chat endpoint /v1/chat/completions allows callers to specify the prompt in terms of an exchange between a user and an assistant, allowing applications to avoid prompt formatting logic and more easily switch between models.

HTTP request

POST https://api.featherless.ai/v1/chat/completions

Request body

{
  "model": "string",
  "messages": [
    {
      "role": "system",
      "content": "string"
    },
    {
      "role": "user",
      "content": "string"
    }
  ],
  "presence_penalty": "float",
  "frequency_penalty": "float",
  "repetition_penalty": "float",
  "temperature": "float",
  "top_p": "float",
  "top_k": "integer",
  "min_p": "float",
  "seed": "integer",
  "stop": ["string"],
  "stop_token_ids": ["integer"],
  "include_stop_str_in_output": "boolean",
  "max_tokens": "integer",
  "max_completion_tokens": "integer",
  "min_tokens": "integer",
  "chat_template_kwargs": "object"
}

Parameters

Parameter

Type

Description

model

string

ID of the model to use. Specify the model to use for generating chat completions.

messages

array

A list of messages comprising the conversation so far.

presence_penalty

float

Penalizes new tokens based on their presence in the generated text so far. Values > 0 encourage new tokens; < 0 encourages repetition.

frequency_penalty

float

Penalizes new tokens based on their frequency in the generated text so far. Values > 0 encourage new tokens; < 0 encourages repetition.

repetition_penalty

float

Penalizes new tokens based on their appearance in the prompt and generated text. Values > 1 encourage new tokens; < 1 encourages repetition.

temperature

float

Controls sampling randomness. Lower values make the model more deterministic; higher values introduce randomness. Zero is greedy sampling.

top_p

float

Controls the cumulative probability of considered top tokens. Must be in (0, 1]. Set to 1 to consider all tokens.

top_k

integer

Number of top tokens to consider. Set to -1 to consider all tokens.

min_p

float

Minimum probability for a token to be considered relative to the most likely token. Must be in 0, 1. Set to 0 to disable.

seed

integer

Random seed for generation. (Not reliable, as we use multiple servers).

stop

array

List of strings that stop generation when generated. The returned output excludes these strings.

stop_token_ids

array

List of tokens that stop generation when generated. The returned output may include these tokens unless they are special tokens.

include_stop_str_in_output

boolean

Boolean to include stop strings in output text. Defaults to False.

max_tokens

integer

Maximum number of tokens generated per output sequence. A value above the model's max_completion_tokens (see /v1/models) returns a 400 error. If the prompt plus this value is longer than the model's context length, the value is lowered to fit.

max_completion_tokens

integer

Same as max_tokens, under the newer name used by the OpenAI SDK. Takes precedence when both are set. A value above the model's max_completion_tokens is lowered to that limit instead of returning an error.

min_tokens

integer

Minimum number of tokens generated per output sequence before EOS or stop_token_ids can be generated.

chat_template_kwargs

object

Optional. Model-specific chat template parameters, such as toggling reasoning. See the chat_template_kwargs guide.

Response body

If successful, the response body will contain data with the following structure:

{
  "id": "string",
  "object": "chat.completion",
  "created": "integer",
  "model": "string",
  "choices": [
    {
      "index": "integer",
      "message": {
        "role": "string",
        "content": "string"
      },
      "finish_reason": "string"
    }
  ],
  "usage": {
    "prompt_tokens": "integer",
    "completion_tokens": "integer",
    "total_tokens": "integer",
    "cached_tokens": "integer",
    "prompt_tokens_details": {
      "cached_tokens": "integer"
    },
    "cost": "number"
  }
}

usage.cost is the settled charge for the request in USD, omitted on subscription plans. cached_tokens and prompt_tokens_details.cached_tokens (the same count, where OpenAI clients read it) are the prompt tokens billed at the cached input rate.

Example request

curl https://api.featherless.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $FEATHERLESS_API_KEY" \
  -d '{
    "model": "GalrionSoftworks/Margnum-12B-v1",
    "messages": [{"role": "user", "content": "Hello!"}],
    "presence_penalty": 0.5,
    "frequency_penalty": 0.5,
    "temperature": 0.7,
    "top_p": 0.9,
    "max_tokens": 100
  }'

Streaming

Set stream: true to receive server-sent events. Each event is a data: line containing a chat completion chunk. The final chunk carries the same usage object with cost on request-pricing plans, and the stream ends with data: [DONE].

Example stream
data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":5,"completion_tokens":15,"total_tokens":20,"cached_tokens":0,"prompt_tokens_details":{"cached_tokens":0},"cost":0.000023}}

data: [DONE]

/v1/completions

The text-completions endpoint remains available, but official technical support for it has ended. Use /v1/chat/completions for supported integrations and troubleshooting.

The text completions endpoint takes a single string (the prompt) and generates a completion. This endpoint allows users to take fine-grained control of chat format or use models in non-assistant scenarios (text formatting)

HTTP Request

POST https://api.featherless.ai/v1/completions

This endpoint generates a completion for the provided prompt and parameters using the Featherless AI model.

Request Body

Parameters

Parameter

Type

Description

model

string

Required. ID of the model to use.

prompt

string | array

Required. The prompt(s) to generate completions for, encoded as a string.

When an array is supplied, each prompt is processed as a parallel inference request using the same sampling parameters. Each active request reserves the selected model’s concurrent-unit cost, and the combined usage must remain within your plan’s available allowance.

For example, with six concurrent units available and a model costing one unit per request, six prompts can run simultaneously; seven would exceed the allowance. Models with higher unit costs support fewer simultaneous prompts. See Concurrent Unit Limits for details.

presence_penalty

float

Optional. Penalizes new tokens based on their presence in the generated text so far. Values > 0 encourage new tokens; < 0 encourages repetition.

frequency_penalty

float

Optional. Penalizes new tokens based on their frequency in the generated text. Values > 0 encourage new tokens; < 0 encourages repetition.

repetition_penalty

float

Optional. Penalizes new tokens based on their appearance in the prompt and generated text. Values > 1 encourage new tokens; < 1 encourages repetition.

temperature

float

Optional. Controls the randomness of sampling. Lower values make the output more deterministic; higher values add more randomness. Zero is greedy sampling.

top_p

float

Optional. Controls the cumulative probability of the most likely tokens. Must be between 0 and 1. Setting this to 1 considers all tokens.

top_k

integer

Optional. The number of top tokens to consider in the sampling process. Set to -1 to consider all tokens.

min_p

float

Optional. Sets a minimum probability threshold relative to the most likely token for a token to be considered. Must be between 0 and 1. Set to 0 to disable.

seed

integer

Optional. Sets a random seed for generation. Not always reliable as multiple servers may be used.

stop

array

Optional. A list of strings that, when encountered in the generated output, will stop further generation. The returned output excludes these strings.

stop_token_ids

array

Optional. Similar to stop, but uses token IDs to halt generation. The output might include these tokens unless they are special tokens.

include_stop_str_in_output

boolean

Optional. If set to true, includes stop strings in the output text. Defaults to false.

max_tokens

integer

Optional. The maximum number of tokens to generate in the completion.

min_tokens

integer

Optional. The minimum number of tokens to generate before EOS or stop_token_ids can be generated.

Example Request

{
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "prompt": "Once upon a time",
  "temperature": 0.7,
  "max_tokens": 150,
  "top_p": 0.9,
  "frequency_penalty": 0.5,
  "presence_penalty": 0.0
}

Example Response

{
  "id": "cmpl-6YgK3ASw92kT14L5f8zJQ7yY",
  "object": "text_completion",
  "created": 1630569482,
  "model": "GalrionSoftworks/Margnum-12B-v1",
  "choices": [
    {
      "text": " in a land far, far away, there lived a wise old owl.",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 15,
    "total_tokens": 20,
    "cached_tokens": 0,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "cost": 0.000023
  }
}

Application Headers

For users building software that uses Featherless as an inference provider, we ask that you please set the header HTTP-Referer and X-Title in your API calls. Doing this will

  • help us support mutual users in the event of issues

  • lays the groundwork for future application-specific analytics (e.g. see what models and sampler settings are common among your users)

HTTP-Referer (and not Referer) should be the URL of your project (e.g. https://app.wyvern.chat/) and X-Title should match the value of <title> provided on that page.

e.g. the application Wyvern chat provides the following values in their API calls:

{
  "HTTP-Referer": "https://app.wyvern.chat/"
  "X-Title": "Wyvern chat"
}

Example

fetch("https://api.featherless.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${FEATHERLESS_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional: Include your app's URL for tracking
    "X-Title": `${YOUR_SITE_NAME}`,     // Optional: Identify your app in API analytics
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "GalrionSoftworks/Margnum-12B-v1",
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"},
    ],
  })
});

Notes on Sampler Parameters

  • Presence Penalty: Encourages the model to generate new tokens instead of repeating the same ones, ideal for creative writing.

  • Frequency Penalty: Helps balance repetition, useful in avoiding the overuse of specific words or phrases.

  • Temperature & Top-p: Together control the randomness and diversity of the output, crucial for customizing the creativity of the completion.

This API structure provides flexibility in generating text by allowing fine-tuning with various sampling parameters, making it adaptable for different use cases.

Last edited: Nov 13, 2026