Tool Calling
How to use tool and function calling
Tool calling (or function tools) give the models access to external tools. The LLM does not call the tool directly. Instead, it will suggest the tool to call, from a pre-defined list of tools. This tool then gets called seperately and provides the result back to the LLM which then formats the response into an answer to the original user’s question.
As of now tool calling is not officially supported by Featherless but plans for this are on the roadmap.
As for now we have some notebooks available in our Featherless cookbook that show you how to simulate tool calling. These notebooks demonstrate how to simulate function calling using the OpenAI Python SDK. This is an alternative to OpenAI’s built-in function calling and works with any model that can follow instructions and output JSON.
We use the response_format={"type": "json_object"} parameter to ensure the model outputs valid JSON, and we instruct the model via the system prompt to only output the required JSON structure.