OpenLIT

OpenLIT is an open-source observability SDK (built on OpenTelemetry) that you can integrate into your Featherless-powered application for automatic monitoring and tracing. By adding OpenLIT, you gain insight into how your application and Featherless models are performing with minimal setup.

Key benefits of this integration include:

• Auto-instrumentation – OpenLIT provides OpenTelemetry-based automatic instrumentation of Featherless API calls (no manual tracing setup required).

• Performance monitoring – Track request latency and throughput for your Featherless model calls.

• Token usage – Automatically capture token usage for each request.

• User interactions – Gain visibility into how users interact with your application’s LLM features (prompts, responses, etc.).

This integration leverages Featherless’s compatibility with the OpenAI SDK, so OpenLIT’s instrumentation hooks directly into the OpenAI client calls. (Ensure you are using OpenAI Python SDK version 1.13.0 or above for full compatibility.)

Installation

To get started, install the OpenLIT SDK in your Python environment:

This will install the OpenLIT Python package and its dependencies. (Featherless applications already use the OpenAI Python client, which OpenLIT supports out of the box.)

Initialization in Your Application

After installing, initialize OpenLIT at the start of your application before making any Featherless API calls. This one-line initialization will enable tracing for all subsequent calls made via the OpenAI/Featherless SDK:

In this example, otlp_endpoint is set to a local OpenTelemetry collector (the default endpoint when self-hosting OpenLIT is http://127.0.0.1:4318). Replace "http://127.0.0.1:4318" with the URL of your OpenTelemetry endpoint or OpenLIT collector if it differs. Once openlit.init(...) is called, OpenLIT will automatically instrument the OpenAI SDK methods that Featherless uses, capturing metrics and traces for each request..

With OpenLIT integrated into your Featherless application, you can now monitor performance and usage in real time and gain valuable insights to optimize your AI services. Happy building!

Additional Resources