Running Open Source LLMs in Popular AI Clients with Featherless: A Complete Guide
A step-by-step walkthrough for developers and AI enthusiasts to integrate any Hugging Face model into their favorite chat client.
Introduction
Remember the excitement of discovering powerful open-source language models like the latest Qwen or LLama 3 AI models, only to face the daunting challenge of actually running them?
You’re not alone. While these models offer amazing capabilities, deploying them efficiently has been a significant hurdle for developers and enthusiasts alike. Today, we’re changing that with Featherless. In this guide, you’ll learn how to integrate the latest cutting-edge AI models into your favorite chat clients - whether you’re building an ai agent, a coding assistant, roleplaying, or setting up a general chat interface. I’ll walk you through practical, step-by-step instructions for popular platforms like SillyTavern, WyvernChat and Typing Mind showing you how to leverage our serverless infrastructure. No GPU or server management required, no complex deployments - just powerful language models ready to use in the tools you already know and love.
Getting started with Featherless
Featherless is your gateway to using powerful open-source language models in your favorite applications. As a serverless inference platform, we make it simple to access the latest models without managing any infrastructure.
OpenAI compatibility
Our API is fully OpenAI-compatible. This means any application that works with OpenAI can be easily reconfigured to use Featherless.
Quick Setup API
1. Sign up for a Featherless account
2. Subscribe to a plan that fits your needs
3. Navigate to the API Keys section in your dashboard
4. Create a new API Key
That’s it! You’re ready to use this API Key in your preferred chat client
Choose your plan
Select the plan that fits your needs:
🌱 Featherless Basic ($10/month)
All models up to 15B parameters
2 concurrent requests
Unlimited monthly usage
⭐ Featherless Premium ($25/month)
All models up to 72B parameters
Everything in Basic
Perfect for power users
🚀 Featherless Scale ($75 per unit/month)
Everything in Premium
2x Premium or 6x Basic model concurrency per unit
Host private models from Hugging Face
🔒 Privacy First: We never log chats, prompts, or completions. Your conversations stay private.
Ready to begin?
Head to the specific integration guide for your preferred client, or join our Discord for support.
Role-playing client integration
Having spent countless hours lost in AI roleplay conversations, let me tell you - there’s nothing quite like that moment when your character truly comes alive. Whether it’s the witty jokes that feel spontaneous, or those surprising, deep exchanges that make you forget you’re talking to an AI.
From creating more complex characters with a deep lore to vibrant anime personalities, what I’ve learned is that finding the right model isn’t just a technical choice - it’s about finding a perfect way to show off the identity and personality behind your character. That’s why I’m so excited about our growing catalog of open-source models. Each one brings something special and different to your character interactions and creative writing, I’ve seen incredible roleplay scenes emerge from unexpected model choices.
Let me now walk you through integrating Featherless with some of your favorite roleplay clients.
SillyTavern
With SillyTavern it’s pretty easy to create a connection to Featherless. Simply click on the plug icon at the top and make the following selections:
API: Chat Completion
Chat Completion source: Custom (OpenAI-compatible)
Custom Endpoint: https://api.featherless.ai/v1
Custom API Key: Your Featherless API key
Enter a Model ID: A model chosen from our model catalog (e.g. meta-llama/Meta-Llama-3.1-8B-Instruct)
Connect!
Once connected, you’ll see a green status indicator after which you’ll be able to send a message to your characters to ensure everything is working properly. You should receive a response within seconds.
WyvernChat
WyvernChat offers native Featherless integration right out of the box. The built-in support means you’ll spend less time configuring and more time chatting. WyvernChat provides a streamlined setup process. Whether you’re new to AI chat platforms or migrating from another client, you’ll appreciate how seamlessly Featherless meshes with WyvernChat’s clean interface. All you need is your Featherless API Key and following steps:
Head on over to https://app.wyvern.chat/
At the bottom right of the page click on the plug icon
Click on ‘+ Add connection’
Select ‘Featherless’ from the Type dropdown
Password (API Key)*: Your Featherless API key
Select a model from the list at the bottom
Scroll down and press ‘Create’
Once your connection is created head over to any of your character chats, then at the bottom right under ‘Settings’ deselect ‘Free Queue’ (this ensures you’re using your Featherless connection). Then simply choose your model from the Connection dropdown. Switching between models is as simple as creating an extra connection. You’ll know everything is working when your character responds using your chosen model - typically within a few seconds.
Venus AI
VenusChat supports OpenAI-compatible APIs out of the box, making our connection process quick and straightforward.
To connect your VenusChat with Featherless, head over to any character chat:
Click on the Gear icon on the top right
Go to “AI Model Settings”
Choose ‘Open AI’ under select an AI Model
Select “Reverse Proxy”
Pick a model from our model catalog and copy the complete url
Paste the url under ‘Open API Reverse Proxy`
(e.g. https://featherless.ai/models/mistralai/Mistral-Nemo-Instruct-2407)Enter your Featherless API Key under ‘Reverse Proxy Key’
JanitorAI
Head over to chat on any character on JanitorAI and let’s get you set up with just a few steps:
Open the dropdown in the topright corner and select ‘API Settings’
Go to ‘Proxy’
Pick a model from our model catalog
Under ‘Model’ choose custom and enter the models name
(e.g. meta-llama/Meta-Llama-3.1-8B-Instruct)Other API/proxy URL: https://api.featherless.ai/v1/chat/completions
API Key: Your Featherless API key
Check API Key/Model to see everything is working
Scroll down and Save Settings
Once you’ve saved your settings, congratulations - you can now chat with your character with any of our compatible models and switching between them is as simple as repeating steps 3-4. Feel free to experiment with different models to find the perfect fit for each character.
anime.gf
Bringing your favorite anime.gf characters to life with Featherless is straightforward. Enhance your interactions by making use of our diverse model catalog. Let’s get you setup with just a few simple steps:
Click on the cog in the top right of your screen
Head on over to ‘A.I. Settings’ and click on ‘provider’
Under API Provider select ‘Proxy’
API Key: Your Featherless API key
Base URL: https://api.featherless.ai/v1/
Model: Choose a model from our model catalog (e.g. anthracite-org/magnum-v4-72b)
Save
Great! Your anime.gf character is now powered by Featherless. Try sending a message to see the integration in action. Feel free to experiment with different models from our catalog to find the perfect match for each character - switching models is as easy as changing the model in your settings.
What’s next?
With your favorite roleplaying client connected to Featherless you’re ready to experiment with a variety of models from our catalog, join our Discord to share your experiences and get model recommendations!
Frontend chat client integration
I’ve always find something satisfying about a clean chat interface - it’s like having a dedicated thinking space where you can have a focused conversation. Whether I’m exploring new concepts, in a deep coding session or just brainstorming ideas - platforms like Typing Mind and our own Phoenix have become essential companions. Let me show you how to set up these tools that give you access to our entire model catalog as I think you’ll find them as invaluable as I do.
Featherless Phoenix
If you’re looking for the most straightforward way to start chatting with our models, look no further than Featherless Phoenix. As our native chat interface, it requires zero additional setup - simply login with your Featherless account, choose your preferred model from the menu in the top left corner and start chatting. It’s the perfect starting point for exploring our model catalog and finding the right language model for your needs.
Typing Mind
Typing mind, as a Chat UI frontend allows you to use AI models from our whole catalog. Featherless integration is as easy as going to ‘Models’, then clicking on ‘+ Add Custom Model’ on the top right of your screen followed by these quick steps:
1. Choosing ‘OpenAI Compatible API’ as API Type
2. Endpoint: https://api.featherless.ai/v1/chat/completions
3. Model ID: Pick a model from our model catalog (e.g. mistralai/Mistral-Nemo-Instruct-2407)
4. Choosing a context length (which you can find on the model’s page)
5. Add a custom header with it’s key ‘authorization’ and the key will be `Bearer <YOUR_FEATHERLESS_API_KEY>`
6. Add a custom body params with a number param `max_tokens' followed by any amount up to your context length. This will be the length of your response.
7. Lastly press Test and if everything went well you can now 'Add Model'
That’s it - Your Typing Mind interface is now connected to Featherless! Try chatting to verify if everything is working properly. Response times will vary by model, but you should typically see results within a few seconds.
Security best practices
Your Featherless API Key is your secure gateway to our services and all the models you love - protecting it is crucial for your account’s security and data privacy. Never share your keys publicly and we recommend creating separate API keys for different applications. If you suspect your key might have been exposed, rotate it immediately through your Featherless dashboard.
Community & Support
Our growing community of developers, enthusiasts, and AI practitioners is here to help you get the most out of Featherless:
Join our Discord community to connect with other users
Share your experiences with us!
Get model recommendations for your specific use case
- Stay updated on the latest models that get added
As the world of AI is evolving rapidly, Featherless is committed to evolving with it. As new models emerge and capabilities expand, we’re working to ensure you have seamless access and integration to all the latest advancements. Our mission to make all AI models available for serverless inference remains unchanged.
We’re excited to see what you all create with Featherless. Whether you’re creating engaging characters for roleplay or exploring new applications we haven’t even imagined yet. Ready to get started? Register for an account, or join our Discord community to connect with other enthusiasts.