This application provides a user interface to interact with various large language models, leveraging the @huggingface/inference library. It allows you to easily test and compare models hosted on Hugging Face, connect to different third-party Inference Providers, and even configure your own custom OpenAI-compatible endpoints.
Local Setup
TL;DR: After cloning, run pnpm i && pnpm run dev --open
Prerequisites
Before you begin, ensure you have the following installed:
Node.js: Version 20 or later is recommended.
pnpm: Install it globally via npm install -g pnpm.
Hugging Face Account & Token: You'll need a free Hugging Face account and an access token to interact with models. Generate a token with at least read permissions from [hf.co/settings/tokens](https:/
This application provides a user interface to interact with various large language models, leveraging the @huggingface/inference library. It allows you to easily test and compare models hosted on Hugging Face, connect to different third-party Inference Providers, and even configure your own custom OpenAI-compatible endpoints.
Local Setup
TL;DR: After cloning, run pnpm i && pnpm run dev --open
Prerequisites
Before you begin, ensure you have the following installed:
Node.js: Version 20 or later is recommended.
pnpm: Install it globally via npm install -g pnpm.
Hugging Face Account & Token: You'll need a free Hugging Face account and an access token to interact with models. Generate a token with at least read permissions from hf.co/settings/tokens.
Follow these steps to get the Inference Playground running on your local machine:
Clone the Repository:
bash
git clone https://github.com/huggingface/inference-playground.git
cd inference-playground
Install Dependencies:
bash
pnpm install
Start the Development Server:
bash
pnpm run dev
Access the Playground:
- Open your web browser and navigate to http://localhost:5173 (or the port indicated in your terminal).
Features
Model Interaction: Chat with a wide range of models available through Hugging Face Inference.
Provider Support: Connect to various third-party inference providers (like Together, Fireworks, Replicate, etc.).
Custom Endpoints: Add and use your own OpenAI-compatible API endpoints.
Comparison View: Run prompts against two different models or configurations side-by-side.
Configuration: Adjust generation parameters like temperature, max tokens, and top-p.
Session Management: Save and load your conversation setups using Projects and Checkpoints.
Code Snippets: Generate code snippets for various languages to replicate your inference calls.
Organization Billing: Specify an organization to bill usage to for Team and Enterprise accounts.
Organization Billing
For Team and Enterprise Hugging Face Hub organizations, you can centralize billing for all users by specifying an organization to bill usage to. This feature allows:
Centralized Billing: All inference requests can be billed to your organization instead of individual user accounts
Usage Tracking: Track inference usage across your organization from the organization's billing page
Spending Controls: Organization administrators can set spending limits and manage provider access
How to Use Organization Billing
In the UI: Navigate to the settings panel and enter your organization name in the "Billing Organization" field
In Code Snippets: Generated code examples will automatically include the billing organization parameter
API Integration: The playground will include the X-HF-Bill-To header in API requests when an organization is specified
Requirements
You must be a member of a Team or Enterprise Hugging Face Hub organization
The organization must have billing enabled
You need appropriate permissions to bill usage to the organization
