Hugging Face Inference API: How It Works, What It Costs, and When to Use It

The Hugging Face Inference API gives developers REST access to hundreds of open-weights models — LLMs, embedding models, image generators, speech, and classifiers — without provisioning any infrastructure. You authenticate with one HF token, send requests to Hugging Face’s routing layer, and it dispatches to whichever underlying provider has the model warm and ready. As of 2025, this service is officially called Inference Providers, but the token flow, base URL, and core usage pattern remain the same as the original Inference API.

What Is the Hugging Face Inference API (and What Changed)

Originally, the “Inference API” referred to a Hugging Face-hosted serverless service at api-inference.huggingface.co that loaded models on demand. That service still exists as the hf-inference provider, but as of July 2025 it focuses on CPU-class inference: embeddings, text classification, NER, summarization, and smaller historically significant models like BERT or GPT-2.

For GPU-accelerated inference — large LLMs, image generation, speech — Hugging Face now routes through partner providers: Groq, Together AI, Fireworks, Cerebras, DeepInfra, Replicate, Fal AI, and others. The interface is unchanged: one token, one base URL, OpenAI-compatible request format. The router URL is https://router.huggingface.co/v1. The old api-inference.huggingface.co URL still handles legacy calls to hf-inference, but new integrations should target the router.

How the Router Works

When you send a request to router.huggingface.co, Hugging Face selects a provider based on a policy you append to the model ID:

Append the policy directly to the model ID string: "deepseek-ai/DeepSeek-R1:cheapest". Omitting a suffix defaults to :fastest. Automatic failover is included — if the selected provider is flagged as unavailable, the router reroutes to an alternative.

Authentication and Your First Request

Go to huggingface.co/settings/tokens, create a Fine-grained token, and enable the Make calls to Inference Providers permission. Set it as HF_TOKEN in your environment.

Python — huggingface_hub

pip install huggingface_hub

import os
from huggingface_hub import InferenceClient

client = InferenceClient()  # reads HF_TOKEN from environment

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3-0324",
    messages=[{"role": "user", "content": "Explain tokenization in two sentences."}],
)
print(completion.choices[0].message.content)

Python — OpenAI drop-in

from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.huggingface.co/v1",
    api_key=os.environ["HF_TOKEN"],
)

completion = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-V3-0324:fastest",
    messages=[{"role": "user", "content": "Explain tokenization in two sentences."}],
)
print(completion.choices[0].message.content)

cURL

curl https://router.huggingface.co/v1/chat/completions 
  -H "Authorization: Bearer $HF_TOKEN" 
  -H "Content-Type: application/json" 
  -d '{
    "model": "deepseek-ai/DeepSeek-V3-0324:fastest",
    "messages": [{"role": "user", "content": "Explain tokenization in two sentences."}]
  }'

The OpenAI-compatible endpoint covers chat completions only. For other tasks — text-to-image, embeddings, speech-to-text — use the huggingface_hub Python library or @huggingface/inference JS SDK, which handle provider-specific request formatting automatically.

Free Tier, Credits, and What Happens Next

Every Hugging Face account gets a monthly credit allowance that applies automatically to routed requests:

Once your credits run out, access does not stop — you purchase additional credits to continue. Hugging Face charges you the same rate the provider charges, with no additional fee. What a given request costs depends on the model and provider; you can track per-model, per-provider spend at huggingface.co/settings/inference-providers/overview.

If you already have accounts with a specific provider, you can set a custom provider key in HF settings. Requests still route through HF, but the provider bills you directly and your HF monthly credits do not apply. Before committing to a volume, use the API cost calculator to project monthly spend by model and call volume.

Cold Starts and the hf-inference Provider

The legacy hf-inference provider loads models on demand. When a model has not been called recently and has gone idle, the first request triggers a model load before the response can begin — a cold start. This adds noticeable latency to that first call.

As of July 2025, hf-inference focuses on CPU inference: embedding models, classifiers, NER, smaller text models. Cold starts are most relevant in this context.

For GPU-accelerated workloads — large LLMs, image generation — the router dispatches to third-party providers like Groq or Together AI that run warm, shared GPU capacity. Cold starts are not a concern for these providers in the same way, though you share capacity and have no throughput guarantee during traffic spikes.

If cold-start latency or throughput variability is unacceptable — say, for a latency-sensitive production endpoint — a dedicated Inference Endpoint is the solution.

Inference API vs Inference Endpoints

The Hugging Face platform has two distinct products for inference. They share the same token system and model Hub, but operate very differently:

Inference Endpoints are billed per minute only while in running or initializing state — paused endpoints cost nothing. AWS GPU options range from a T4 at $0.50/hr (14 GB VRAM) up to an H200 at $5.00/hr per card (141 GB VRAM). GCP options include H100 at $10.00/hr per card (80 GB VRAM). Before selecting a tier, use the VRAM calculator to verify your model fits the target GPU.

How Pricing Compares with Other Providers

Because Hugging Face routes to the same underlying providers — Together AI, Fireworks, DeepInfra, Groq — that you can also reach directly or through OpenRouter, the per-token rates for a given model are generally identical. HF adds no markup.

The practical differences:

• Free credits: HF gives $0.10–$2.00/month in free usage automatically. OpenRouter and direct providers have no equivalent monthly allowance.
• Model breadth: HF Inference Providers focuses on open-weights models from the Hub. OpenRouter also covers closed-source models (GPT-4o, Claude, Gemini). If you need both open and proprietary models in one router, OpenRouter covers more ground.
• Non-chat tasks: Embeddings, image generation, and speech are available through HF’s SDK. Most competing routers are chat-completion only.
• Billing consolidation: One HF account covers all providers, with a single usage dashboard. Direct provider accounts require separate billing relationships for each.

For a full model comparison by price and capability, see the AI models database covering specs and pricing across major models.

When Dedicated Endpoints or Self-Hosting Wins

Use Inference Providers when you are prototyping, have variable or unpredictable load, and need access to a broad public model catalog without managing servers.

Switch to a dedicated Inference Endpoint when:

• You need to serve a fine-tuned or private model not available on the public Hub.
• Latency consistency is a requirement — shared providers can have variable response times under load.
• You need guaranteed throughput for a production SLA.

Consider self-hosting when your call volume is high enough that per-token costs exceed the amortized cost of owning hardware, or when data-privacy requirements prohibit sending inputs to third-party APIs. Theself-hosting vs API break-even calculator gives a concrete cost comparison based on your request volume and model size. For hardware recommendations if you go that route, see the best GPUs for running LLMs locally.

Frequently Asked Questions

What is the difference between the Inference API and Inference Endpoints?

The Inference API (now Inference Providers) is a shared, pay-per-request service that routes to partner GPU providers and HF’s own CPU infrastructure. Inference Endpoints are dedicated GPU instances you provision in a cloud region; they run a single model continuously and are billed per minute of uptime. Use the API for prototyping and variable workloads; use Endpoints for production latency guarantees and private or fine-tuned models.

Do I need a Pro subscription to use the Inference API?

No. A free account gets $0.10/month in credits, which is sufficient for light experimentation. PRO subscribers receive $2.00/month. Both tiers support pay-as-you-go credit purchases once the monthly allowance is exhausted. The PRO plan’s main inference advantage is the higher monthly credit amount, not gated access to models or providers.

Why is my first request much slower than subsequent ones?

If you are routing to the hf-inference provider, it loads models on demand. A model that has gone idle must be loaded into memory before your request can complete, adding latency to that first call. This does not apply to GPU providers like Groq or Together AI, which run warm shared infrastructure. Specifying :fastest or a named GPU provider in the model ID will avoid this delay entirely.

Is the HF Inference API compatible with the OpenAI Python SDK?

Yes, for chat completions. Set base_url="https://router.huggingface.co/v1" and pass your HF token as api_key. The /v1/chat/completions and /v1/models endpoints are OpenAI-compatible. For other task types — embeddings, image generation, speech — you need the huggingface_hub Python library or @huggingface/inference JS SDK; those are not covered by the OpenAI-compatible endpoint.

Can I serve a fine-tuned model through the Inference API?

Not through Inference Providers — it only serves models available in the public Hub catalog and supported by a partner provider. For a private or fine-tuned model, deploy a dedicated Inference Endpoint, which supports private Hub repos and lets you bring your own model weights to a GPU instance you control.

How do I track and control costs?

Your usage breakdown by model and provider is at huggingface.co/settings/inference-providers/overview. Team and Enterprise admins can set spending limits and disable specific providers from the organization settings page. For forward-looking cost estimates before you start building, use the API cost calculator.