Skip to main content
This guide moves where your models run, not where they live. Your models, datasets, and access tokens stay on the Hugging Face Hub, and Runpod pulls weights from the Hub by repository ID, the same way Inference Endpoints does today. It’s written for teams that run production inference on Hugging Face compute. Inference Endpoints map to Runpod Serverless endpoints, Inference Providers calls map to Public Endpoints where the model is listed, GPU Spaces map to Pods, and Jobs map to @app.task in the Runpod SDK or to a Pod. At the end, each workload runs on Runpod, your clients call Runpod URLs with a Runpod API key, and the Hugging Face compute is paused or deleted.
You can hand this migration to a coding agent. See Migrate with a coding agent.

How Hugging Face maps to Runpod

What’s different

What works differently

Where the model lands inside the container. Inference Endpoints mounts the selected repository at /repository. Runpod’s cached models land in the standard Hugging Face cache layout at /runpod-volume/huggingface-cache/hub/models--ORG--NAME/snapshots/HASH/. Code that loads from /repository needs the snapshot path instead. Step 4 has a helper that resolves it. What happens to a request when nothing is running. An Inference Endpoint that has scaled to zero answers with status 503 while the new replica initializes, unless the client sends the X-Scale-Up-Timeout header to make the proxy hold the request for a set number of seconds. A Runpod queue-based endpoint puts the request in the endpoint’s queue and a worker picks it up when it’s ready, so you can remove 503 retry loops. Load balancing endpoints don’t queue: a request that arrives before a worker is ready can return no workers available, and it times out after 2 minutes without a worker. The first synchronous request after a cold start. /runsync waits up to 90 seconds by default, and synchronous routes sit behind an edge timeout of about 100 seconds. A cold vLLM worker loading a large model can take longer than that. Use /run and poll /status for the first request, or keep one active worker for latency-sensitive endpoints. Idle time is measured in seconds. Scale-to-zero on Inference Endpoints is set in minutes (1 hour by default). Runpod’s idle timeout is set in seconds, from 1 to 3600. The console default is 5 seconds and the REST v2 default is 10 seconds. A worker is billed while it waits out the idle timeout. Authentication. Inference Endpoints has three endpoint types: public (no authentication), authenticated (a Hugging Face token), and private. Requests to a Runpod Serverless endpoint carry Authorization: Bearer RUNPOD_API_KEY. If you run a public endpoint today, put your own API or gateway in front of the Runpod endpoint and keep the key server-side. Request and response envelope. The Inference Toolkit accepts your JSON body as is, with inputs at the top level. A queue-based endpoint expects {"input": {...}} and returns your handler’s result under output, next to id, status, delayTime, and executionTime. The model field on OpenAI-compatible calls. On a vLLM worker, model must be the Hugging Face repo ID you deployed, or the name you set in OPENAI_SERVED_MODEL_NAME_OVERRIDE. If your clients send the Inference Endpoint’s name as model, set that override to the same string and they won’t need a code change beyond the URL and key. TGI’s native routes. TGI serves /generate and /generate_stream next to its OpenAI-compatible routes. The vLLM worker serves /chat/completions, /completions, and /models. Move /generate callers to /completions. Hugging Face put TGI in maintenance mode in late 2025 and recommends vLLM or SGLang, so this is the same move you’d make on Inference Endpoints. Autoscaling signals. Inference Endpoints scales on hardware utilization, pending requests, or (in beta, for vLLM and SGLang) inference load. Runpod scales on queue delay (seconds a request waits) or request count (queued plus in-progress requests divided by a target). Request count is the closer match to pending requests. Pause and resume. A paused Inference Endpoint stays defined and isn’t billed. On Runpod, an endpoint at zero workers isn’t billed for compute, so there’s no separate paused state. To block new workers from starting, set max workers to 0. Unused endpoints scale down on their own. After 3 days without a request, Runpod reduces an endpoint’s max workers to 2 and emails you. After 7 days it sets max workers to 0, and it stays there until you raise it. An Inference Endpoint with min replicas above 0 keeps running until you change it. Worker limits. Your account balance sets the maximum number of workers across all endpoints (5 by default). See Max worker limits before you size a large endpoint.

What has no direct equivalent

In the other direction, moving gives you a few things to build on: a request queue with async jobs, status polling, and webhooks on queue-based endpoints, persistent network volumes shared across workers, and per-second billing.

Requirements

  • A Hugging Face account with access to the endpoints, Spaces, and Jobs you’re moving, and the hf CLI logged in (hf auth login).
  • A Hugging Face access token with read access to every gated or private repo you serve.
  • A Runpod account with credits and a Runpod API key.
  • Docker, if you’re moving a custom handler or a custom container.
  • Python 3.10 to 3.14, if you use the Runpod SDK.

Step 1: Take inventory of what you run on Hugging Face

List every Inference Endpoint in every namespace you belong to, then print the full configuration of each one:
To export everything at once, dump the raw API response for each endpoint. It includes the repository and revision, engine image, environment variable names, instance type and size, vendor and region, replica limits, and the scale-to-zero timeout.
export_endpoints.py
Secret values aren’t returned, so collect them from wherever you keep them. For each endpoint, record which engine it runs (the key under model.image in raw, such as vllm, tgi, tei, llamacpp, huggingface, or custom) and whether the model repository contains a handler.py. Those two facts decide where the endpoint lands in Step 3. Then list the rest of your Hugging Face compute:
Any Space whose hardware isn’t cpu-basic is using paid compute. Note each Space’s SDK (Gradio, Docker, or static), its app_port, and its variables and secrets. Finally, find the code that calls Hugging Face compute or manages it:
Code that only reads from the Hub (from_pretrained, hf_hub_download, snapshot_download, load_dataset) doesn’t need to change.

Step 2: Set up Runpod

Install runpodctl and the Runpod SDK, which includes the rp CLI, then store your API key:
runpodctl doctor walks through first-time setup, and rp login saves credentials to ~/.runpod/config.toml. To let a coding agent manage resources, add the MCP server. If you manage infrastructure as code, the Terraform provider covers endpoints, templates, and volumes. Next, store your Hugging Face token as a Runpod secret so it never appears in an endpoint’s configuration:
The command prompts for the value. You can also create the secret on the Secrets page in the console. Secret names can’t start with runpod. Reference the secret from any environment variable as {{ RUNPOD_SECRET_hf_token }}.

Step 3: Choose where each workload lands

Queue-based or load balancing is the main decision for an Inference Endpoint. Choose queue-based when requests can wait for a worker, run longer than a few minutes, or need retries. Choose load balancing when you need your own routes, non-JSON payloads, or WebSockets, and your requests finish within the 5.5 minute processing limit and the 30 MB payload limit.

GPU mapping

Match on VRAM and GPU generation, then confirm the model fits. Instance names come from the Inference Endpoints pricing page, and pools and GPU IDs come from the GPU types reference. The instance size (x1, x2, x4, x8) is the number of GPUs per replica. Set the same number with --gpu-count, or gpu.count in REST v2. You can list up to three GPU choices in priority order on one endpoint, which helps when your first choice is busy. Stock varies by data center, so check live availability with runpodctl gpu list or GET /v2/catalog/gpus?include=AVAILABILITY&product=SERVERLESS before you commit to a GPU.

Step 4: Move your images and code

LLM endpoints (vLLM, TGI, SGLang)

There’s no image to build. The vLLM worker takes the same repo ID, and engine settings become environment variables. Any vllm serve flag works as an upper snake case variable (--max-model-len 8192 becomes MAX_MODEL_LEN=8192), and VLLM_EXTRA_ARGS passes flags through as written. TGI’s batching limits (Max Input Tokens, Max Batch Prefill Tokens, Max Batch Total Tokens) have no one-to-one vLLM setting. Start with MAX_MODEL_LEN and MAX_NUM_SEQS, then tune from there. See vLLM environment variables and Configure vLLM for model-specific settings.

Custom handlers (handler.py)

On Inference Endpoints, the Inference Toolkit imports EndpointHandler from a handler.py in your model repository, calls __init__ once with the path to the weights, and calls __call__ with the request body for every request. Here’s a typical handler:
handler.py (Hugging Face)
On Runpod, the worker is a container that you build. Load the model at module level so it happens once per worker, and read the request from job["input"]:
handler.py (Runpod)
Three things changed. path is now the snapshot directory of the cached model, which Runpod places at /runpod-volume/huggingface-cache/hub/ when you attach a model to the endpoint (see Use Hugging Face models). The handler keeps the inputs and parameters keys, so clients only wrap their existing body in input. Errors are returned as {"error": "..."}, which marks the job as FAILED.
To change as little code as possible, copy your existing handler.py into the image unchanged as hf_handler.py and wrap it: create EndpointHandler(path=resolve_snapshot_path(MODEL_ID)) at module level, then return endpoint_handler(job["input"]) from handler(job).
Package the handler with a Dockerfile. Copy the dependencies from the requirements.txt in your model repository and add runpod:
Dockerfile
requirements.txt
Build for linux/amd64 and push to a registry Runpod can pull from:
The base image uses CUDA 12.8, so set a matching CUDA floor when you create the endpoint in Step 6. The weights aren’t in the image: Runpod stages them on the host. For a private registry, save credentials with runpodctl registry create and pass the ID with --registry-auth-id on the template. You can also skip the registry and deploy from GitHub.

Custom handlers without a Dockerfile

The Runpod SDK builds and deploys the worker from a Python file. runpod.Model stages the weights from the Hub before the worker starts, and model.path points to them:
main.py
Run it against temporary endpoints with rp flash dev main.py, then deploy with rp flash deploy. For a gated model, add env={"HF_TOKEN": runpod.Secret("hf_token")} to the decorator. This example builds the pipeline inside the function, the same way the SDK docs do. To load the model once per worker, serve it from an @app.api class and load it in a method marked @runpod.init. See Build an app and Resource parameters.

Custom containers, TEI, and llama.cpp

A container that serves HTTP on Inference Endpoints runs on a load balancing endpoint with three changes: If you don’t want to rebuild the image, set HEALTH_CHECK_PATH to your existing health route. Changing PORT doesn’t change PORT_HEALTH, so set both, and expose the port on the endpoint. If the server loads the model from /repository, make that path configurable and point it at the cached snapshot directory or a Hub repo ID. For TEI, run the image from the TEI docs (for example ghcr.io/huggingface/text-embeddings-inference:cuda-1.9). TEI reads MODEL_ID, HF_TOKEN, and PORT from the environment and serves /health, /embed, /rerank, /predict, and /v1/embeddings. Runpod’s docs don’t cover a dedicated embeddings worker, but the Hub has an embedding category you can search:
Read a listing’s required environment variables with runpodctl hub get before you deploy it.

Spaces and Jobs

A Docker Space is already a container image. Build it for linux/amd64, push it, and run it on a Pod with the Space’s app_port exposed (the default is 7860). For a Gradio or Streamlit Space, add a Dockerfile that installs requirements.txt and starts the app bound to 0.0.0.0. Space variables and secrets become Pod environment variables and Runpod secrets. A Job is an image, a command, and a hardware flavor. With the Runpod SDK, the same work is a function:
Each call to train provisions one Pod and terminates it when the function returns. Scheduled Jobs (hf jobs scheduled run) have no built-in counterpart: trigger the task from your own scheduler or CI.

Step 5: Move your data and models

In most migrations there’s nothing to copy, because the weights are on the Hub. Pick how each endpoint gets them: Cached models are the closest match to how Inference Endpoints delivers weights. Runpod schedules workers on hosts that already hold the model where it can, and you aren’t billed while a model downloads. An endpoint supports one cached model, and a repo with several quantization variants downloads all of them. The reference includes a ref (:main, a tag, or a commit), so pin the commit you serve today if your Inference Endpoint pins a revision. To pre-load a network volume, create it in the data center you’ll serve from, start a small Pod with the volume attached, and download from the Hub:
A network volume pins every endpoint and Pod that mounts it to the volume’s data center, so choose one that has the GPUs you need. On Serverless the same files appear under /runpod-volume/models/. Delete the loader Pod when the download finishes. Fine-tuned weights that only exist on a replica’s disk or in a Space’s ephemeral storage should be pushed to a Hub repository first (hf upload), then served from there. Datasets on the Hub load with load_dataset from Runpod as they do today.

Step 6: Deploy on Runpod

vLLM worker for an LLM endpoint

Here’s an Inference Endpoint definition and its Runpod equivalent:
--hub-id pulls the image, container disk size, and default environment from the Hub release, and --env overrides or adds variables. --model-reference attaches the cached model, so workers start with the weights already on the host where possible. The command prints the endpoint ID that goes in every request URL. In the console, the same deployment is Serverless > New Endpoint > Hugging Face, where you enter the repo ID and token. Replicas translate directly: min_replica is --workers-min (active workers, billed continuously) and max_replica is --workers-max. scale_to_zero_timeout=15 is in minutes, and --idle-timeout is in seconds with a maximum of 3600. A short idle timeout costs less and causes more cold starts.

Queue-based endpoint for a custom handler

Create a Serverless template for the image you pushed in Step 4, then create the endpoint from it:
With --template-id, environment variables come from the template, so set them on template create. Each Serverless template binds to one endpoint. In REST v2, type and scaling are required, timeout is in milliseconds, and FlashBoot is off unless you set it. The REST example creates the endpoint without a cached model, and the handler above loads from the cache, so its workers fail at startup with Cached model not found until you attach the model in the Model field in the console. To attach it at creation time, use the runpodctl tab.

Load balancing endpoint for a custom container or TEI

Create load balancing endpoints through REST v2 or the console (Serverless > New Endpoint, endpoint type Load Balancer). This example runs TEI:
Load balancing endpoints scale on request count. The endpoint is served at https://ENDPOINT_ID.api.runpod.ai/, and every path your server defines is available under it. For your own container, swap in your image and port, and set HEALTH_CHECK_PATH to the health route you configured on Inference Endpoints. This TEI configuration downloads the model when a worker starts, so expect a longer cold start than with a cached model.

Public Endpoint for an Inference Providers call

There’s nothing to deploy. Check the model reference for the model you call through router.huggingface.co. If it’s listed, switch the client in Step 7. If it isn’t, deploy it with the vLLM worker above.

Pod for a Space

Run the Space’s image on a Pod and expose its port over HTTP:
The app is served at https://POD_ID-7860.proxy.runpod.net.
The Pod proxy URL is public and unauthenticated. If your Space relied on Hugging Face’s access controls, add authentication in the app or put it behind your own proxy before you expose it. Proxy connections also time out after 100 seconds, so a request that takes longer needs a TCP port or a queue-based endpoint behind the app.
A Pod bills per second while it runs, whether or not anyone is using the app, like an upgraded Space with no sleep time. If you relied on a Space’s sleep time, stop the Pod yourself with runpodctl pod stop POD_ID and start it again with runpodctl pod start POD_ID.

Step 7: Update the code that calls Hugging Face

OpenAI-compatible clients

Swap the base URL and the key. Everything else in the call stays the same.
model is the repo ID you deployed, or the value of OPENAI_SERVED_MODEL_NAME_OVERRIDE. Streaming works with stream=True. Raise the client’s timeout for the first request after a cold start, and drop the X-Scale-Up-Timeout header.

InferenceClient

InferenceClient accepts a base_url and exposes the same chat.completions.create method, so you can keep it and point it at Runpod:
Task methods that call TGI’s native routes, such as client.text_generation() or endpoint.client.text_generation(), have no route on the vLLM worker. Replace them with chat.completions.create or the OpenAI client’s completions.create.

TGI /generate callers

Move raw /generate requests to the completions route and rename the parameters:
The response is an OpenAI completion object, so read choices[0].text. /generate_stream callers move to the same route with "stream": true.

Custom handler clients

Wrap the body in input, change the URL and key, and read the result from output:
/runsync returns the result when the job finishes within the wait window. If the job is still queued or running when the window closes, the response has a status of IN_QUEUE or IN_PROGRESS, and you fetch the result from /status/JOB_ID. For anything longer than a few seconds, submit to /run, which returns a job ID right away, then poll /status/JOB_ID or pass a webhook URL in the request. Results are kept for 30 minutes after an async job and 1 minute after a sync job. Payloads are limited to 10 MB on /run and 20 MB on /runsync, so send large inputs as URLs. The same calls are available from Python through runpod.Endpoint("ENDPOINT_ID").run_sync({...}). See Send API requests.

Custom container and TEI clients

Only the host and the key change. Paths and payloads stay the same:
Add retries for the cold start window. A request that arrives before a worker is healthy can return no workers available. Use at least 3 retries with a 5 to 10 second delay.

Inference Providers clients

For a model that’s listed as a Public Endpoint with OpenAI compatibility, such as Qwen3 32B, swap the router URL for the model’s endpoint:
Each Public Endpoint has its own URL, so there’s no single router and no provider suffix (:fastest, :cheapest, :groq) on the model name. The Public Endpoint may serve a different variant of the model than the provider you used (here, an AWQ quantization), so compare outputs before you switch. Image, video, and audio models use /runsync with an input object. See Make API requests.

Control plane automation

After any update that sets a worker count to 0, confirm it with runpodctl serverless get ENDPOINT_ID, and update runpodctl first if it’s an old build. A batch script that resumes an endpoint, waits, runs inference, and pauses it again gets simpler: submit the jobs to /run on an endpoint with zero active workers. Workers start for the queue and stop when it’s empty.

Step 8: Verify, cut over, and shut down

  1. Send a real request from outside Runpod. Check the endpoint’s health, then send the same prompt or payload you send to Hugging Face:
  2. Compare outputs and latency. Run a fixed set of requests against both platforms with the same sampling settings (set temperature to 0 where you can). Expect small differences when the engine changes, for example TGI to vLLM, or when the quantization differs. Measure a cold start as well as warm latency: the response includes delayTime and executionTime, and the endpoint’s Metrics tab shows cold start percentiles.
  3. Run both in parallel. Keep the Inference Endpoint up and send a share of traffic to Runpod through a feature flag or your gateway.
  4. Shift the rest of the traffic, then watch error rates and queue delay for a full traffic cycle.
  5. Stop billing on Hugging Face. Pause first so you can roll back, then delete once you’re sure:
    Deleting an Inference Endpoint is permanent and removes its configuration, logs, and usage metrics. Keep the model repositories and the token that Runpod uses to read them. Revoke tokens that only the old clients used.
  6. Set cost guards on Runpod. Keep --workers-min at 0 unless you need warm capacity, set --workers-max to the concurrency you’re willing to pay for, set an execution timeout on every endpoint, and keep the idle timeout short. Delete loader Pods and unused network volumes, which bill for storage until you remove them. Track spend on the billing page.

Reference

CLI command map

API map

The Inference Endpoints API is at https://api.endpoints.huggingface.cloud. Runpod’s REST API v2 is at https://api.runpod.io. Both use Authorization: Bearer.

Configuration field map

Environment variable map

Status map

Inference Endpoints reports one state per endpoint. Runpod reports a state per worker, and a state per job on queue-based endpoints. Job states on queue-based endpoints are IN_QUEUE, IN_PROGRESS, COMPLETED, FAILED, CANCELLED, and TIMED_OUT.

Migrate with a coding agent

Install the Runpod skills and connect the MCP server:
The MCP server signs in with OAuth (Sign in with Runpod), so you don’t need an API key to start. See Agent skills and MCP servers. Then give your agent this prompt from the root of your repository:

Troubleshooting

The worker fails with Cached model not found or tries to load /repository. The handler still expects Hugging Face’s mount path, or the endpoint has no cached model attached. Attach the model with --model-reference or the Model field, and load from the snapshot directory under /runpod-volume/huggingface-cache/hub/. A gated model returns 401 or 403 while a worker starts. The endpoint has no HF_TOKEN, or the token’s account hasn’t accepted the model’s terms. Set HF_TOKEN to {{ RUNPOD_SECRET_hf_token }} and confirm the token can read the repo. Workers pick up a changed secret on their next start. The first request after idle returns a 524 or times out. The cold start took longer than the synchronous wait. Send the first request to /run and poll /status, raise the idle timeout, or set one active worker. OpenAI calls return an invalid model error. The client still sends the Inference Endpoint’s name as model. Send the Hugging Face repo ID, or set OPENAI_SERVED_MODEL_NAME_OVERRIDE to the name your clients use. /generate returns 404 or an error. TGI’s native routes aren’t served by the vLLM worker. Call /openai/v1/completions or /openai/v1/chat/completions. A load balancing worker never becomes healthy. The load balancer polls /ping on port 80 by default. Set HEALTH_CHECK_PATH to your health route, set PORT and PORT_HEALTH to your server’s port, and expose that port on the endpoint. With misconfigured ports, workers stay up for 8 minutes and return 502. Requests return no workers available. A load balancing endpoint has no healthy worker yet. Retry with a delay, or keep an active worker. If the request can wait, a queue-based endpoint holds it for you. The vLLM worker runs out of GPU memory on a model that fit before. vLLM pre-allocates VRAM for its KV cache, and your new GPU may have less memory than the old instance. Lower MAX_MODEL_LEN or GPU_MEMORY_UTILIZATION, or move up a pool. Workers stay Throttled or Initializing for a long time. The selected GPU is busy in the allowed data centers, or the model is still downloading to the host. Add a second and third GPU choice, remove data center restrictions, and for large models raise RUNPOD_INIT_TIMEOUT. An endpoint that worked last week returns no workers. Runpod set max workers to 0 after 7 days without requests. Raise max workers again.

Next steps

vLLM workers

Configure and tune LLM endpoints.

Cached models

Serve Hugging Face models without baking them into images.

Endpoint settings

Set workers, timeouts, and autoscaling.

Runpod SDK

Define endpoints and tasks in Python.
Last modified on November 14, 2026