Skip to main content
This guide is for teams that run GPU workloads on Microsoft Azure and want to run some or all of them on Runpod. Most teams move the GPU-heavy part (training, inference, batch) and keep data, identity, and the rest of the application on Azure, so the guide covers that hybrid setup first and a full exit second. GPU virtual machines map to Pods. Azure Machine Learning online and batch endpoints and Azure Container Apps serverless GPUs map to Serverless endpoints. Open-weight models you call through Microsoft Foundry map to Public Endpoints or a Serverless vLLM endpoint. By the end, you’ll have one workload running on Runpod, verified with a real request, and a checklist for stopping the matching Azure charges.
You can hand most of this migration to a coding agent. See Migrate with a coding agent.

How Azure maps to Runpod

What’s different

What works differently

You choose a GPU, not a VM size. An Azure size such as Standard_NC24ads_A100_v4 fixes the GPU, vCPUs, memory, and disk together. On Runpod you pick a GPU type and count, and the Pod reports the vCPUs and memory it was allocated. Use the min_vcpu_count_per_gpu and min_ram_per_gpu placement filters when your workload needs a floor. Pods are containers. There’s no cloud-init, no kernel modules, no systemd, and no NVIDIA driver extension to install: the host provides the driver and your image provides CUDA libraries. Anything your cloud-init script did moves into the image or a start command. Docker Compose and UDP ports aren’t supported. Stopping means something different. A stopped Azure VM that isn’t deallocated still bills for compute. A stopped Pod doesn’t bill for compute, and its container disk is erased. Its volume disk is kept and still bills. See Pod pricing. Serverless scales on the queue, not on Azure Monitor rules. A managed online deployment runs a fixed instance_count unless you attach Azure Monitor autoscale rules. A Runpod endpoint has a minimum and maximum worker count and scales on queue delay or request count. The minimum defaults to zero, so an idle endpoint has no running workers. Requests are wrapped. A queue-based endpoint takes {"input": {...}} and returns {"id", "status", "output"}. Your scoring script’s run(raw_data) receives the raw body, so both the handler and every client change shape. A load balancing endpoint keeps your own request and response shapes. Limits differ. A managed online endpoint caps a request at 180 seconds. A queue-based endpoint’s execution timeout defaults to 10 minutes in the console and can be raised, with a 10 MB payload limit on /run and 20 MB on /runsync. A load balancing endpoint has a 5.5 minute processing timeout and a 30 MB payload limit. Rollouts replace workers instead of splitting traffic. Azure Machine Learning splits traffic by percentage across deployments and can mirror traffic. A Runpod endpoint update is a rolling release: idle workers are replaced immediately and busy workers after their current job. For a percentage split, run two endpoints and split in your client or gateway. Auth is one API key. Runpod endpoints accept Authorization: Bearer YOUR_API_KEY. There’s no equivalent of aad_token auth or per-endpoint keys. A Pod’s HTTP proxy URL is public and unauthenticated, so put your own auth in front of anything you expose on a Pod. Images must be linux/amd64. Build with docker build --platform linux/amd64, especially on Apple silicon or ARM build agents. Data transfer. Azure charges for outbound data transfer to the internet beyond a monthly free allowance, so budget for the one-time copy of datasets and model weights out of Azure. Runpod doesn’t charge for data ingress or egress.

What has no direct equivalent

On Runpod:
  • OpenAI’s proprietary GPT chat models. Runpod doesn’t host the GPT deployments you create in Azure OpenAI. The migration applies when you move to an open-weight model, either one listed in the Public Endpoint model reference or one you serve yourself with vLLM. If you need to keep a proprietary model, keep that call on Azure.
  • VNet, NSGs, and Private Link. Runpod has no customer-defined private network or private connectivity to Azure. Pods in your account can reach each other over global networking at POD_ID.runpod.internal, if you enable it when you deploy them. It’s available on NVIDIA GPU Pods in supported data centers. Traffic between Runpod and Azure goes over the public internet with TLS, so Azure resources behind private endpoints need a public endpoint with firewall rules, or a proxy you run.
  • Managed identities and Entra ID RBAC for workloads. A Runpod worker can’t assume an Azure identity. Store a least-privilege service principal secret or a SAS token as a Runpod secret and pass it as an environment variable.
  • Key Vault. Runpod secrets store values and inject them at boot. They don’t provide key operations, certificates, or versioning. Keep Key Vault for those and copy only the values workloads need.
  • Windows and VDI. NV and NG family workloads, Windows GPU VMs, and Data Science VM Windows images stay on Azure.
  • Managed Kubernetes. Runpod runs containers directly instead of offering a Kubernetes control plane. Run a scheduler on top of Runpod with SkyPilot or dstack, or move each workload to Pods or Serverless.
  • Model registry, pipelines, and experiment tracking. These Azure Machine Learning workspace features stay on Azure, or move to a tool you host. Runpod runs the compute.
  • Traffic mirroring and percentage splits across deployments of one endpoint.
  • Bicep and ARM templates. Use Terraform or REST API v2.
  • Spot capacity through the current API. REST API v2, runpodctl, and the Terraform provider create on-demand Pods. Plan Spot VM workloads as on-demand Pods or Serverless.
On Azure:
  • A job queue built into the inference endpoint. A queue-based Runpod endpoint includes the queue, status polling, streaming, retries, and webhooks. On Azure you’d typically pair an online endpoint with a separate queue service, or use a batch endpoint.
  • GPU access without per-family quota requests. Azure gates GPU capacity with per-region, per-VM-family quota. Runpod uses an account-level spend limit and per-data-center stock instead, so check availability before you plan around a GPU type.
  • One volume shared across data centers. A global volume mounts on Pods and workers in any data center.

Requirements

  • An Azure account with permission to read the resources you plan to move, and the Azure CLI with the ml and containerapp extensions if you use those services.
  • A Runpod account with credits and an API key.
  • Docker, to rebuild or retag images for linux/amd64.
  • Python 3.10 or later if you use the Runpod SDK.
  • Terraform 1.6 or later if you manage infrastructure as code.

Step 1: Take inventory of what you run on Azure

List every GPU resource, then export its configuration. Start with virtual machines. This command lists N-series VMs across the subscription with their size, power state, and location:
For each VM, record the size, image, OS and data disk sizes, open ports, and the cloud-init or custom script it boots with. az vm show -g RESOURCE_GROUP -n VM_NAME returns the full definition. VMs in the Stopped state without being deallocated are still billing, so note those too. Next, list Azure Machine Learning resources in each workspace:
For each online deployment, record instance_type, instance_count, the model and environment references, environment_variables, request_settings, and whether it uses a scoring script or a custom container with inference_config routes. Check the endpoint’s auth_mode and traffic with az ml online-endpoint show, and any Azure Monitor autoscale settings attached to the deployment. List Container Apps, then read each GPU app’s definition:
From the show output, record the workload profile name, image, CPU and memory, environment variables and secrets, ingress target port, minimum and maximum replicas, and scale rules. Then list the storage and images those workloads depend on:
Finally, find the code and configuration that calls Azure. Run this from the root of each repository:
Sort the matches into three groups: code that runs inside the workload (scoring scripts, training scripts), code that calls the workload (clients), and automation that manages it (CI, Terraform, scripts).

Step 2: Set up Runpod

Create an account at console.runpod.io, add credits, and create an API key. Then install the tools you’ll use.
Install the CLI and check that it can reach your account:
On macOS you can also run brew install runpod/runpodctl/runpodctl.
To let a coding agent manage Runpod for you, connect the Runpod MCP server.

Step 3: Choose where each workload lands

Pick a target for each workload based on how it’s invoked, not on which Azure service it runs on today. Use this table to translate current Azure GPU sizes to Runpod GPU type IDs. GPU counts and memory come from Microsoft’s size documentation, and the Runpod IDs come from the GPU types reference. Runpod’s list doesn’t include the T4 or A10, so those rows give the closest memory match. Choose by what the workload needs (GPU memory, then throughput), and check live stock before you commit to a type: run runpodctl gpu list, or call GET /v2/catalog/gpus. Multi-GPU counts and specific types aren’t available in every data center.

Step 4: Move your images and code

Use your Azure Container Registry images

You can keep images in ACR. Runpod pulls private images with a saved username and password, so create a credential that works without an interactive login. Microsoft documents two options for unattended pulls: a repository-scoped token and a service principal. The registry admin user also works, but Microsoft doesn’t recommend sharing it because it has full push and pull access.
Create a token that can only read the repository Runpod needs:
The output includes two generated passwords, shown once. The username is the token name. Save it on Runpod, reading the password from an environment variable so it stays out of your shell history:
runpodctl registry create returns a registry credential ID. Pass it as --registry-auth-id when you create a template or Pod, or as registry in REST API v2 and Terraform. Reference the image by its full name, such as myregistry.azurecr.io/inference/worker:v1.
Runpod pulls images over the public internet. If your registry only allows private endpoint access, enable public access with authentication, or push a copy of the image to a registry that’s reachable from outside your VNet.

Check the image

  • Architecture: Runpod runs linux/amd64 images only.
  • CUDA: Your image ships the CUDA runtime, and the host ships the driver. Set a CUDA floor when you create the Pod or endpoint (--min-cuda-version 12.8, or gpu.minCudaVersion in REST API v2) so the host driver supports your image.
  • Azure base images: Images built from Azure Machine Learning inference or curated environments include the Azure inference server, which expects the Azure Machine Learning runtime around it. Rebuild from a Runpod PyTorch image such as runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2404, or from an nvidia/cuda base, and install your dependencies from the environment’s conda or pip file.
  • Start command: A Pod runs the image’s CMD. Override it with --docker-args in runpodctl, args in Terraform, or cmd in REST API v2.

Translate a scoring script to a handler

An Azure Machine Learning scoring script has two functions. init() runs once when the container starts and loads the model from AZUREML_MODEL_DIR. run(raw_data) runs for every request. Here’s a typical GPU scoring script:
score.py
On Runpod, module-level code replaces init() and a handler function replaces run(). The handler receives a job whose input field is already parsed JSON:
handler.py
Three things changed. AZUREML_MODEL_DIR doesn’t exist on Runpod, so the model path is a variable you control. The request body is no longer a raw string, so json.loads goes away. And the return value is placed in the output field of the job result. To return an error, raise an exception (the job is marked FAILED) or return a dictionary with an error key. Azure Machine Learning mounts the registered model for you. On Runpod, download the model files first and bake them into the image:
Check the layout of the downloaded folder and make sure model.pt ends up directly under ./model. Then build the worker image:
Dockerfile
The two extra pip flags are needed on this base image: Ubuntu 24.04 marks the system Python as externally managed, and the image’s cryptography package is installed by the OS, so pip can’t upgrade it in place. Build and push it for the right platform:
Baking works well for models up to a few gigabytes. For larger models, keep the image small and load weights from a network volume at /runpod-volume, or use cached models for models hosted on Hugging Face.

Translate a custom container or Container Apps app

If your deployment is a custom container with liveness_route, readiness_route, and scoring_route, or a Container Apps app with ingress, your image already runs an HTTP server. A load balancing endpoint runs it with three changes:
  1. Health check. Runpod polls GET /ping on the worker. 200 means healthy, 204 means still initializing, and anything else means unhealthy. If your server already has a readiness path, set the HEALTH_CHECK_PATH environment variable to it instead of adding a route.
  2. Port. Runpod routes to the port in the PORT environment variable (default 80) and health checks the port in PORT_HEALTH (default 80). Set both to your server’s port, and list that port in the endpoint’s ports.
  3. Auth and URL. Callers send Authorization: Bearer YOUR_API_KEY to https://ENDPOINT_ID.api.runpod.ai/YOUR_PATH. Your routes and payloads stay the same.
A FastAPI app that ran on Container Apps needs only the health route and the port:
app.py
Return 204 from the health route while the model is still loading so the load balancer waits instead of marking the worker unhealthy.

Translate a training job or cloud-init script

For an Azure Machine Learning command job, the job YAML’s command, environment, and compute become a start command, an image, and a GPU type. For a VM, move what cloud-init installed into a Dockerfile and what it ran into the image’s CMD. To keep the job as Python and skip the image entirely, use @app.task from the Runpod SDK, which runs each call on an ephemeral Pod:
train.py
Run it with rp flash dev train.py. The Pod is created for the call and terminated when the function returns. The network volume keeps the checkpoint, and keeps billing for storage until you delete it.

Step 5: Move your data and models

Keep data in Blob Storage

In a hybrid setup, the simplest option is to leave data in Blob Storage and read it over HTTPS from Runpod. Create a SAS token scoped to the container and the permissions the workload needs, with an expiry date, or a service principal with a narrow role assignment. Store the container SAS URL (https://ACCOUNT.blob.core.windows.net/CONTAINER?SAS_TOKEN) as a Runpod secret, reading it from an environment variable so it stays out of your shell history:
Reference the secret from an environment variable on the Pod, template, or endpoint by setting the value to {{ RUNPOD_SECRET_azure_container_sas_url }}. Inside the workload, the Azure SDK reads it like any other URL:
Code that uses DefaultAzureCredential keeps working if you set AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET from secrets, because the credential chain reads a service principal from those variables. Managed identity only works on Azure-hosted compute. If the storage account’s firewall restricts access to selected networks, requests from Runpod arrive from the public internet, so plan for public network access protected by the SAS token or service principal.
Secret names can’t start with runpod. A running Pod or worker keeps the value it booted with, so restart it after you rotate a SAS token or client secret.

Copy data to a network volume

For training data and model weights that workers read often, copy the data to a network volume. Create the volume in the data center where you plan to run, because a network volume and the Pods or workers that mount it must be in the same data center:
Pick the data center after you check GPU stock there. Then choose a transfer method:
Start a Pod with the volume attached, install AzCopy from Microsoft’s Linux download, and sync straight from Blob Storage to the volume:
The data goes directly from Azure to Runpod. azcopy sync is incremental, so you can rerun it before cutover to pick up changes. A CPU Pod is enough for the copy.
rclone’s Azure Blob backend also works from inside a Pod if you already use rclone.

Map each storage type

Network volumes mount at /workspace on Pods by default and at /runpod-volume on Serverless workers, so use an environment variable for model and data paths if the same code runs on both.

Step 6: Deploy on Runpod

Pod (from a GPU VM)

On Azure, a GPU VM that boots a training container looks like this:
The cloud-init file prepares the host and runs the container. This one is trimmed to the last step, after the GPU driver install, Docker install, and data disk mount:
cloud-init.txt
On Runpod, the Pod is the container. The image, environment, volume, and command go straight on the create call:
The network volume mounts at /workspace. If the create fails for lack of stock, try another GPU type that fits, or another data center before you’ve put data on a volume there. Follow the Pod with runpodctl pod logs POD_ID --follow and wait for runtimeStatus to read running in runpodctl pod get POD_ID. The same create is available as POST https://api.runpod.io/v2/pods, and in the console under Pods > Deploy.
A Pod bills until you stop or delete it, even after its command exits. End a batch job by deleting the Pod (runpodctl pod delete POD_ID) from your pipeline, or use @app.task, which terminates the Pod for you.

Queue-based endpoint (from a managed online endpoint)

On Azure, the endpoint and deployment are two YAML files:
endpoint.yml
blue-deployment.yml
You create them with az ml online-endpoint create -f endpoint.yml and az ml online-deployment create -f blue-deployment.yml --all-traffic. On Runpod, the image from Step 4 holds the code, model, and environment. Create a Serverless template for it, then an endpoint:
--gpu-id takes a pool ID here, so workers can land on any 16 GB GPU in the pool. --workers-min 0 scales to zero. Set it to 1 or more to keep a worker running, which is the closest match to instance_count: 1. --workers-max is the ceiling the autoscaler can reach. Send a first request to check it:
The first request includes a cold start while the worker pulls the image and loads the model. If workers don’t start, read runpodctl serverless logs ENDPOINT_ID.

Load balancing endpoint (from Container Apps or a custom container)

On Azure, a Container Apps GPU app is created against a GPU workload profile:
On Runpod, create the endpoint through REST API v2. The endpoint type is set at creation and can’t be changed later, and runpodctl serverless create creates queue-based endpoints:
The field mapping is direct: --target-port becomes ports plus PORT and PORT_HEALTH, --min-replicas and --max-replicas become workers.min and workers.max, and the HTTP concurrency scale rule becomes REQUEST_COUNT scaling, which is the only scaler a load balancing endpoint accepts. timeout is the per-request execution timeout in milliseconds. The response includes requestUrls.base, the URL your clients call. In the console, choose Load Balancer as the endpoint type when you create the endpoint. Load balancing endpoints don’t queue. If no worker is ready within 2 minutes the request fails, so clients need a retry with backoff for cold starts, and long-running or must-not-drop work belongs on a queue-based endpoint.

Public Endpoint or vLLM endpoint (from Azure OpenAI or Foundry models)

If the open-weight model you need is in the Public Endpoint model reference, there’s nothing to deploy: go to Step 7. To serve a different open-weight model, deploy the vLLM worker from the Hub with runpodctl:
The endpoint serves an OpenAI-compatible API at https://api.runpod.ai/v2/ENDPOINT_ID/openai/v1. See Get started with vLLM for GPU sizing and gated models.

Instant Cluster (from CycleCloud or multi-node ND VMs)

Create a Slurm cluster from the console under Instant Clusters > Create Cluster, selecting Slurm Cluster, the GPU type, the Pod count, and an optional network volume in the same region. Slurm and munge are preconfigured, and managed Slurm Clusters require an official Runpod PyTorch image. See Slurm Clusters. Multi-node stock for a given GPU type varies, so confirm it in the console before you schedule a run.

Step 7: Update the code that calls Azure

Scoring requests

Azure clients call the scoring URI with an endpoint key, or go through the SDK:
On a queue-based endpoint, wrap the payload in input and read the result from output:
Check status as well as the HTTP code, because a job that fails in the handler reports "status": "FAILED" in the response body. /runsync waits 90 seconds by default (adjust with the ?wait= query parameter, in milliseconds). If the job is still running when the wait ends, the status is IN_QUEUE or IN_PROGRESS, and you poll /status/JOB_ID for the result. For a load balancing endpoint, only the base URL and the auth header change:

Batch and long-running requests

A batch endpoint invocation (az ml batch-endpoint invoke --name my-batch-endpoint --input ...) starts a job against data in storage. On Runpod, submit one /run request per unit of work, pass the data location in the input, and let a webhook notify you:
The handler reads the blob with the SAS URL from Step 5 and writes results back to Blob Storage. Runpod posts the job result to the webhook when the job finishes, and retries the call up to two more times if your server doesn’t return 200. Results from /run jobs also stay available at /status/JOB_ID for 30 minutes after the job completes. Azure Machine Learning parallelizes a batch job across files for you. On Runpod you fan out by submitting one job per file or shard, and the endpoint’s worker count sets the parallelism.

OpenAI-compatible clients

Azure OpenAI code uses either the AzureOpenAI client with azure_endpoint, api_version, and a deployment name, or the newer v1 API through the standard OpenAI client:
On Runpod, use the standard OpenAI client with a Runpod base URL and API key. There’s no api_version, and model is the model name, not a deployment name:
vLLM workers serve /chat/completions, /completions, and /models. If your code uses the Responses API, Azure content filtering results, or Entra ID token providers, replace those parts: call chat completions, apply your own moderation, and authenticate with the API key. Changing models changes output quality and behavior, so rerun your evaluations before you switch production traffic.

Control-plane automation

Replace Azure CLI and SDK calls in CI and scripts with runpodctl, REST API v2, or the Runpod SDK. The CLI command map and API map list the equivalents. Store RUNPOD_API_KEY in your CI secret store in place of the service principal credentials used for az login.

Step 8: Verify, cut over, and shut down

  1. Send real requests from outside Runpod. Use production-shaped payloads, not a health check. For Pods, connect over SSH or the proxy URL and run the actual job.
  2. Compare outputs. Send the same inputs to both sides and diff the results. Expect small numeric differences if the GPU model changed.
  3. Measure latency, including cold starts. Check the endpoint’s Metrics tab for delay time, execution time, and cold start time. If cold starts are too slow, raise the minimum workers or the idle timeout, or enable FlashBoot.
  4. Run both in parallel. Shift a share of traffic in your client or gateway, watch error rates, then raise the share. Keep the Azure deployment until you’ve seen a full traffic cycle.
  5. Set a cost guard on Runpod. Cap workers-max on every endpoint, enable low balance notifications, and review runpodctl billing pods and runpodctl billing serverless after the first day.
When Runpod has carried full traffic for long enough, stop the matching Azure charges. Deleting a resource group removes everything in it, so only do that if nothing else lives there. Otherwise, go resource by resource:
Then check for what’s commonly left behind:
  • Managed disks and snapshots. Depending on each VM’s delete options, deleting the VM can leave its OS disk, data disks, NIC, and public IP. List unattached disks with az disk list --query '[?managedBy==`null`].[id]' -o tsv, and review az snapshot list.
  • Public IP addresses. Review az network public-ip list --output table and delete addresses that are no longer attached.
  • Compute instances and clusters. A compute instance bills while it’s running, and a stopped one keeps holding quota, so delete the ones you won’t restart. Compute clusters with a minimum node count above zero keep nodes running.
  • Online deployments. Deleting the endpoint deletes its deployments. If you keep the endpoint, delete unused deployments with az ml online-deployment delete.
  • GPU workload profiles. Dedicated GPU workload profiles in a Container Apps environment are allocated per node, not per app, so remove the profile (or the environment) when no app needs it.
  • ACR storage. Delete repositories you no longer pull from with az acr repository delete, or keep the registry if Runpod still pulls from it.
  • Log Analytics workspaces and Application Insights. Workspaces created alongside Container Apps environments and Azure Machine Learning workspaces outlive the workloads and can keep accruing charges for the data they hold. List them with az monitor log-analytics workspace list.
  • Storage accounts. Delete Blob containers and file shares only after you’ve verified the copy on Runpod and you have another backup.
  • Reservations and savings plans. These are commitments that continue after the resources are gone. Review them in the Azure portal.
  • Azure OpenAI and Foundry deployments. Delete provisioned deployments you no longer call.

Migrate your Terraform

The Runpod Terraform provider 2.x is built on REST API v2 and manages Pods, Serverless endpoints, templates, network and global volumes, registry credentials, and Instant Clusters. If you use Bicep or ARM templates, there’s no Runpod equivalent: use Terraform or call REST API v2 from your pipeline.

Resource map

Run both providers side by side

During the transition, declare both providers in one configuration so a single plan shows what’s added on Runpod and what’s removed from Azure:
providers.tf
Keep whatever azurerm version constraint you already use. Add the Runpod resources first and apply, verify the workload, move traffic, and only then remove the Azure resource blocks in a separate change, so terraform destroy behavior for the Azure side is reviewed on its own. Resources from the two providers can reference each other, for example an Azure storage account name in a Runpod endpoint’s env.

Example 1: GPU Linux VM to a Pod

Before, a GPU VM needs a network interface, a disk, and the VM itself, on top of an existing resource group, virtual network, and subnet:
azure-vm.tf
After, the Pod, its storage, and the credential for pulling from ACR are three resources:
runpod-pod.tf
start_ssh injects the SSH keys registered on your Runpod account, which replaces admin_ssh_key, and needs an image that honors the convention (the official Runpod images do). disk is the ephemeral container disk, so anything that must survive goes on the network volume. Changing gpu_type_id, gpu_count, cloud, min_cuda_version, or data_center_ids replaces the Pod. To stop the Pod without removing it from the configuration, set desired_state = "EXITED".

Example 2: Container Apps GPU app to a Serverless endpoint

Before, a GPU app needs an environment with a GPU workload profile and the app with its registry secret, ingress, and scale settings:
azure-container-app.tf
After, a Serverless template holds the container settings and the endpoint holds the GPU pool and scaling. It reuses runpod_registry.acr from the first example:
runpod-endpoint.tf
endpoint_type is required and changing it replaces the endpoint. A load balancing endpoint only accepts REQUEST_COUNT scaling. The request_urls.base output replaces the Container App’s ingress FQDN in your clients. For an Azure Machine Learning online deployment declared with azapi_resource, the target is the same pair of resources with a queue-based endpoint:
runpod-queue-endpoint.tf
instanceType and sku.capacity from the azapi_resource body become gpu_pools and the workers_min to workers_max range. timeout is in milliseconds, like request_timeout_ms. For every argument, see the provider reference on the Terraform Registry. To report a provider problem, open an issue on GitHub.

Reference

CLI command map

API map

Configuration field map

Environment variable map

Status map

Serverless jobs move through IN_QUEUE, IN_PROGRESS, and then COMPLETED, FAILED, CANCELLED, or TIMED_OUT. Workers report INITIALIZING, IDLE, RUNNING, THROTTLED, or UNHEALTHY. THROTTLED means the worker is waiting on GPU capacity.

Migrate with a coding agent

A coding agent can run the inventory, propose the mapping, and migrate one workload at a time. Install the Runpod skills and MCP server first.
The plugin also connects the hosted MCP server:
The MCP server signs in with OAuth (“Sign in with Runpod”), so you don’t need an API key to start. See Agent skills and MCP servers for details. Then give the agent a prompt like this one:

Troubleshooting

The Pod or worker fails to pull an image from ACR. The registry credential is wrong, expired, or scoped to another repository, or the registry blocks public network access. Check that docker login myregistry.azurecr.io works with the same username and password from outside Azure, that the token has content/read on that repository, and that the template or Pod references the registry credential ID. The container exits immediately with an exec format error. The image was built for ARM. Rebuild with docker build --platform linux/amd64. CUDA or driver errors at startup that didn’t happen on Azure. The host’s driver is older than your image’s CUDA runtime expects. Set --min-cuda-version (or gpu.minCudaVersion) to match the CUDA version in your image. The handler fails with a TypeError or KeyError on the input. The code still treats the request as a raw JSON string or expects the payload at the top level. On Runpod, read job["input"], and make sure clients send {"input": {...}}. os.getenv("AZUREML_MODEL_DIR") returns None. That variable is set only by Azure Machine Learning. Point the code at the path where you baked or mounted the model. A load balancing worker shows as running but requests time out or return 502. The health check isn’t passing. Make sure the server listens on 0.0.0.0, the port is listed in ports, both PORT and PORT_HEALTH are set to it, and GET /ping (or your HEALTH_CHECK_PATH) returns 200. {"error":"no workers available"} on the first request. The endpoint scaled to zero and the worker wasn’t ready in time. Retry with backoff, or set the minimum workers to 1 for latency-sensitive endpoints. DefaultAzureCredential fails inside a Pod or worker. There’s no managed identity outside Azure. Set AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET from Runpod secrets, or switch to a SAS URL. Requests to Blob Storage, Key Vault, or a database time out. The Azure resource only allows traffic from a VNet or private endpoint. Enable its public endpoint with authentication and firewall rules, or keep the component that needs private access on Azure and call it through an API you expose. Data written by a job is gone after the Pod restarts. It was written to the container disk. Write to /workspace on a volume disk or network volume, or to /runpod-volume on Serverless. A Pod create fails for lack of capacity. The GPU type isn’t in stock in that data center right now. Try another type that fits the workload or another data center, or deploy from the console with Deploy when available. If a network volume pins you to a data center, check stock there before you copy data.

Next steps

Serverless overview

How endpoints, workers, and scaling work.

Build a load balancing worker

Serve your own HTTP API on Serverless.

Terraform provider

Manage Runpod resources as code.

Network volumes

Persistent storage for Pods and Serverless.
Last modified on November 14, 2026