How Azure maps to Runpod
What’s different
What works differently
You choose a GPU, not a VM size. An Azure size such asStandard_NC24ads_A100_v4 fixes the GPU, vCPUs, memory, and disk together. On Runpod you pick a GPU type and count, and the Pod reports the vCPUs and memory it was allocated. Use the min_vcpu_count_per_gpu and min_ram_per_gpu placement filters when your workload needs a floor.
Pods are containers. There’s no cloud-init, no kernel modules, no systemd, and no NVIDIA driver extension to install: the host provides the driver and your image provides CUDA libraries. Anything your cloud-init script did moves into the image or a start command. Docker Compose and UDP ports aren’t supported.
Stopping means something different. A stopped Azure VM that isn’t deallocated still bills for compute. A stopped Pod doesn’t bill for compute, and its container disk is erased. Its volume disk is kept and still bills. See Pod pricing.
Serverless scales on the queue, not on Azure Monitor rules. A managed online deployment runs a fixed instance_count unless you attach Azure Monitor autoscale rules. A Runpod endpoint has a minimum and maximum worker count and scales on queue delay or request count. The minimum defaults to zero, so an idle endpoint has no running workers.
Requests are wrapped. A queue-based endpoint takes {"input": {...}} and returns {"id", "status", "output"}. Your scoring script’s run(raw_data) receives the raw body, so both the handler and every client change shape. A load balancing endpoint keeps your own request and response shapes.
Limits differ. A managed online endpoint caps a request at 180 seconds. A queue-based endpoint’s execution timeout defaults to 10 minutes in the console and can be raised, with a 10 MB payload limit on /run and 20 MB on /runsync. A load balancing endpoint has a 5.5 minute processing timeout and a 30 MB payload limit.
Rollouts replace workers instead of splitting traffic. Azure Machine Learning splits traffic by percentage across deployments and can mirror traffic. A Runpod endpoint update is a rolling release: idle workers are replaced immediately and busy workers after their current job. For a percentage split, run two endpoints and split in your client or gateway.
Auth is one API key. Runpod endpoints accept Authorization: Bearer YOUR_API_KEY. There’s no equivalent of aad_token auth or per-endpoint keys. A Pod’s HTTP proxy URL is public and unauthenticated, so put your own auth in front of anything you expose on a Pod.
Images must be linux/amd64. Build with docker build --platform linux/amd64, especially on Apple silicon or ARM build agents.
Data transfer. Azure charges for outbound data transfer to the internet beyond a monthly free allowance, so budget for the one-time copy of datasets and model weights out of Azure. Runpod doesn’t charge for data ingress or egress.
What has no direct equivalent
On Runpod:- OpenAI’s proprietary GPT chat models. Runpod doesn’t host the GPT deployments you create in Azure OpenAI. The migration applies when you move to an open-weight model, either one listed in the Public Endpoint model reference or one you serve yourself with vLLM. If you need to keep a proprietary model, keep that call on Azure.
- VNet, NSGs, and Private Link. Runpod has no customer-defined private network or private connectivity to Azure. Pods in your account can reach each other over global networking at
POD_ID.runpod.internal, if you enable it when you deploy them. It’s available on NVIDIA GPU Pods in supported data centers. Traffic between Runpod and Azure goes over the public internet with TLS, so Azure resources behind private endpoints need a public endpoint with firewall rules, or a proxy you run. - Managed identities and Entra ID RBAC for workloads. A Runpod worker can’t assume an Azure identity. Store a least-privilege service principal secret or a SAS token as a Runpod secret and pass it as an environment variable.
- Key Vault. Runpod secrets store values and inject them at boot. They don’t provide key operations, certificates, or versioning. Keep Key Vault for those and copy only the values workloads need.
- Windows and VDI. NV and NG family workloads, Windows GPU VMs, and Data Science VM Windows images stay on Azure.
- Managed Kubernetes. Runpod runs containers directly instead of offering a Kubernetes control plane. Run a scheduler on top of Runpod with SkyPilot or dstack, or move each workload to Pods or Serverless.
- Model registry, pipelines, and experiment tracking. These Azure Machine Learning workspace features stay on Azure, or move to a tool you host. Runpod runs the compute.
- Traffic mirroring and percentage splits across deployments of one endpoint.
- Bicep and ARM templates. Use Terraform or REST API v2.
- Spot capacity through the current API. REST API v2,
runpodctl, and the Terraform provider create on-demand Pods. Plan Spot VM workloads as on-demand Pods or Serverless.
- A job queue built into the inference endpoint. A queue-based Runpod endpoint includes the queue, status polling, streaming, retries, and webhooks. On Azure you’d typically pair an online endpoint with a separate queue service, or use a batch endpoint.
- GPU access without per-family quota requests. Azure gates GPU capacity with per-region, per-VM-family quota. Runpod uses an account-level spend limit and per-data-center stock instead, so check availability before you plan around a GPU type.
- One volume shared across data centers. A global volume mounts on Pods and workers in any data center.
Requirements
- An Azure account with permission to read the resources you plan to move, and the Azure CLI with the
mlandcontainerappextensions if you use those services. - A Runpod account with credits and an API key.
- Docker, to rebuild or retag images for
linux/amd64. - Python 3.10 or later if you use the Runpod SDK.
- Terraform 1.6 or later if you manage infrastructure as code.
Step 1: Take inventory of what you run on Azure
List every GPU resource, then export its configuration. Start with virtual machines. This command lists N-series VMs across the subscription with their size, power state, and location:az vm show -g RESOURCE_GROUP -n VM_NAME returns the full definition. VMs in the Stopped state without being deallocated are still billing, so note those too.
Next, list Azure Machine Learning resources in each workspace:
instance_type, instance_count, the model and environment references, environment_variables, request_settings, and whether it uses a scoring script or a custom container with inference_config routes. Check the endpoint’s auth_mode and traffic with az ml online-endpoint show, and any Azure Monitor autoscale settings attached to the deployment.
List Container Apps, then read each GPU app’s definition:
show output, record the workload profile name, image, CPU and memory, environment variables and secrets, ingress target port, minimum and maximum replicas, and scale rules.
Then list the storage and images those workloads depend on:
Step 2: Set up Runpod
Create an account at console.runpod.io, add credits, and create an API key. Then install the tools you’ll use.- runpodctl
- Runpod SDK
- Terraform
Install the CLI and check that it can reach your account:On macOS you can also run
brew install runpod/runpodctl/runpodctl.Step 3: Choose where each workload lands
Pick a target for each workload based on how it’s invoked, not on which Azure service it runs on today.
Use this table to translate current Azure GPU sizes to Runpod GPU type IDs. GPU counts and memory come from Microsoft’s size documentation, and the Runpod IDs come from the GPU types reference.
Runpod’s list doesn’t include the T4 or A10, so those rows give the closest memory match. Choose by what the workload needs (GPU memory, then throughput), and check live stock before you commit to a type: run
runpodctl gpu list, or call GET /v2/catalog/gpus. Multi-GPU counts and specific types aren’t available in every data center.
Step 4: Move your images and code
Use your Azure Container Registry images
You can keep images in ACR. Runpod pulls private images with a saved username and password, so create a credential that works without an interactive login. Microsoft documents two options for unattended pulls: a repository-scoped token and a service principal. The registry admin user also works, but Microsoft doesn’t recommend sharing it because it has full push and pull access.- Repository-scoped token
- Service principal
Create a token that can only read the repository Runpod needs:The output includes two generated passwords, shown once. The username is the token name. Save it on Runpod, reading the password from an environment variable so it stays out of your shell history:
runpodctl registry create returns a registry credential ID. Pass it as --registry-auth-id when you create a template or Pod, or as registry in REST API v2 and Terraform. Reference the image by its full name, such as myregistry.azurecr.io/inference/worker:v1.
Runpod pulls images over the public internet. If your registry only allows private endpoint access, enable public access with authentication, or push a copy of the image to a registry that’s reachable from outside your VNet.
Check the image
- Architecture: Runpod runs
linux/amd64images only. - CUDA: Your image ships the CUDA runtime, and the host ships the driver. Set a CUDA floor when you create the Pod or endpoint (
--min-cuda-version 12.8, orgpu.minCudaVersionin REST API v2) so the host driver supports your image. - Azure base images: Images built from Azure Machine Learning inference or curated environments include the Azure inference server, which expects the Azure Machine Learning runtime around it. Rebuild from a Runpod PyTorch image such as
runpod/pytorch:1.0.3-cu1281-torch291-ubuntu2404, or from annvidia/cudabase, and install your dependencies from the environment’s conda or pip file. - Start command: A Pod runs the image’s
CMD. Override it with--docker-argsinrunpodctl,argsin Terraform, orcmdin REST API v2.
Translate a scoring script to a handler
An Azure Machine Learning scoring script has two functions.init() runs once when the container starts and loads the model from AZUREML_MODEL_DIR. run(raw_data) runs for every request. Here’s a typical GPU scoring script:
score.py
init() and a handler function replaces run(). The handler receives a job whose input field is already parsed JSON:
handler.py
AZUREML_MODEL_DIR doesn’t exist on Runpod, so the model path is a variable you control. The request body is no longer a raw string, so json.loads goes away. And the return value is placed in the output field of the job result. To return an error, raise an exception (the job is marked FAILED) or return a dictionary with an error key.
Azure Machine Learning mounts the registered model for you. On Runpod, download the model files first and bake them into the image:
model.pt ends up directly under ./model. Then build the worker image:
Dockerfile
pip flags are needed on this base image: Ubuntu 24.04 marks the system Python as externally managed, and the image’s cryptography package is installed by the OS, so pip can’t upgrade it in place.
Build and push it for the right platform:
/runpod-volume, or use cached models for models hosted on Hugging Face.
Translate a custom container or Container Apps app
If your deployment is a custom container withliveness_route, readiness_route, and scoring_route, or a Container Apps app with ingress, your image already runs an HTTP server. A load balancing endpoint runs it with three changes:
- Health check. Runpod polls
GET /pingon the worker.200means healthy,204means still initializing, and anything else means unhealthy. If your server already has a readiness path, set theHEALTH_CHECK_PATHenvironment variable to it instead of adding a route. - Port. Runpod routes to the port in the
PORTenvironment variable (default80) and health checks the port inPORT_HEALTH(default80). Set both to your server’s port, and list that port in the endpoint’sports. - Auth and URL. Callers send
Authorization: Bearer YOUR_API_KEYtohttps://ENDPOINT_ID.api.runpod.ai/YOUR_PATH. Your routes and payloads stay the same.
app.py
204 from the health route while the model is still loading so the load balancer waits instead of marking the worker unhealthy.
Translate a training job or cloud-init script
For an Azure Machine Learning command job, the job YAML’scommand, environment, and compute become a start command, an image, and a GPU type. For a VM, move what cloud-init installed into a Dockerfile and what it ran into the image’s CMD. To keep the job as Python and skip the image entirely, use @app.task from the Runpod SDK, which runs each call on an ephemeral Pod:
train.py
rp flash dev train.py. The Pod is created for the call and terminated when the function returns. The network volume keeps the checkpoint, and keeps billing for storage until you delete it.
Step 5: Move your data and models
Keep data in Blob Storage
In a hybrid setup, the simplest option is to leave data in Blob Storage and read it over HTTPS from Runpod. Create a SAS token scoped to the container and the permissions the workload needs, with an expiry date, or a service principal with a narrow role assignment. Store the container SAS URL (https://ACCOUNT.blob.core.windows.net/CONTAINER?SAS_TOKEN) as a Runpod secret, reading it from an environment variable so it stays out of your shell history:
{{ RUNPOD_SECRET_azure_container_sas_url }}. Inside the workload, the Azure SDK reads it like any other URL:
DefaultAzureCredential keeps working if you set AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET from secrets, because the credential chain reads a service principal from those variables. Managed identity only works on Azure-hosted compute. If the storage account’s firewall restricts access to selected networks, requests from Runpod arrive from the public internet, so plan for public network access protected by the SAS token or service principal.
Copy data to a network volume
For training data and model weights that workers read often, copy the data to a network volume. Create the volume in the data center where you plan to run, because a network volume and the Pods or workers that mount it must be in the same data center:- AzCopy from a Pod
- Cloud Sync
- S3-compatible API
Start a Pod with the volume attached, install AzCopy from Microsoft’s Linux download, and sync straight from Blob Storage to the volume:The data goes directly from Azure to Runpod.
azcopy sync is incremental, so you can rerun it before cutover to pick up changes. A CPU Pod is enough for the copy.Map each storage type
Network volumes mount at
/workspace on Pods by default and at /runpod-volume on Serverless workers, so use an environment variable for model and data paths if the same code runs on both.
Step 6: Deploy on Runpod
Pod (from a GPU VM)
On Azure, a GPU VM that boots a training container looks like this:cloud-init.txt
/workspace. If the create fails for lack of stock, try another GPU type that fits, or another data center before you’ve put data on a volume there. Follow the Pod with runpodctl pod logs POD_ID --follow and wait for runtimeStatus to read running in runpodctl pod get POD_ID. The same create is available as POST https://api.runpod.io/v2/pods, and in the console under Pods > Deploy.
Queue-based endpoint (from a managed online endpoint)
On Azure, the endpoint and deployment are two YAML files:endpoint.yml
blue-deployment.yml
az ml online-endpoint create -f endpoint.yml and az ml online-deployment create -f blue-deployment.yml --all-traffic.
On Runpod, the image from Step 4 holds the code, model, and environment. Create a Serverless template for it, then an endpoint:
--gpu-id takes a pool ID here, so workers can land on any 16 GB GPU in the pool. --workers-min 0 scales to zero. Set it to 1 or more to keep a worker running, which is the closest match to instance_count: 1. --workers-max is the ceiling the autoscaler can reach. Send a first request to check it:
runpodctl serverless logs ENDPOINT_ID.
Load balancing endpoint (from Container Apps or a custom container)
On Azure, a Container Apps GPU app is created against a GPU workload profile:runpodctl serverless create creates queue-based endpoints:
--target-port becomes ports plus PORT and PORT_HEALTH, --min-replicas and --max-replicas become workers.min and workers.max, and the HTTP concurrency scale rule becomes REQUEST_COUNT scaling, which is the only scaler a load balancing endpoint accepts. timeout is the per-request execution timeout in milliseconds. The response includes requestUrls.base, the URL your clients call. In the console, choose Load Balancer as the endpoint type when you create the endpoint.
Load balancing endpoints don’t queue. If no worker is ready within 2 minutes the request fails, so clients need a retry with backoff for cold starts, and long-running or must-not-drop work belongs on a queue-based endpoint.
Public Endpoint or vLLM endpoint (from Azure OpenAI or Foundry models)
If the open-weight model you need is in the Public Endpoint model reference, there’s nothing to deploy: go to Step 7. To serve a different open-weight model, deploy the vLLM worker from the Hub withrunpodctl:
https://api.runpod.ai/v2/ENDPOINT_ID/openai/v1. See Get started with vLLM for GPU sizing and gated models.
Instant Cluster (from CycleCloud or multi-node ND VMs)
Create a Slurm cluster from the console under Instant Clusters > Create Cluster, selecting Slurm Cluster, the GPU type, the Pod count, and an optional network volume in the same region. Slurm and munge are preconfigured, and managed Slurm Clusters require an official Runpod PyTorch image. See Slurm Clusters. Multi-node stock for a given GPU type varies, so confirm it in the console before you schedule a run.Step 7: Update the code that calls Azure
Scoring requests
Azure clients call the scoring URI with an endpoint key, or go through the SDK:input and read the result from output:
status as well as the HTTP code, because a job that fails in the handler reports "status": "FAILED" in the response body. /runsync waits 90 seconds by default (adjust with the ?wait= query parameter, in milliseconds). If the job is still running when the wait ends, the status is IN_QUEUE or IN_PROGRESS, and you poll /status/JOB_ID for the result.
For a load balancing endpoint, only the base URL and the auth header change:
Batch and long-running requests
A batch endpoint invocation (az ml batch-endpoint invoke --name my-batch-endpoint --input ...) starts a job against data in storage. On Runpod, submit one /run request per unit of work, pass the data location in the input, and let a webhook notify you:
200. Results from /run jobs also stay available at /status/JOB_ID for 30 minutes after the job completes. Azure Machine Learning parallelizes a batch job across files for you. On Runpod you fan out by submitting one job per file or shard, and the endpoint’s worker count sets the parallelism.
OpenAI-compatible clients
Azure OpenAI code uses either theAzureOpenAI client with azure_endpoint, api_version, and a deployment name, or the newer v1 API through the standard OpenAI client:
OpenAI client with a Runpod base URL and API key. There’s no api_version, and model is the model name, not a deployment name:
/chat/completions, /completions, and /models. If your code uses the Responses API, Azure content filtering results, or Entra ID token providers, replace those parts: call chat completions, apply your own moderation, and authenticate with the API key. Changing models changes output quality and behavior, so rerun your evaluations before you switch production traffic.
Control-plane automation
Replace Azure CLI and SDK calls in CI and scripts withrunpodctl, REST API v2, or the Runpod SDK. The CLI command map and API map list the equivalents. Store RUNPOD_API_KEY in your CI secret store in place of the service principal credentials used for az login.
Step 8: Verify, cut over, and shut down
- Send real requests from outside Runpod. Use production-shaped payloads, not a health check. For Pods, connect over SSH or the proxy URL and run the actual job.
- Compare outputs. Send the same inputs to both sides and diff the results. Expect small numeric differences if the GPU model changed.
- Measure latency, including cold starts. Check the endpoint’s Metrics tab for delay time, execution time, and cold start time. If cold starts are too slow, raise the minimum workers or the idle timeout, or enable FlashBoot.
- Run both in parallel. Shift a share of traffic in your client or gateway, watch error rates, then raise the share. Keep the Azure deployment until you’ve seen a full traffic cycle.
- Set a cost guard on Runpod. Cap
workers-maxon every endpoint, enable low balance notifications, and reviewrunpodctl billing podsandrunpodctl billing serverlessafter the first day.
- Managed disks and snapshots. Depending on each VM’s delete options, deleting the VM can leave its OS disk, data disks, NIC, and public IP. List unattached disks with
az disk list --query '[?managedBy==`null`].[id]' -o tsv, and reviewaz snapshot list. - Public IP addresses. Review
az network public-ip list --output tableand delete addresses that are no longer attached. - Compute instances and clusters. A compute instance bills while it’s running, and a stopped one keeps holding quota, so delete the ones you won’t restart. Compute clusters with a minimum node count above zero keep nodes running.
- Online deployments. Deleting the endpoint deletes its deployments. If you keep the endpoint, delete unused deployments with
az ml online-deployment delete. - GPU workload profiles. Dedicated GPU workload profiles in a Container Apps environment are allocated per node, not per app, so remove the profile (or the environment) when no app needs it.
- ACR storage. Delete repositories you no longer pull from with
az acr repository delete, or keep the registry if Runpod still pulls from it. - Log Analytics workspaces and Application Insights. Workspaces created alongside Container Apps environments and Azure Machine Learning workspaces outlive the workloads and can keep accruing charges for the data they hold. List them with
az monitor log-analytics workspace list. - Storage accounts. Delete Blob containers and file shares only after you’ve verified the copy on Runpod and you have another backup.
- Reservations and savings plans. These are commitments that continue after the resources are gone. Review them in the Azure portal.
- Azure OpenAI and Foundry deployments. Delete provisioned deployments you no longer call.
Migrate your Terraform
The Runpod Terraform provider 2.x is built on REST API v2 and manages Pods, Serverless endpoints, templates, network and global volumes, registry credentials, and Instant Clusters. If you use Bicep or ARM templates, there’s no Runpod equivalent: use Terraform or call REST API v2 from your pipeline.Resource map
Run both providers side by side
During the transition, declare both providers in one configuration so a single plan shows what’s added on Runpod and what’s removed from Azure:providers.tf
azurerm version constraint you already use. Add the Runpod resources first and apply, verify the workload, move traffic, and only then remove the Azure resource blocks in a separate change, so terraform destroy behavior for the Azure side is reviewed on its own. Resources from the two providers can reference each other, for example an Azure storage account name in a Runpod endpoint’s env.
Example 1: GPU Linux VM to a Pod
Before, a GPU VM needs a network interface, a disk, and the VM itself, on top of an existing resource group, virtual network, and subnet:azure-vm.tf
runpod-pod.tf
start_ssh injects the SSH keys registered on your Runpod account, which replaces admin_ssh_key, and needs an image that honors the convention (the official Runpod images do). disk is the ephemeral container disk, so anything that must survive goes on the network volume. Changing gpu_type_id, gpu_count, cloud, min_cuda_version, or data_center_ids replaces the Pod. To stop the Pod without removing it from the configuration, set desired_state = "EXITED".
Example 2: Container Apps GPU app to a Serverless endpoint
Before, a GPU app needs an environment with a GPU workload profile and the app with its registry secret, ingress, and scale settings:azure-container-app.tf
runpod_registry.acr from the first example:
runpod-endpoint.tf
endpoint_type is required and changing it replaces the endpoint. A load balancing endpoint only accepts REQUEST_COUNT scaling. The request_urls.base output replaces the Container App’s ingress FQDN in your clients.
For an Azure Machine Learning online deployment declared with azapi_resource, the target is the same pair of resources with a queue-based endpoint:
runpod-queue-endpoint.tf
instanceType and sku.capacity from the azapi_resource body become gpu_pools and the workers_min to workers_max range. timeout is in milliseconds, like request_timeout_ms. For every argument, see the provider reference on the Terraform Registry. To report a provider problem, open an issue on GitHub.
Reference
CLI command map
API map
Configuration field map
Environment variable map
Status map
Serverless jobs move through
IN_QUEUE, IN_PROGRESS, and then COMPLETED, FAILED, CANCELLED, or TIMED_OUT. Workers report INITIALIZING, IDLE, RUNNING, THROTTLED, or UNHEALTHY. THROTTLED means the worker is waiting on GPU capacity.
Migrate with a coding agent
A coding agent can run the inventory, propose the mapping, and migrate one workload at a time. Install the Runpod skills and MCP server first.- Claude Code
- Other agents
The plugin also connects the hosted MCP server:
Troubleshooting
The Pod or worker fails to pull an image from ACR. The registry credential is wrong, expired, or scoped to another repository, or the registry blocks public network access. Check thatdocker login myregistry.azurecr.io works with the same username and password from outside Azure, that the token has content/read on that repository, and that the template or Pod references the registry credential ID.
The container exits immediately with an exec format error. The image was built for ARM. Rebuild with docker build --platform linux/amd64.
CUDA or driver errors at startup that didn’t happen on Azure. The host’s driver is older than your image’s CUDA runtime expects. Set --min-cuda-version (or gpu.minCudaVersion) to match the CUDA version in your image.
The handler fails with a TypeError or KeyError on the input. The code still treats the request as a raw JSON string or expects the payload at the top level. On Runpod, read job["input"], and make sure clients send {"input": {...}}.
os.getenv("AZUREML_MODEL_DIR") returns None. That variable is set only by Azure Machine Learning. Point the code at the path where you baked or mounted the model.
A load balancing worker shows as running but requests time out or return 502. The health check isn’t passing. Make sure the server listens on 0.0.0.0, the port is listed in ports, both PORT and PORT_HEALTH are set to it, and GET /ping (or your HEALTH_CHECK_PATH) returns 200.
{"error":"no workers available"} on the first request. The endpoint scaled to zero and the worker wasn’t ready in time. Retry with backoff, or set the minimum workers to 1 for latency-sensitive endpoints.
DefaultAzureCredential fails inside a Pod or worker. There’s no managed identity outside Azure. Set AZURE_TENANT_ID, AZURE_CLIENT_ID, and AZURE_CLIENT_SECRET from Runpod secrets, or switch to a SAS URL.
Requests to Blob Storage, Key Vault, or a database time out. The Azure resource only allows traffic from a VNet or private endpoint. Enable its public endpoint with authentication and firewall rules, or keep the component that needs private access on Azure and call it through an API you expose.
Data written by a job is gone after the Pod restarts. It was written to the container disk. Write to /workspace on a volume disk or network volume, or to /runpod-volume on Serverless.
A Pod create fails for lack of capacity. The GPU type isn’t in stock in that data center right now. Try another type that fits the workload or another data center, or deploy from the console with Deploy when available. If a network volume pins you to a data center, check stock there before you copy data.
Next steps
Serverless overview
How endpoints, workers, and scaling work.
Build a load balancing worker
Serve your own HTTP API on Serverless.
Terraform provider
Manage Runpod resources as code.
Network volumes
Persistent storage for Pods and Serverless.