Azure AI Private Endpoints and Managed Identity

Why Private Endpoints for AI Services

Azure AI services are public by default. That means Azure OpenAI, AI Search, and Document Intelligence all expose endpoints that anyone on the internet can reach (assuming they have the API key). For dev/test, that might be fine. For production workloads handling sensitive data, it is not.

Private endpoints put these services inside your VNet. Combined with disabling public network access and disabling local auth (API keys), you get a setup where only identity-authenticated traffic from your VNet can reach the services. No keys to rotate, no public surface to attack.

Architecture Overview

The target architecture is a RAG (Retrieval-Augmented Generation) pipeline with four core services:

Azure OpenAI for inference (chat completions, embeddings)
AI Search for vector and keyword retrieval
Document Intelligence for extracting text and structure from uploaded documents
Blob Storage for raw document storage

All four sit behind private endpoints in the same VNet. They authenticate to each other via managed identity. No API keys, no connection strings, no public endpoints.

Bicep: Core Network and Private DNS

Start with the networking foundation: a VNet with a dedicated subnet for private endpoints and another for compute resources (VMs, App Service, Container Apps, etc.). Each Azure service needs its own private DNS zone so that DNS resolution routes traffic to the private IP instead of the public endpoint.

You will also need to link each DNS zone to the VNet (via Microsoft.Network/privateDnsZones/virtualNetworkLinks) and create DNS zone groups on each private endpoint. Those are omitted here for brevity, but they follow the same pattern for every service.

Bicep: Azure OpenAI with Private Endpoint

Deploy the Azure OpenAI resource with three security settings: publicNetworkAccess: 'Disabled' blocks all internet traffic, disableLocalAuth: true forces Entra ID authentication (no API keys), and the system-assigned managed identity gives the resource its own identity for RBAC.

The same pattern applies to AI Search and Document Intelligence. Each gets its own Cognitive Services account (with the appropriate kind), its own private endpoint, and the same three security properties.

Managed Identity Chain for Service-to-Service Auth

With API keys disabled, services authenticate to each other using managed identity and Azure RBAC. In the RAG pattern, AI Search needs two role assignments:

Storage Blob Data Reader on the storage account, so AI Search indexers can read documents from Blob Storage
Cognitive Services OpenAI User on the Azure OpenAI resource, so AI Search can call the embedding model during indexing

No connection strings, no shared secrets. The identity chain is fully declarative in Bicep:

The role definition IDs above are the well-known GUIDs published by Microsoft. They are stable and the same across all Azure tenants.

Verifying the Setup

After deployment, verify that private endpoints are connected and public access is actually off. The Azure CLI makes this straightforward:

You should see Approved for each private endpoint status, Disabled for PublicAccess, and true for LocalAuth (meaning local auth is disabled).

Locking Down Azure AI: Private Endpoints and Identity-Based Access for OpenAI, AI Search, and Document Intelligence