Managed Identity for Azure AI Services

The Problem with API Keys

Every Azure AI service generates two API keys by default. Copy one into your app config and you're running. Fast to set up, but not safe to keep.

API keys are static secrets. They don't expire unless you manually rotate them. They get committed to Git repos, pasted into Slack messages, and hardcoded into environment variables that nobody updates. If a key leaks, anyone with that string can call your Azure OpenAI endpoint or query your AI Search index. On your bill.

Managed identity eliminates this risk. Instead of passing a secret, your Azure resource authenticates through Entra ID, and Azure handles the token exchange behind the scenes. No keys to rotate, no secrets to leak, no credentials in your code.

System-Assigned vs User-Assigned

Azure offers two types of managed identity:

System-assigned is created automatically when you enable it on a resource. The identity lifecycle is tied to the resource itself. Delete the App Service, the identity goes with it. This is the simplest option for most scenarios.
User-assigned is a standalone Azure resource you create separately and attach to one or more resources. Use this when multiple services need the same permissions, or when the identity needs to persist independently.

For a single application calling Azure OpenAI, system-assigned is the right default. For a microservices architecture where five apps all need the same AI Search index, user-assigned avoids managing five separate role assignments.

Enable Managed Identity and Assign Roles

Here's how to set up managed identity access to Azure OpenAI, step by step.

The key RBAC roles for Azure AI services:

Cognitive Services OpenAI User for inference calls (chat, completions, embeddings)
Cognitive Services OpenAI Contributor adds the ability to manage model deployments
Cognitive Services User covers all Cognitive Services types (broader scope)
Search Index Data Reader for querying Azure AI Search indexes

Same Pattern for Document Intelligence and AI Search

The pattern is identical across all Azure AI services: enable managed identity on the compute resource, then assign the right RBAC role on the target service. The only thing that changes is the role name.

Role Assignment with Bicep

If you deploy infrastructure with Bicep, role assignments are declarative:

The GUID 5e0bd9bd-7b93-4f28-af87-19fc36ad61bd is the built-in role definition ID for Cognitive Services OpenAI User. All built-in role IDs are listed in the Azure RBAC documentation.

Disabling API Key Access

Once managed identity is working, disable API key authentication entirely. This removes the attack surface completely:

After this, only Entra ID tokens work. Any attempt to authenticate with an API key returns 403.

Making the Switch on Existing Applications

For new deployments, start with managed identity from day one. For existing applications running on API keys:

Enable managed identity and set up RBAC roles alongside the existing API key
Update your application code to use DefaultAzureCredential (more on this in our next post)
Test thoroughly in a staging environment
Disable API key access in production once confident

Both authentication methods can coexist during migration, giving you a safe rollback path. You do not need to switch everything at once.

Managed Identity for Azure AI Services: Why You Should Stop Using API Keys