Citadel backend contract
Declares the governed AI capacity the gateway can route to, enabling dynamic backend routing without modifying APIM policies:
- Multi-provider backends (Microsoft Foundry, Azure OpenAI, Amazon Bedrock, external/OpenAI-compatible providers)
- Automatic load balancing and failover across backends serving the same model
- Model aliases that abstract a client-facing name over multiple underlying models (with priority/weighted routing)
- Dynamic model discovery (GET /deployments) so clients and Microsoft Foundry can enumerate available models
Example config:
param llmBackendConfig = [
// ----------------------------------
// AI Foundry Backend - Primary
// ----------------------------------
// This backend connects to an Azure AI Foundry project endpoint
// Models deployed in AI Foundry use the OpenAI-compatible inference API
{
backendId: 'aif-citadel-primary'
backendType: 'ai-foundry'
endpoint: 'https://aif-RESOURCE_TOKEN-0.cognitiveservices.azure.com/' // Replace with your AI Foundry endpoint
authType: 'managed-identity'
// Each model has its own metadata for get-available-models response
supportedModels: [
{ name: 'gpt-4o-mini', sku: 'GlobalStandard', capacity: 100, modelFormat: 'OpenAI', modelVersion: '2024-07-18', retirementDate: '2026-09-30' }
{ name: 'gpt-4o', sku: 'GlobalStandard', capacity: 100, modelFormat: 'OpenAI', modelVersion: '2024-11-20', retirementDate: '2026-09-30' }
// gpt-4.1 supports the stateful OpenAI Responses API — flag it so its pool gets session affinity
{ name: 'gpt-4.1', sku: 'GlobalStandard', capacity: 100, modelFormat: 'OpenAI', modelVersion: '2025-04-14', retirementDate: '2026-10-14', apiVersion: '2025-04-01-preview', timeout: 180, sessionAwareModel: true }
{ name: 'DeepSeek-R1', sku: 'GlobalStandard', capacity: 1, modelFormat: 'DeepSeek', modelVersion: '1', retirementDate: '2099-12-30', inferenceApiVersion: '2024-05-01-preview' }
{ name: 'Phi-4', sku: 'GlobalStandard', capacity: 1, modelFormat: 'Microsoft', modelVersion: '3', retirementDate: '2099-12-30', inferenceApiVersion: '2024-05-01-preview' }
{ name: 'text-embedding-3-large', sku: 'GlobalStandard', capacity: 100, modelFormat: 'OpenAI', modelVersion: '1', retirementDate: '2027-04-14' }
]
priority: 1
weight: 100
}
Attention points:
- When load balancing over multiple backend pools, session affinity needs to be configured when using models that are stateful (such as the Responses API). Completion APIs are stateless, so there's no need there.
- Circuit breaking and load balancing use APIM "backends", so make sure you have permissions to create backends.
- Depending on the backend type, URL rewriting and passing extra parameters may be needed.