Deployments, API versions and the gateway
Azure OpenAI addresses models by deployment name inside a resource, with an api-version query parameter on every call. PALM SOUQ preserves both. You register each Azure resource once in the PALM SOUQ console with its endpoint, then point the AzureOpenAI client at gateway.palmsouqltd.com/azure/<resource-name>. The deployment path and api-version pass through untouched.
Authentication can stay as an api-key header or move to Microsoft Entra ID bearer tokens. The gateway forwards either to Azure and never stores the credential.
Shared TPM is the real constraint
Azure assigns tokens-per-minute capacity to each deployment. When one tenant's agent bursts, every other tenant on that deployment gets 429s from Azure. PALM SOUQ's rolling-window TPM limiter runs per tenant in front of the deployment, so a single customer can be capped at, for example, 15% of the deployment's capacity while everyone else keeps their headroom.
When Azure itself returns 429 or 5xx errors, a fallback rule can move the request to a second deployment in another region, or to OpenAI direct, without code changes. The decision is logged with the reason so you can see how often regional capacity is the bottleneck.
Pricing and metering on Azure
Azure OpenAI responses carry the same usage fields as OpenAI, including cached prompt tokens on models that support caching. PALM SOUQ prices them against your Azure rate card rather than OpenAI list prices, because enterprise agreements, Provisioned Throughput Units and regional pricing all change the effective rate.
For PTU deployments, the monthly commitment is allocated across tenants by token share, the same method PALM SOUQ uses for Bedrock provisioned throughput. Finance sees a cost per customer that sums to the Azure invoice.
Rolling it out safely
Register one Azure resource, route one deployment through the gateway, and watch the per-tenant TPM view during a normal business day. It usually shows that a handful of tenants account for most of the capacity pressure, which tells you where to set the first per-tenant limits.
Add a second deployment in another region as a failover target only after the first is stable, and test the failover rule by temporarily lowering its error-rate threshold in staging.
Content filtering stays on the Azure side. When Azure blocks a prompt or a completion, the filter result is passed back to your client unchanged, and PALM SOUQ records the request as filtered for the tenant, so you can see which customers trigger filters most often without the gateway ever storing the content that triggered them.
