How the OpenAI integration works
PALM SOUQ exposes an OpenAI-compatible endpoint at gateway.palmsouqltd.com/v1. Your existing OpenAI SDK keeps its own API key and request shape; you only change the base URL and add tenant headers. The gateway authenticates the request with your PALM SOUQ key, evaluates the tenant's quota and breaker state in memory at the edge, then forwards the call to OpenAI over a pooled, keep-alive connection.
Responses come back exactly as OpenAI sent them. For streaming calls PALM SOUQ reads the server-sent event stream as it passes through, so your users see the first token at the same moment they would without the gateway. When a stream finishes, the usage block is parsed and the call is priced against the current list rate for that model.
What gets metered
OpenAI reports usage in three places that matter for margin: prompt tokens, completion tokens, and the details objects underneath them. PALM SOUQ reads prompt_tokens_details.cached_tokens so automatic prompt caching (which applies to prompts of 1,024 tokens and longer) is billed at the discounted cached rate instead of the full input rate. It reads completion_tokens_details.reasoning_tokens so o3 and o4-mini reasoning spend shows up as its own line, even though OpenAI bills it as output.
If your code streams without usage, the gateway sets stream_options.include_usage on the upstream call and strips the extra usage chunk from what your client receives unless you asked for it. You get exact counts without changing how your client parses the stream.
Governance you can apply to OpenAI traffic
All four loop heuristics run on OpenAI traffic: repeated tool calls with identical argument hashes, token velocity surges, semantic echo between consecutive completions, and step depth per session. Quotas can be set in dollars or tokens per tenant, and fallback chains can downgrade GPT-4.1 to GPT-4.1 mini or GPT-5 to GPT-5 mini when a tenant crosses a budget threshold. A tripped breaker returns HTTP 429 with X-PalmSouq-Reason: circuit_breaker_triggered, which the OpenAI SDK surfaces as a RateLimitError your code already handles.
Batch API jobs can be attributed too. Tag the batch with a tenant header when you create it and PALM SOUQ reconciles the completed output file against the 50% batch discount when it lands.
Rolling it out safely
Most teams move one feature at a time. Point a single OpenAI client, for example the one behind ticket summaries, at the gateway in staging, confirm that usage for your test tenant appears in the live tenants view, then promote it. Breakers start in observe mode by default, so the first week shows what would have been blocked without blocking anything.
Once the numbers match your OpenAI usage dashboard for the same window, move the remaining clients and switch the heuristics you trust to enforce. Rolling back is the same one-line change in reverse.
