Platform overview
The Financial OS for GenAI Engineering & SaaS Margins.
PALM SOUQ sits in the request path between your agents and every model provider. It prices each token for the tenant that caused it, stops runaway loops before the next call is billed, and syncs usage into your billing system.
Request path
Six stages, one round trip.
Step through what happens to a single governed call, from your SDK to the billing webhook. Everything before the provider runs inside the latency budget; everything after it runs off the request path.
Client SDK
0 msYour existing OpenAI or Anthropic client with base_url set to the gateway. Tenant, tier, feature and user ride along as headers. The PALM SOUQ SDK wrapper adds fail-open behaviour: if the gateway is unreachable, it calls the provider directly and queues the usage record.
- POST
- https://gateway.palmsouqltd.com/v1/chat/completions
- X-PalmSouq-Tenant
- acme-legal
- X-PalmSouq-Tier
- pro
- X-PalmSouq-Feature
- contract-review
- fail_mode
- open
- Added latency at p99
- <5 ms
- Loop detected to 429
- <50 ms
- Gateway SLA (Scale)
- 99.999%
- Edge points of presence
- 280+
High availability
Fail-open by default, so the gateway is never your outage.
The gateway runs active-active across five regions behind anycast. Even so, a network partition between your servers and the edge is possible, and you decide in advance what it means.
- Fail-open keeps 100% of requests flowing and replays usage later.
- Fail-closed guarantees no unmetered spend during the window.
- Set it globally, per tenant tier or per feature.
SDK fail mode
Network to gateway
- Route
- SDK → provider (gateway bypassed after 250 ms)
- Metering
- Usage queued in SDK, replayed on recovery
- Breakers & quotas
- Paused for the outage window
- User impact
- None
Fail-open keeps your product up. The tradeoff: calls made during the partition are not governed in real time, only metered after the fact.
Four pillars
Everything that touches token spend, in one path.
Per-tenant metering
Input, output, cached and reasoning tokens priced per request and attributed to tenant, workspace, user, feature and model.
Runaway loop circuit breakers
Four heuristics evaluated in path. A tripped breaker returns HTTP 429 before the next call is billed.
Quotas & tier enforcement
Monthly dollar caps, daily token limits and rolling TPM per tier, with soft-cap webhooks at the threshold you choose.
Model fallbacks
Downgrade to a cheaper model or a different provider instead of locking a paying customer out.
Usage billing sync
Overage tokens become meter events in Stripe, Metronome, Orb or Lago, with revenue vs COGS per account.
Scope
What PALM SOUQ does not do.
Clear boundaries make a proxy easier to trust. These are deliberate choices, not gaps on a roadmap.
| PALM SOUQ does not | What that means for you |
|---|---|
| Sell or resell AI models | You keep your own provider accounts, keys, rate limits and enterprise agreements. PALM SOUQ never marks up tokens. |
| Store or train on your prompts | Zero Payload Retention: prompts and completions stream through memory and are never written to disk. We keep counts, ids and costs. |
| Require Kubernetes sidecars | No agents to deploy and no service mesh changes. One base URL and a few headers; an optional thin SDK adds fail-open. |
| Replace your observability stack | We export metrics and spans to Datadog or any OpenTelemetry backend instead of asking you to move. |
| Decide your pricing | We give finance exact COGS per customer. What you charge, and how, stays in your billing system. |
Integration ecosystem
Your providers, your billing engine, your dashboards.
Model providers and clouds
Billing, observability and frameworks
Platform questions
How much latency does the gateway add?
Under 5 ms at p99 for requests handled by the nearest edge PoP. Policy and counters live in edge memory, so the in-path decision does not wait on a database. Streaming responses are relayed chunk by chunk, so time to first token is unchanged apart from that budget.
What happens if PALM SOUQ is unavailable?
With the default fail-open SDK setting, your application calls the provider directly when the gateway cannot be reached, and usage is queued and replayed when it recovers. You can choose fail-closed per tenant or per feature when budget containment matters more than availability.
Does PALM SOUQ see my prompts?
Prompts and completions pass through gateway memory so they can be forwarded, and are never written to disk, logs or analytics. Loop heuristics work on hashes of tool names and arguments, token counts and short-lived similarity fingerprints that expire with the session.
Which providers are supported?
OpenAI, Anthropic, Google Gemini and Vertex AI, AWS Bedrock, Azure OpenAI, DeepSeek, Groq, Mistral and Together AI, plus any OpenAI-compatible endpoint you register, such as a self-hosted vLLM server.
Don't wait for a surprise $10,000 OpenAI invoice.
Route your agent traffic through PALM SOUQ in 2 minutes. Change one base URL, pass a tenant header, and every token is attributed and governed.
14-day free production trial · 100k requests included · no card required