Every security review of an LLM gateway starts with the same question: does the proxy keep our prompts? For PALM SOUQ the answer is no, and we call the design Zero Payload Retention. This post is the field-by-field version of that answer, written for the security engineer filling in the vendor questionnaire.
The principle
Prompts and completions stream through gateway memory and are never written to disk, to logs, to queues or to analytics stores. The gateway reads what it needs to price and govern the request (the model name, the usage block in the provider response, the PALM SOUQ headers) and discards the body when the response finishes streaming.
This is not a retention period. There is no 30-day window after which payloads are deleted, because payloads are never persisted in the first place.
Field by field
| Field | Handling | Why |
|---|---|---|
| Prompt and message text | Streamed through memory, never stored | Not needed for metering |
| Completion text | Streamed through memory, never stored | Not needed for metering |
| Tool call arguments | Normalized and hashed in memory for loop detection, hash expires with the session window | Tool thrashing heuristic |
| Assistant turn fingerprint | Similarity fingerprint in memory, expires with the session window | Semantic echo heuristic |
| Token counts (input, output, cache read, cache write, reasoning) | Stored | Metering and billing |
| Model, provider, latency, status code | Stored | Metering and reliability |
| X-PalmSouq-Tenant, Tier, Feature, User | Stored as sent | Attribution |
| Provider API keys | Encrypted at rest, decrypted in memory per request | Forwarding |
What the headers should contain
The attribution headers are stored, so they should not carry personal data you do not want in a metering store. We recommend opaque identifiers: your internal customer ID in X-PalmSouq-Tenant and a pseudonymous user ID in X-PalmSouq-User, never an email address or a name. The SDK can hash the user header client-side if you prefer.
Debugging without payloads
The usual objection is that payload logging is useful when debugging a loop. We agree that it is useful, and we think it belongs in your own systems, under your retention policy. The incident webhook gives you the session ID, timestamps, heuristic counters and cost so you can join against your own application logs. If you need payload capture for a specific feature, you can export spans through the OpenTelemetry integration to your own collector, and the gateway still does not keep a copy.
How we verify it
- Gateway nodes run with no writable volume for request data; only the metering stream leaves the process.
- Structured logging uses an allow list of fields. Body fields are not on it, and a CI check fails the build if a log call references a request body.
- Our SOC 2 Type II audit covers the retention controls, and the report is available under NDA from the trust center.
- Penetration tests include attempts to recover payloads from memory dumps, crash reports and log pipelines.
Regions and residency
Because payloads are never persisted, residency questions reduce to two things: which point of presence processes the request in memory, and where the metering records live. Requests are processed at the nearest of 280+ edge points of presence unless you pin a tenant to a region. Metering records are stored in the region you choose for your organization, with EU and India options for teams that need them.
Where to go next
The security page has an interactive data flow that shows each field's path through the gateway, and the trust center lists subprocessors, policies and how to request the SOC 2 report. For HIPAA workloads, a Business Associate Agreement is available on the Scale plan.
Kabir Sethi
Head of Security & Compliance, PALM SOUQ
IIT Delhi. Eight years in security engineering at Infosys and a cloud security startup. Leads the SOC 2 Type II program and Zero Payload Retention design reviews.