Rate limiting is the first control most teams add when they put an LLM feature in front of customers. It is a good control. It protects you from bursts, keeps one tenant from starving others, and maps cleanly onto provider limits. What it does not do is cap what a tenant costs you in a month, and the gap between the two is larger than most people expect.
Speed is not spend
Suppose every tenant on your pro plan is limited to 20,000 tokens per minute. A tenant that runs at the limit for two hours a working day, 22 days a month, sends 20,000 x 60 x 2 x 22 = 52.8M tokens. With 90% of those as input, that tenant costs $73.92 on Claude Haiku 4.5 and $1,108.80 on Claude Opus 4.1.
Same limit, same tenant behavior, a 15.0x difference in cost. A rate limit is denominated in tokens per unit of time. Your margin is denominated in dollars per month. Converting between them requires the model, the cache behavior and the active hours, and all three change.
Where rate limits still belong
- Fairness between tenants that share a provider account and its upstream limits.
- Burst protection, so one tenant's batch job does not trigger provider 429s for everyone.
- Abuse protection on free tiers, where the risk is volume rather than any one expensive call.
Add a dollar budget beside the limit
A per-tenant budget counts cost, not tokens. Each response is priced from its usage block at the serving model's price, including cache reads and writes and reasoning tokens, and the running total is compared with the tenant's monthly budget on the next request. When you switch a feature from one model to another, the rate limit means something different overnight, but the budget does not.
In PALM SOUQ both live on the tier policy. The rate limit uses a sliding window at the edge; the budget uses the metering ledger. A request is admitted only if both allow it.
tiers:
pro:
rate_limit:
tokens_per_minute: 20000
requests_per_minute: 120
budget:
monthly_usd: 150
soft_cap_percent: 80
on_exhausted: fallback
fallback_chain: ["claude-haiku-4.5"]A note on reasoning tokens
Reasoning models make the gap wider. Hidden reasoning tokens are billed as output tokens but are not part of the visible completion, so a limit that counts only visible tokens undercounts spend badly. Whatever you rate limit on, budget on the provider-reported usage, which includes reasoning tokens.
How to set the two together
Set the rate limit from the upstream provider limits you share and from what an interactive user can plausibly need. Set the budget from the plan price and your target gross margin: a $99 plan with a 75% margin target can afford about $25 of model cost per tenant per month, before infrastructure. Then check the budget against the rate limit on your most expensive model. If a tenant can exhaust the budget in a few days at the rate limit, the budget is doing the real work and the rate limit is only a burst guard, which is fine as long as you know it.
Finally, review both after every model change. Moving a feature to a newer model often changes the price per token, the typical output length and the cache behavior at once, and each of those shifts what a given rate limit means in dollars. The budget absorbs the change automatically; the rate limit may need a new number.
Vineet Saresar
Founder & CEO, PALM SOUQ
Founder of PALM SOUQ. Sets company direction, product strategy and customer partnerships, and leads the work to make runaway LLM spend impossible for AI SaaS teams.