SaaS gross margin math when LLM tokens are part of COGS

Traditional SaaS gross margin is dominated by hosting and support, both of which scale slowly with usage. Once an AI feature ships, model tokens become a cost of revenue line that scales with every customer action, and the margin you report starts to depend on how your customers behave. This guide shows how to classify the new costs, compute margin at the tenant level, and present the result.

What belongs in cost of revenue

Cost of revenue includes the costs required to deliver the service your customers pay for. For an AI product that means the following, in addition to the hosting and support costs you already count:

• Model API tokens for production traffic: input, output, cached input, cache writes and reasoning tokens.
• Embedding calls and vector database usage that serve customer requests.
• Self-hosted inference infrastructure, if you run open-weight models.
• Gateway, observability and guardrail services that sit in the production request path.
• Third-party data or tool APIs your agents call on behalf of customers.

What stays out

Tokens spent on evaluation runs, prompt experiments, internal tools and model fine-tuning research are research and development, not cost of revenue. The cleanest way to separate them is to tag traffic at the source: production requests carry a real tenant ID in X-PalmSouq-Tenant, and internal traffic carries a reserved tenant such as internal-eval. If you cannot separate them, your gross margin will look worse in the months you run big evaluations, and nobody will be able to explain why.

A worked monthly view

Take a company with $180,000 of monthly revenue. Hosting costs $14,000, the support team allocated to cost of revenue costs $9,000, and third-party tooling in the delivery path costs $4,000. Before AI features, gross margin is ($180,000 - $27,000) / $180,000 = 85.0%.

The AI assistant adds $41,000 of model tokens a month. Gross margin becomes ($180,000 - $68,000) / $180,000 = 62.2%. A 22.8 point drop is material, but on its own it says nothing about whether the pricing is wrong, a few tenants are expensive, or the product team picked an oversized model.

Move from company margin to tenant margin

Company gross margin is an average over customers whose token cost varies by an order of magnitude or more. To act on it you need gross margin per tenant: the tenant's revenue for the period minus the costs you can attribute to it directly, divided by the revenue.

Token costs are directly attributable if every request carries a tenant identifier and is priced at the model that served it. Hosting and support are usually allocated, either evenly per tenant or in proportion to requests. Keep the two separate in your reporting: direct token COGS per tenant is a measurement, allocated costs are a policy, and mixing them hides which one changed.

Three numbers to report every month

• Gross margin with model tokens in cost of revenue, compared with the same month last year.
• The distribution of tenant token COGS as a share of tenant revenue: median, 90th percentile and the count of tenants above 100%.
• Model tokens per dollar of revenue, which isolates pricing and model choice from customer mix.

Common mistakes

The most common mistake is using provider invoice totals as the only source. The invoice is correct for the company and arrives after the month closes, so it cannot be split by tenant and cannot inform anything that happens during the month. The second most common is pricing cached tokens at the uncached input rate, which overstates COGS for tenants with warm caches and understates the effect of cache-breaking prompt changes. The third is ignoring reasoning tokens, which are billed as output even though the customer never sees them.

A metering layer that records input, output, cache reads, cache writes and reasoning tokens per request per tenant, priced at the serving model, fixes all three. Reconcile its monthly total against the provider invoices; the difference should be small and explainable, usually by internal traffic you chose not to route through the gateway.

Presenting it to a board

Boards understand margin compression when they can see its cause. Show the company gross margin with and without model tokens, the tenant distribution, and the actions you are taking on the tail: caps, fallbacks, overage pricing or plan changes. Then show the same chart a quarter later. Per-tenant attribution turns an alarming line into a managed one.