Coding agents are the most loop-prone workloads we meter. They call tools that fail in ways the model cannot see, they carry large context into every step, and they run unattended. This is a teardown of the pattern we use as our reference case, with the cost of each phase shown.
The setup
An autonomous coding agent is asked to rebase a feature branch. The rebase stops on a merge conflict. The agent reads the conflicted file, writes a resolution, and runs git add followed by git rebase --continue. A pre-commit hook rejects the result because a generated lockfile is still out of date. The hook output is truncated by the tool wrapper, so the model only sees a non-zero exit code.
The model does what a reasonable model does with no new information: it tries again. Its next resolution is nearly identical, the hook fails the same way, and the loop begins.
Why each call is expensive
Every step carries the full working context: the system prompt, the tool definitions, the conflicted file and the previous attempts. In our reference case that is 150,000 input tokens per call and 2,000 output tokens. On Claude Sonnet at $3.00 input and $15.00 output per 1M tokens, each call costs $0.48.
That sounds small until you multiply. The agent fires 2,500 calls over 45 minutes: 2,500 x $0.48 = $1,200. If the agent belongs to a customer on a $49 plan, that single session costs more than two years of their subscription.
Where the heuristics fire
Each of the four PALM SOUQ loop heuristics sees this session differently. In practice the first one to trip ends it.
| Heuristic | What it sees | When it trips |
|---|---|---|
| Cyclical tool thrashing | The same run_shell tool with near-identical arguments | Call 15, the configured repeat limit |
| Semantic echo | Assistant messages that are near duplicates of earlier ones | Shortly after, once similarity stays above threshold |
| Token velocity surge | About 140,741 tokens per second for the session | Above the 50k tokens/s default if traffic bursts |
| Max step / recursion depth | Step count climbing without a terminal state | At the step ceiling you configure for the agent |
What the containment cost
Tool thrashing tripped first, at call 15. The session spent 15 x $0.48 = $7.20. The gateway answered call 16 with HTTP 429 and X-PalmSouq-Reason: circuit_breaker_triggered, the agent framework surfaced an error to the user, and the on-call engineer got a webhook with the session ID, the tenant, the repeated tool call and the cost so far.
Prompt caching would have helped too, but not enough on its own. If 140k of the 150k input tokens were served from cache at $0.30 per 1M, each call would cost $0.10, and the full loop $255. Cheaper loops are still loops.
Fixes on the agent side
- Return full tool stderr to the model, or at least the last 40 lines. Most loops start with information the model never saw.
- Track attempts per subgoal in the agent state and escalate to the user after three failures on the same subgoal.
- Treat HTTP 429 with a circuit_breaker_triggered reason as terminal for the session. Retrying it turns a contained loop into a retry storm.
- Give each agent run a cost envelope that the orchestrator checks between steps, in addition to the gateway breaker.
The policy we ship for coding agents
This is the starting policy we recommend for coding agent tenants. Tune the repeat limit after a week of metering; most teams settle between 8 and 20.
breakers:
tool_thrashing:
enabled: true
max_identical_calls: 15
window_seconds: 600
token_velocity:
enabled: true
max_tokens_per_second: 50000
window_seconds: 10
semantic_echo:
enabled: true
max_steps:
enabled: true
limit: 120
on_trip:
respond: 429
notify: ["slack:#agent-oncall", "webhook:https://api.example.com/hooks/palmsouq"]Nisha Pillai
Head of Customer Engineering, PALM SOUQ
BITS Pilani. Previously solutions architect at Wipro for generative AI rollouts. Runs onboarding, breaker tuning workshops and the 15-minute demo.