Field notes / AI tooling
Receipts from a month of AI agents.
I run most of my studio's engineering through orchestrated coding agents. Here is what a month of that actually used, what it would have cost at API prices, and what an experiment comparing agent teams with subagents showed.
It's easy to say AI agents make you faster. It's harder to show it. So I kept receipts: every session, every token, every automated review, every merged ticket. These are the numbers, with their limits stated plainly.
01 / THE SETUP
One person, many agents.
The work spans Rally, our browser music studio, and a second, newer internal project. A lead agent breaks work into small tickets. Specialist workers each own one phase: one writes a failing test, another makes it pass, and reviewers check the result. Every commit or branch gets an automated code review, and every session is archived locally so usage can be counted afterwards.
- Agents: Claude Code and Codex, on two $100 monthly subscriptions.
- Measurement: agentsview for sessions, tokens and API-equivalent cost; roborev for automated reviews; git, GitHub and project retros for delivery.
02 / USAGE
What a month used.
Over the 30 days to 2 November, the agents processed 14.33 billion tokens. Priced at public API list rates, that's $5,184, about 26 times the $200 actually paid. That figure describes usage, not savings: it's what the same work would have cost on a different plan.
| Window | Tokens | API-equivalent value | Per day |
|---|---|---|---|
| 21 Sep - 20 Oct | 16.16B | $5,931 | $198 |
| 21 Oct - 2 Nov (13 days) | 6.42B | $2,127 | $164 |
| Latest 30 days | 14.33B | $5,184 | $173 |
- 96% of tokens are cache reads. Agents re-read the same context again and again. Freshly generated output is under 0.2% of the total. Big token counts mostly measure long working sessions, not wasted reasoning.
- Opus does the heavy lifting. Claude Opus 5.5 accounted for $1,424 of the latest 13 days; Codex for about a fifth of the value.
- Rally shipped. 17 staging releases since mid-October (v0.74 to v0.92), with the release check growing from 6,278 to 7,676 unit tests.
03 / REVIEW
Reviewing the reviewers.
Automated review is powerful, and it can also loop. In mid-October I capped review panels at two per ticket and stopped reviewing every single commit. The change shows up clearly:
Pass rates rose too: post-commit reviews went from 51% to 79% passing. One caveat: work also shifted to a newer project in the same period, so not all of this is the policy. And the newer project quietly turned per-commit review back on, running 1,108 of them in four days. Rules only work where they're actually configured.
04 / THE EXPERIMENT
Agent teams vs subagents.
Claude Code can coordinate parallel workers in two ways. Agent teams are long-lived teammates that message each other and the lead. Subagents are launched fresh for one job and hand a single result back to the lead. I ran one milestone (26 tickets) as a team, then the next three milestones (35 tickets so far) with subagents.
Even holding the model constant, the same role cost more as a subagent: a median implementer ran $0.79 as a subagent against $0.23 as a teammate. Subagents start with a fresh context on every launch (8.5 launches per ticket, against 4.9), while teammates keep a warm cache across tasks.
Read this as an observation, not a verdict. The runs covered different tickets, partly different models (a rival model filled in during usage limits), a different harness, and a stricter review process for the subagent run that roughly doubled review rounds on its own. One milestone against three is not a controlled test.
05 / FAILURE MODES
How each one breaks.
| Agent teams | Subagents |
|---|---|
| A lead that hits its usage limit strands half-finished work. | Survived three usage-limit stops; paused workers resumed. |
| Workers died silently after a tool upgrade while the lead reported them as running. | One launch path dropped workers' role definitions and hooks. |
| Handoffs over about 16,000 characters were truncated. | Resumed workers lost their isolated checkouts and improvised new ones. |
| Transcripts scatter across folders; my own tooling missed 55 of them. | One transcript per phase, with a typed role and a parent link. Much easier to audit. |
My rule of thumb now: teams for waves of small, related tickets where I'm watching the lead; subagents for long or unattended runs, strict test-first phases, and anything I may need to audit later.
06 / ROI
So, is it worth it?
The financial bar is low. At my own hourly rate, the $200 a month pays for itself if it saves me about two and a half hours a month. Every number above says the agents did a lot of work.
But none of them measures the thing ROI actually depends on: how much of my time it saved. Tokens, sessions, commits and passing tests are activity. The closest proxy I have, time slots where I typed a prompt, counts how often I steered, not how long a task would have taken me without agents. So the honest answer is: very likely yes, not yet proven.
07 / NEXT
What I'm measuring next.
- Hands-on minutes per accepted ticket, recorded as I go. Without it, ROI stays a threshold.
- A fair rematch: matched tickets in each mode, with the same models, effort and review rules.
- A baseline: a few comparable tasks with a single assistant and no orchestration.
- Consistent review rules across every project, plus cost tracking for every review job.
Numbers come from local session archives and review logs, priced at public API list rates. API-equivalent value is not an invoice, and none of it is counted as savings.
More from the studio