$799 a Month for One App

The invoice came in on August 1 and I read it twice.

Three hundred and eighty-three dollars and eighty-one cents. For continuous integration. For a company with no revenue, shipping an app with 68 monthly active users. The month before, the same invoice had been $32.66.

I found the cause in about ten minutes. Then I spent three weeks discovering that the cause was the least interesting part of it.

What changed was iOS tests. In July we turned on an end-to-end suite that drives a real iOS simulator, and simulators need Macs, and Macs in CI are expensive. I knew that going in. What I did not know was the actual rate.

A doc I had written myself in early July recorded macOS runners at eight cents a minute and concluded the suite would "often be within the free tier." When the invoice arrived I divided the dollars by the minutes. It came out to about thirty-seven and a half cents a minute. I had been off by a factor of 4.7, in my own notes, for a month, and nothing anywhere in the system told me until the bill did.

That is the first thing I would say to anyone reading this. A per-minute price is not a rate you know. It is a rate you assume, right up until you divide a real invoice by real minutes. We now size jobs on a rule of thumb that came out of that arithmetic: one macOS minute costs about what 47 Linux minutes cost. You make very different decisions at that exchange rate.

The bill nobody publishes

Finding one bad line made me want to see all of them, which I had somehow never done in one sitting. So I pulled every invoice and every cost report for July, the last full month before I touched anything.

Seven hundred and ninety-eight dollars and sixty-two cents, to run one product. A web app, the API behind it, and a mobile client. On top of that sit an Apple developer account at $100 a year and a domain at $10, which add about nine dollars a month if you spread them out, plus a one-time $25 for Google Play.

The zeroes on that last row are not padding. Sentry and PostHog both fit inside free tiers at our volume, and RevenueCat charges nothing until $2,500 of monthly tracked revenue, then takes one percent of what it tracks. It is worth knowing which of your vendors scale with your success and which charge you for existing. The second kind is what this whole thing turned out to be about.

Two lines in that table took me longer than they should have to accept. CI cost more than all of AWS put together. And our dev environment cost more than production.

Only $1.60 of it scaled with usage

Here is that $150.64 of production, broken into billing lines rather than services. I had never looked at it this way either.

The AI features are in there somewhere. They are the part of the product that gets more expensive the more people use it, which is the part everyone worries about. In July they cost $1.60.

One dollar sixty, inside a hundred and fifty dollar bill.

I want to be careful about what that does and does not mean, because it is easy to read it wrong. It does not mean the other $149 was waste, or that it had nothing to do with customers. Production exists so customers can use the product. The NAT gateway so containers in private subnets can reach the internet, the load balancer, the firewall, the database, the two containers never allowed to scale to zero because they serve a website: every one of those is there to serve users, and none of it is optional.

What it means is that almost none of it varies with users. It is the cost of being ready, not the cost of serving. We pay that $149 whether 68 people show up in a month or 68,000. The bill has a floor, and at our size the floor is essentially the whole bill.

I had assumed, without ever examining it, that servers were the expensive part. The servers are $37. The plumbing that lets those servers sit safely in private subnets is $70. Nobody tells you that a properly built AWS environment has a floor somewhere around $120 to $150 a month before a single user shows up, and that the floor is mostly networking.

Our own infrastructure boilerplate estimates $150 a month for a production environment. The real bill was $150.64. The estimate was fine. Production was never the problem.

The cache cost more than the thing it accelerated

Before the dev environment, one more thing I found while looking, because it is both the most embarrassing and the most useful.

Blacksmith gives you a sticky disk for caching Docker layers. It bills at fifty cents per gigabyte per month, and by default it grows forever. Ours had reached 79.68 GB by the end of July. That is about $40 a month, and it existed to speed up $24.30 a month of Docker builds.

The cache cost more than the compute it was there to save. Nothing in any workflow run mentions it. You will not find it by reading logs, because it is storage, and storage does not appear in a log.

Here is the part that still bothers me. When I did my own analysis of the July blowup, I wrote the number down as $346.42, because I had counted Actions minutes off the usage dashboard. The invoice says $383.81. The $37.39 I missed is almost exactly that cache. My investigation into why CI got expensive under-counted it by precisely the one line item that never shows up in a workflow run.

The most expensive thing we owned was one line of YAML

Now the dev environment.

Some of that gap is AI usage, $71.11 of July, from an agent doing work against dev. But strip that out and dev still carried the same fixed networking floor as production, because it was the same architecture. A NAT gateway does not know it is a dev NAT gateway. It charges $33 a month either way.

I looked at those bars for three months and did nothing. I want to be honest about why, because I do not think my reason was unusual.

The dev environment felt load-bearing. It was where changes went before production, and it was doing that job in a very concrete way: both production deploy pipelines gated on Playwright tests run against the deployed dev environment. The production job declared a dependency on the dev end-to-end job. One line of YAML. Delete dev and every production deploy fails at the gate.

So questioning the $189 felt like questioning testing itself, and I did not want to be the person who saved two hundred dollars by turning the tests off. I left it alone. For three months.

What I eventually noticed is that I had been conflating two different things. The tests did not need a deployed environment. They needed a running one. Those are not the same claim, and the distance between them was costing about $200 a month.

A CI runner is a machine with Docker on it. It can start Postgres as a service container, run the migrations, boot the API, build and start the web app, and point Playwright at localhost. The tests cannot tell the difference. There is no NAT gateway in that picture, no load balancer, no database instance, and nothing sitting idle at three in the morning waiting to be tested against.

services:
  postgres:
    image: postgres:16-alpine

steps:
  - run: npx drizzle-kit migrate

  - name: Start API
    run: |
      npm run dev > /tmp/api.log 2>&1 &
      npx wait-on http://localhost:3001/health

  - name: Build and start web (production build, for parity)
    run: |
      npm run build
      npm run start > /tmp/web.log 2>&1 &
      npx wait-on http://localhost:3000

  - run: npx playwright test

Two details make that work rather than half-work. The web app runs as a real production build, not a dev server, so the tests hit what actually ships. And the API takes its configuration from the same secret the mobile test lanes already used, with only the database URL redirected, so there is one source of truth instead of a second copy quietly drifting inside a workflow file.

I built the whole thing locally before touching any pipeline and ran the suite against it. Fifty-four specs, all green. In CI the web suite takes 7.4 minutes and the API suite 1.4, about what the deployed version took, except now nothing has to stay alive between runs.

Then two things happened that I did not expect, and they are why I now think this setup is better rather than merely cheaper.

The first is that it found a real bug in the product. A moderation spec failed locally while passing against dev. The admin queue was rendering timestamps with a bare call to toLocaleString(), no locale pinned. When the server's locale differs from the browser's, React's hydration mismatches, the tree gets regenerated, and the row's click handlers are silently detached. The Remove button did nothing at all. It had been passing for months against dev only because the server and the browser there happened to agree on locale. A more varied environment surfaced a bug that a uniform one had been hiding.

The second is that it found a real bug in the tests. A spec hung for its entire 180-second budget. The trace showed exactly one Playwright call that never completed: a check on whether an element was enabled, on an element the app had re-rendered away between locating it and checking it. Playwright's default action timeout is unlimited, so it waited forever, and the surrounding catch could not save it, because a promise that never settles never rejects.

Neither bug was caused by the migration. Both were already there, and both surfaced because the test environment stopped being one comfortable configuration. After that, tearing dev down was a single command against a directory, which is what you get for describing an environment in code instead of clicking it into existence.

Cutting without lowering the bar

The rest was smaller and less interesting one at a time, so here is all of it at once, with what each one cost us. That third column is the one I would read first in someone else's version of this post.

Four of those deserve their reasoning spelled out, because "we deleted a test lane to save money" is exactly the sentence that should make a reader suspicious.

The iOS lane. I profiled a representative twelve-minute run: five minutes of bundler and pre-warm, three minutes of actual test flows, three and a half minutes of native build. Only that last piece genuinely needs a Mac. The rest is ordinary work we were paying Mac prices for. And the flows themselves were protecting logic that is already unit-tested. The crash that motivated writing them, a hand parser rejecting a ten written as "10" instead of "T", is asserted in a unit test that runs in seconds on every commit. The app IDs are identical across platforms, so the same flows run unedited on an Android emulator on Linux. We moved routine coverage there and kept iOS as the gate on real release binaries, where it still catches what only iOS can break: passkey enrollment, the Google sign-in cookie, safe-area layout.

The deleted CI workflow. This is the one where we genuinely reduced coverage, so I want to be precise about it. It ran type-check, lint, test and build on every pull request push, 292 runs in 26 days, about $13.50. Its suite duplicated the first job of the deploy pipeline exactly. So I checked what it had actually caught: across 203 deploy runs in August, the check jobs and Docker builds failed zero times, and all four genuine failures that month were the end-to-end suite, which that workflow did not run. The compensating control is that the same suite is mandated locally before every commit. CI was not the gate. It was a second, slower, paid copy of the gate. That reasoning could still turn out to be wrong, and if it does, the fix is one file.

Two CI providers on purpose. This is a design decision rather than a cut, and it is the piece I would steal. Our GitHub plan includes 3,000 free Actions minutes a month, and one line of fine print decides everything: those minutes only apply to standard two-core runners. Ask GitHub for anything larger and you pay from the first minute, at rates above what Blacksmith charges for the same cores. Blacksmith's four-core runner is eight tenths of a cent a minute against GitHub's one and a fifth. So the two are not competitors for our workload, they are complements. Anything that fits in two cores goes to GitHub and costs nothing. Anything that needs real cores goes to Blacksmith. Our secret scanner was the obvious first candidate and went from about $4 a month to zero, and we are using a couple of hundred of those 3,000 free minutes, so every future light job has a free home waiting for it.

Halving the runners. We benchmarked rather than guessed: same commit, same steps, same cache, three repetitions each. Eight cores finished in 137 seconds, four cores in 163. Nineteen percent slower for forty-one percent cheaper. That ratio is not luck. We profiled the steps and found the dominant one, the Next.js build, only extracts about two cores of real parallelism. Paying for eight cores to run a two-core step is a donation. The test runs do saturate everything, so those kept their big runners, and Docker builds stayed on eight cores, because slow deploys are not worth a cent and a half.

The verdict on Blacksmith, since this reads like a complaint and is not one: their Linux runners are a real saving and we moved zero Linux jobs away from them. What we got wrong was ours. We treated their macOS runners as the same product with a bigger number, when the exchange rate is closer to 47 to 1, and we treated a sticky disk as a cache rather than as storage that bills forever and appears in no log.

What I would not cut

A cost-cutting post that only lists cuts is marketing. The decisions where I said no are more useful, because that is where the judgment actually lives.

The production NAT gateway is $33 a month, the single largest fixed line in the account, and there is an obvious way to remove it: put the containers in public subnets with public IPs. I am not going to do that. Private subnets with controlled egress is the boundary that means a compromised container is not directly reachable from the internet. Trading that for $30 a month is choosing to be cheaper in a way that shows up later as an incident. The firewall is the same argument at $14.77.

The bastion is $5.52, and my first instinct was to stop it when idle and start it on demand to save about half of that. Then I looked at the session history: twelve sessions in a week, from two different people, nearly every day. Adding friction to a daily-use tool to save three dollars a month is a bad trade, and I had been about to make it purely because it looked like an optimization.

And a Fargate savings plan would take about six dollars a month off, which I turned down. A one-year commitment made by a pre-revenue company against the shape of its current workload is exactly the sort of small saving that becomes a constraint the moment the architecture changes. That one is a real judgment call and I can see it going the other way.

The only line I want to go up

Add it all up and the same system should now run near $230 a month instead of $798.62. About a 71 percent cut, with production architecturally unchanged, the security posture slightly better than it was, and a test suite that covers more than it did. We shipped 236 commits in August while doing it, so none of this was a slowdown.

Every one of those cuts is the same move, and it took me until the end to see it. Something was running continuously to serve a need that was intermittent. A dev environment existed for 720 hours a month to be tested against for maybe three. A pipeline ran on every push to catch a class of failure that had occurred zero times in 203 runs. A cache grew without a ceiling to accelerate builds worth half its cost. A macOS runner spent eight minutes doing Linux-shaped work to reach three minutes of Mac-shaped work.

The question that finds all of them is not "can this be cheaper." It is "what is the duty cycle of this thing, and what am I paying for the other ninety-nine percent of the time." And the answer is almost never to lower the standard. We did not stop testing, we moved the environment from always-on to on-demand. We did not stop scanning for secrets, we stopped scanning the same commit twice. We did not stop testing iOS, we stopped testing iOS on every pull request.

But I keep coming back to that $1.60.

After all of this, production still costs about $125 a month before anybody shows up, and I do not think there is a version of "built properly" that goes much lower. The savings came almost entirely from things that should not have existed, not from the architecture. The architecture was fine. It was always fine.

What those two numbers say together is more useful than either one alone. We have built something that costs roughly $125 a month to keep ready and roughly nothing to actually run. Serving the sixty-ninth user costs about a cent. That is the right shape for a software business, and it is the shape you are supposed to want.

It is also a shape that only pays off with people on it. At 68 users the floor is the entire bill. At 68,000 it would be a rounding error, and not one line of this post would have been worth writing. So I spent three weeks getting very good at cutting the cost of being ready. The harder problem, the one no invoice can help me with, is giving it something to be ready for.

Update log

The July numbers are billed. The $230 is not, yet. Rather than quietly edit this post later, I will keep the changes here.