Your AI-built app works. But is it ready to launch?

Mik Bry · 2026-07-22 · ~9 min read · AI · Security

I've sat with founders at the exact moment they reach for the deploy button. The app works — pages load, sign-up works, a payment form appears — and for a few seconds that feels like enough. It never is. The demo answers one question: does the expected path work? What I actually get asked to answer is harder: what happens the moment someone stops following that path — a payment notification that fires twice, an identifier changed in the URL, a database write that fails halfway through at 3am on a Sunday?

A demo shows you the expected path works. Production starts the moment users step off it.

I'm not going to pretend AI is bad at writing or testing code. It's getting genuinely good at both — give it a task with a clear right answer, and it checks its own work faster than a person can. What it can't do is decide whether your payment flow stays safe across weeks of real traffic, or whether every code path really blocks the wrong customer from someone else's data. Those aren't pass/fail tests — they're judgment calls about risk, made by someone who can still be asked, later, why they made that call.

A test suite can check known cases. It cannot certify every live outcome, or decide whether the risk that's left over is one you should accept. A named human still owns that decision.

In a small comparison I ran between public AI-built projects and projects that went through my own review process, on similar small-scale work — mostly small backend services, nothing exotic — almost none of the public projects automatically re-checked themselves before a change went live. Mine consistently did. The gap was in the part a founder can't see by clicking around: whether anything would have caught a change that quietly broke something.

Five areas where I still want a human decision

Not every change deserves this much attention. Fixing a typo isn't the same as touching money. These five places are where a mistake gets expensive or hard to take back. If you want the compressed, six-question version of the same discipline, Six things I check before trusting an AI-built app in production is the companion checklist.

1. Payments

Charged twice for the same order. A card charged with no confirmation email. Live keys switched on by accident while testing. None of that shows up in a single successful checkout — which is usually the only checkout anyone actually tests. Has anyone actually watched what happens when a payment fails halfway through? Stripe's own go-live checklist exists precisely because most teams haven't. If it goes wrong, customers get charged the wrong amount, or twice, and you hear about it from an angry email before you see it in any dashboard.

2. Logging in, and getting back in

Login answers who you are right now. Recovery answers something harder: can you prove it again after forgetting your password — without letting an attacker run the same trick in your place? I've seen recovery flows that felt finished because the reset email worked once in a demo, and nobody had tried resetting a stranger's account using only what's already public about them. NIST's Digital Identity Guidelines treat proving identity, session safety, and account recovery as one connected chain, not separate features — and a chain only holds at its weakest link. A well-built login screen with a weak recovery flow is a strong front door next to an unlocked side gate — the door was never the problem. Could someone get into another person's account today using only what's on their public profile? If you don't know, that's the answer.

3. What one customer can see of another's

Picture a support ticket that reads: "I can see someone else's invoice when I change a number in the address bar." That's not a hypothetical — it's the single most common thing I find. An app can correctly know who someone is and still let them see someone else's data, if the permission check is missing on even one request out of hundreds. The OWASP Authorization Cheat Sheet recommends checking permission on every single request, not just the obvious ones — because the obvious ones are rarely where it breaks. Has anyone deliberately tried to open a different customer's account and confirmed, in writing, that it failed? Keeping one customer's data away from another is the access side of a wider discipline — Why mikbry.com has no cookie banner covers the collection side: what a site gathers about visitors in the first place, and why less is safer than more.

4. Changing the database that holds real data

Could you actually undo this change tonight, or are you assuming you could? That's the honest question, and most teams have never tested the answer — a backup that has never been restored isn't a backup, it's a hope. Database changes are dangerous because the old state is often gone by the time anyone notices a problem — the app keeps writing new data over it. Google Cloud's migration guidance recommends testing the change first, backing up the target, and having a real plan to put things back — not just believing one exists. If it goes wrong, customer data is silently corrupted, and the last clean backup nobody's tested is hours old.

5. The keys that put code into production

A compromised laptop, or a stolen API key, pushes new code straight to customers. No second pair of eyes, no delay, no undo. That's what the fifth area is about. This one is easy to miss. It also decides whether the other four checks mean anything at all — if anyone with a laptop can push straight to production, none of the above matters. GitHub environments, for one, can require a second person's approval before a deploy runs, and hold back production keys until that happens. Similar controls exist everywhere else; the shape is the same. Could one person, or one agent, push a change straight to production alone, tonight, with nobody else watching?

Approval is not a magic shield

None of this is fixed by bolting an approval button onto a deploy. A rushed reviewer can wave through weak evidence just as easily as an agent can miss it — approval theatre is worse than no approval, because it puts someone's name on a decision nobody actually made without changing the actual risk underneath it.

The approval is worth something when it's backed by a short, honest account, not a hundred-page report nobody reads: what's actually changing, which cases were tested (including the ones meant to fail), who could be affected if it's wrong, what's still unknown, and how to undo it if it goes bad. That's a page, not a book. A named person reads it, understands what's still uncertain, and decides whether the risk that's left is one worth taking.

A Scan can catch known patterns. That's useful. But it can't decide whether the risk left over is acceptable — that's a judgment call, not a computation. The score is evidence, not permission to launch.

Before you flip the switch, five questions worth answering honestly:

• Has anyone tested what happens when a payment fails halfway through?
• Could a stranger get into someone else's account using only public information?
• Has anyone deliberately tried to open another customer's data and confirmed it failed?
• Have you actually tested a restore from backup recently, or just assumed one would work?
• Could one person, or one agent, push straight to production alone?

If any of those is "I don't know," that's not a launch decision. It's a guess wearing a launch decision's clothes.

Launch is only the beginning

Passing these checks does not mean the product is finished.

It means you have removed some of the risks that should not be discovered by your first real customers.

Then the real work starts.

Users will do things you never expected. Payments will fail in ways you did not test. A feature nobody cared about will suddenly matter. Another one you spent a week polishing may barely get used. Costs move, dependencies change, traffic grows unevenly, and every new feature creates another place where something can break.

That is normal.

A successful product is not a straight line from prototype to launch. It is a permanent loop: ship, observe, learn, fix, simplify, ship again.

The goal before launch is not to make the product perfect. It is to make sure the obvious risks are understood, the dangerous ones are controlled, and you have enough visibility to learn safely from what happens next.

Launch readiness gets you to the starting line in better shape.

What happens after that is product work.

Trust Report and Audit

These five areas are the security and access side of the discipline. If you want the money side — what actually makes a bill grow after launch — What does it cost to launch an AI-built app? covers it.

Three levels: a free automated Scan, a deeper automated Trust Report, then an Audit combining my tools with a human CTO review.

Before requesting a CTO review, you can start with the Launch Readiness Scan — it runs locally, so your source stays on your machine, and only the excerpts you explicitly approve are ever transmitted. It gives you an automated first read, but it doesn't see your production environment or business context.