Vibe Coding for People Who Can’t Code: A five-part series. Part IV — Knowing whether it actually works

In Part I of the series, we talked about vibe coding basics, in Part II, about how to get setup, and in Part III, the build. Today in Part IV, we talk about making sure it works once it’s built.

The one thing to understand

This kind of software rarely crashes. It fails plausibly. Broken is cheap, because you see it. Plausible is what costs you a month. Four real examples, all mine:

• A scheduled job that reported success and did nothing at all — for a month. No error, no alert, looked completely healthy.
• A setting that existed, was spelled correctly, and was empty. Caught only when someone finally checked what was inside it instead of whether it was there.
• A quality bar set to a number that, against the actual material available, nothing could ever clear. Nobody had checked whether the number was even reachable.
• A detailed, confident, beautifully formatted report verifying code that, it turned out, couldn’t be read at all. The whole thing was invented, because the underlying files were unreachable and nothing said so. It looked completely real.

The habits that fall out

• Ask for the number, not the assurance. “It’s working” is not evidence.
• Read the actual file, not the summary of the file. The AI reports success optimistically — that’s the shape of the tool, not dishonesty.
• Check what’s in a setting, not just that it exists.
• Isolate one layer at a time instead of guessing across four. One session ate hours on a connection problem and was solved by a single command that bypassed everything except the one layer in question.
• Describe the symptom to your Project, not your theory of the fix. My worst debugging sessions were the ones where I handed the tool my diagnosis and it dutifully implemented it.

Why the written record is the whole game

This is where the spec and the decision log from Part II earn their keep — and I have an accidental experiment to prove it. Same builder, same month, same tools. Bedrock had both from early, and it’s the reason the project held together: two dozen separate conversations that shared no memory, plus a separate build tool on a different computer, and it still came out as one coherent product, because the truth lived in files instead of in anyone’s head. The other two projects added their logs about four weeks in. Both drifted in the meantime. Both ended up with entries carrying guessed dates, because the record didn’t exist when the decisions were made — and one of them still describes, in its own spec, a system that was quietly torn out and replaced without anyone writing down why.

A description of your project that nobody updates doesn’t go stale. It goes false — which is worse than nothing, because you and your AI will both keep trusting it. (Andrew ran a version of this with a small stack of tracking docs and landed on one or two too many; the useful lesson is that the principle is universal even where the exact number isn’t.)

The bug the AI can’t find for you

The relationship tool’s scoring worked. It produced numbers, it ranked people — and the ranking was wrong. A burst of email from ten years ago made an old colleague look like a current contact; internal staff meetings pushed my own employees above genuine outside relationships. Nothing surfaced either problem except a human reading the output and thinking that’s not right.

The AI will write flawless software that gives you the wrong answer, and it will not tell you. Only you know your own data well enough to catch it. That’s why “you are the user” was the first rule in Part I.

The security hole you won’t see coming

Everything so far has been about your thing working. This section is about your thing being safe — and it’s the one place where “it works when I try it” and “it’s actually fine” come apart, because a security hole doesn’t show up when you use the product normally. It only shows up when someone uses it in a way you didn’t intend.

Here’s the trap, in plain terms. When you ask for a rule — “only paying users can see this,” “nobody can edit someone else’s entry” — you tend to get it enforced on the screen: the button is hidden, the page won’t load, the thing looks locked. But the screen is the one part of your app that lives on the user’s computer, which means a determined person can get around it. The real lock has to sit on the server, where they can’t reach. The tool built exactly what I asked for. I never thought to ask whether the lock was on the screen or on the server — and for a while, on Bedrock, some of them were only on the screen.

I found out because my friend Brad had his own instance of Claude do a deep review of Bedrock’s code after it launched, and he sent me a document with a raft of problems. The problems were real, and all the same shape: doors that looked locked from the front but weren’t actually bolted from the back. Things like functions that cost money and checked nobody, and a paywall anyone could step around. All of it was fixable — three focused work sessions, one batch of problems at a time, checking each fix against the real code instead of trusting a summary that it was done.

This is the bill from Part I coming due. Vibe coding got a real product onto the internet and mostly working — and it also shipped exactly the kind of security gap a professional would have caught on sight, because the whole method skips the person whose job is to read the code and ask “but what if someone’s hostile?” I didn’t even know the questions to ask. That’s not a reason to avoid vibe coding. It’s a reason to schedule the security check the method can’t produce on its own — and to do it at the halfway point, not after launch. Mine were fixable in an afternoon. They were also live on the open internet for two weeks first.

And the relief valve: if what you’re building never leaves your own laptop or your own family — the grocery list, the personal tool — none of this applies. There’s no hostile stranger, so there’s nothing to bolt the back door against. This whole worry starts the moment real outsiders can reach your thing, and not one second before.

What the timeline actually looks like

Grocery: a single digit number of days from idea to daily family use. The relationship tool: about a week to something real, and about a month to something genuinely good — and the second three weeks were almost entirely data quality, not features, which is the most useful ratio in this whole series for setting your own expectations, and now I’m in constant refinement mode. Bedrock: 27 days to launch, and the launch was not a victory lap, just a launch of “minimum viable product,” from which now I’m building and tweaking all the time.

Everything so far has assumed you’re building for yourself, on your own machine, with your own data, where the only person you can hurt is you. Almost everyone reading this wants to do it at work. That’s a different game — and I learned how different by getting it wrong, which we’ll cover in Part V.