What AI-written code breaks, and why the metrics will not tell you

What we had to work with

Four codebases of our own, so no permission was needed and no anonymising either:

• Two that crossed the boundary mid-life — a booking platform with four human years behind it before a burst of AI work in March 2026, and an internal tool that ramped over two months.
• Two born with it — a SaaS product past 1,600 commits since May, and this website.

The two that crossed are the useful ones. Same codebase, same product, same people: only the way the code got written changed.

Four things we could measure

Rework: the one consistent signal

We counted commits that touch a file somebody touched within the previous seven days. It went, in both codebases that crossed the boundary:

• Booking platform — 60% → 69% (877 commits before, 319 after)
• Internal tool — 60% → 69% (151 before, 275 after)

Two independent transitions, same number, same direction. The two AI-native codebases sit at 69% and 75%.

And it does not mean what you probably assumed. Our own read, from the people who did the work, is that this is iteration rather than correction: it became cheap enough to try the next version that files got revisited sooner, not that the previous version was broken. The metric cannot tell those two apart. Nothing we found can.

That limitation is the article. If somebody sells you an AI productivity figure, ask which of the two their number is measuring, and watch what happens.

Commit size: no signal at all

We expected AI commits to be larger. In one codebase the median commit went from 12 lines to 76. In the other it went from 112 lines down to 71.

That is not a weak effect, it is opposite effects. We are reporting it because the temptation to leave it out was real, and because a result that survives in only one of two cases is not a result.

Commits titled “fix”: a naming convention wearing a lab coat

Fix-titled commits look damning at first — 5–9% in the older projects, 21–22% in the two AI-native ones. Then you look at each project separately and the effect disappears: the booking platform is at 6% both before and after, and the internal tool at 9% on both sides.

The split is not human against AI. It is projects that use conventional commit prefixes against projects that do not. The measurement was lexical, and lexical measurements measure writing habits.

Authorship: you frequently cannot tell

The booking platform's burst is 274 commits in a single month, three times any month in the previous four years. 4% of them carry an AI co-authorship trailer.

The work was AI-assisted. The history does not say so. Co-authorship trailers depend on which tool was used and how it was configured, so any study counting them — including the ones you will read — is counting configuration.

So what actually broke

Nothing above answers it, so this part comes from notes rather than from git: the defects found while building this website, each recorded when it was found.

• An og:image that rendered as [object Object], because a raw CMS image object reached a function expecting a resolved URL. It type-checked, it rendered, and it passed every test in the suite.
• A label recipe that drifted across four different letter-spacing values at eight call sites. Every one compiled.
• The same hairline rule written in two different class orders across five files. Both correct, neither canonical.
• Tailwind classes written outside the two scanned directories, which do not compile and produce no error. No lint rule catches it.
• Design tokens tree-shaken out of the stylesheet because a directive was missing one keyword, which removed a colour nothing had consumed yet.
• A CMS image with alt text but no file — an editor filling two fields in the order they appear on screen — which did not break a page. It broke the build.
• A WebGL scene that ran on the CPU on machines without a GPU and froze the page. Found by CI timing out, not by anybody reading it.
• A shader reading past the end of an array, which rendered correctly anyway.
• Orbit guides drawn on the wrong plane, which looked like perfectly good ellipses.
• A ring system attached to the wrong planet, which perspective made entirely plausible for an afternoon.
• Dates formatted in whatever time zone the build machine used, which was green locally and green in CI and gave different answers.
• A WebGL context released on unmount and handed back dead to the next mount, which left the canvas correctly sized and completely blank.

Twelve defects, and not one of them announced itself. No exception, no crash, no red test. Every single one compiled, and most of them rendered.

That is the actual pattern

An AI does not typically produce code that fails to run. It produces code that runs and is wrong in a way that has no runtime symptom — a plausible constant, a missed wiring step, a rule applied in eleven places out of twelve. Those are exactly the defects human review is worst at, because review looks for things that look wrong, and this all looks fine.

Which is why the response here has not been more review. It has been guards: tests that read the source and fail the build when a rule is broken, rather than tests that exercise behaviour.

Half the test files in this repository exist for that — 12 of 24. They check that a label recipe has one source, that a border rule is not retyped, that no page hands a raw image to the metadata builder, that a date formats in a fixed time zone. None of them would catch an interesting bug. All of them catch the boring ones that compile.

What to look out for

• Anything that has to be right in more than one place. A rule applied correctly eleven times and wrongly once is the signature failure. Make it a constant, then write a test that fails when somebody types it out by hand.
• Anything with no runtime symptom. Metadata, structured data, build configuration, accessibility attributes. Nothing goes red, so nothing tells you.
• Plausible constants. A tilt of 0.29 instead of 0.16, an index one off, a colour a shade out. These survive review because they look like decisions.
• Claims in the comments. More than one comment in this codebase confidently described behaviour the code did not have, and comments are the one artefact nothing verifies.

And do not go looking for the answer in your commit history. We tried four ways. It only ever told us how fast we were going.