My own site lied to me for three weeks

I spend my working life building systems that can tell a human when something has gone wrong.

My own site spent three weeks serving a version two commits behind, and I had no idea.

Here is what happened. On 6 August I pushed two changes to a subpage. Three deploys ran. All three failed. Each one recorded a build duration of zero milliseconds and the message "Page build failed." — no detail, no notification, nothing in my inbox.

Zero milliseconds means the build never even started, so it was not my code. Something upstream fell over.

And the site? The site was fine. HTTP 200, valid certificate, everything loading. Because the last successful build was still being served, exactly as it had been the day before. The failure had no visible surface at all. From the outside, from my own browser, from anyone else's — a healthy website.

I only found it because I was poking at something unrelated and checked the deploy status out of curiosity.

The uncomfortable parallel

This is the same failure mode I build gates against all day, arriving from a direction I did not defend.

An agent reports success. The tests pass. The diff looks clean. Everything on the surface says the work is done, and the only way to know otherwise is to check something the reporting party does not control — run the actual application, query the actual state, compare against something the optimiser cannot touch.

I had that instinct about agents. I did not have it about my own deployment pipeline, which had been quietly serving stale content for twenty-one days.

The fix, incidentally, was deleting work rather than adding it. The site is plain HTML, but every deploy was running it through a template engine it never needed — 165 seconds to publish half a megabyte. One empty file later, deploys are a straight file copy and take 19 seconds. Nine tenths of the time was spent on a step whose only real contribution was being able to fail.