The 404 page and the 404 status are two different things

A 404 has two halves and they travel separately. There is the status line, which nobody sees and every machine reads, and there is the document that comes after it, which every person sees and no machine cares about. A site can ship one without the other, in either direction, and most of the broken setups I have inherited from other agencies do exactly that.

So the useful question about a missing page is never "does it show a 404 page". It is: what did the server put on the first line.

Reading the first line

One command, and it answers the whole thing:

curl -sSI https://example.com/a-url-that-cannot-exist-93f2

Use a URL you invented. A real dead URL might have a redirect rule on it already, which would tell you about that rule rather than about the site's default behaviour. The first line of the response is the answer. Anything other than HTTP/2 404 or HTTP/2 410 is worth ten minutes of your attention.

If you want the whole story in one shot, including where redirects go:

curl -sSIL -o /dev/null -w '%{http_code} %{url_effective}\n' https://example.com/a-url-that-cannot-exist-93f2

The four combinations, and three of them are wrong

404 status, styled page. Correct. The crawler drops the URL, the visitor gets something they can act on.

404 status, blank white page. Technically fine, commercially wasteful. Search engines behave correctly and the person leaves. This is the default state of a WordPress site whose theme has no 404.php, and of nearly every Nginx installation.

200 status, page saying "not found". This is the one that costs money, and it is common enough to have its own name in Google's documentation. The crawler is told the URL is a valid page and keeps it. Ten thousand deleted product URLs become ten thousand near-identical pages in the index, all saying the same three words.

301 or 302 to the homepage. The most popular wrong answer, because it feels generous. It is not. It tells a machine that the missing page became the homepage, and it tells a person that their link worked when it did not. Google treats a mass redirect to the root as a soft 404 anyway, so you get the index problem back with a worse user experience on top.

That last one deserves the argument in full, because somebody on the team always defends it. The reasoning goes: a visitor who lands on the homepage can find their way, a visitor who lands on a 404 leaves. In practice the visitor on the homepage does not know they were redirected. They think they clicked a link to a homepage. They do not search for the thing they wanted, because they do not know it is missing. A 404 page with a search box and three relevant links converts better than a silent redirect, and it does not corrupt the index. The exception is real and narrow: a URL that genuinely moved goes to its new location with a 301, one URL to one URL, and you should be able to name the pair.

Why this ends up broken in the first place

Nobody chooses a soft 404. It arrives sideways.

A page builder or a routing plugin catches the request before the CMS decides the URL does not exist, and renders a template. The template contains "Page not found", so a human reviewing the site sees the correct behaviour. Nothing in the interface shows a status code, so nobody notices.

Or a catch-all rule in .htaccess or in the front controller was added years ago for a migration and never removed. Or the CDN has a custom error page configured to return 200 because someone was fixing a display problem and that made it go away.

Or the site is a single-page application, and the server has been told to answer every unknown path with index.html, which is exactly right for the router and exactly wrong for the status code. The framework then renders a client-side "not found" view. The status is 200 and it can only ever be 200, because the decision happens in the browser, after the response.

Every one of those is invisible to visual review. All of them are visible to curl -I.

410 exists, and it is not a synonym

404 means the server does not have this. 410 means the server had it and it is gone on purpose. Both remove the URL from the index; 410 tends to do it faster, and it stops the crawler from politely coming back to check.

Use 410 when you deliberately deleted something and it is not coming back: a product line dropped, an event that happened, a client's old blog category. Use 404 for everything else, including typos and probing.

One warning from our own hosting, and it is the kind of thing you only learn by measuring. On o2switch, which runs LiteSpeed rather than Apache, the [G] and [R=410] flags in .htaccess are accepted without complaint and do nothing: the URL keeps answering 200. We only found it by requesting the URLs after deploying the rules. If you emit gone-status on shared hosting, verify each rule type with a real request rather than trusting that the file was read.

What an agency should actually watch

Two things, and neither is "does the 404 page look nice".

First, the default status, once per site, at handover. It takes the one command above and it is the single cheapest audit in this business. Write the result down next to the site in your inventory, because it will change silently the next time somebody installs a redirect plugin.

Second, and this is where a checker earns its place: a page that starts answering 404 without anyone touching the server. A permalink structure changed, a plugin that owned a custom post type was deactivated during an update, a client deleted a page they thought was a draft. The site is up. The homepage is fine. Uptime is one hundred percent. A specific URL that mattered is now gone, and the client finds out from a customer.

This is why a monitor that only stores up and down is not enough. The codes that are not outages are the ones that tell you something changed, which is the same argument as for rate limiting and for maintenance mode, and the shortlist of codes worth reading at all is in the nine that matter in production. A page that starts answering 401 or 403 instead of 404 is the same class of silent change, and those two are routinely treated as interchangeable when they answer opposite questions. If you check one URL per client site, check the one that pays: the pricing page, the contact form, the top landing page. Not the homepage, which is the last thing to break.