Past the Last Row: Britain's Quietest Database Is Still an .xlsx

The email arrives at 4:52pm on a Friday with an attachment called Housing_Register_FINAL_v7(2).xlsx. Somewhere between the sixth and seventh revision, a few hundred households stopped being in it. Nobody was told. The file opened fine, the totals looked fine, and the dashboard downstream looked fine too, because the dashboard was built from the same totals.

This is not a rare event. It is arguably the most common way things break in British public life: not with a crash, not with a red banner, but with a number that is simply, quietly, a bit short.

The number in the headline is a lie, which is the point

Everyone quotes 16,384. It's the number people reach for when they want to sound knowing about spreadsheet limits, and it's wrong in a specific and delightful way: 16,384 is the column ceiling in a modern .xlsx file (column XFD, if you've ever scrolled right and found yourself in the alphabet's afterlife). The row ceiling is 1,048,576. The old .xls format managed 65,536.

So even the folk wisdom about spreadsheet failure is stored in the wrong axis. That feels about right for a country whose relationship with the spreadsheet is one of deep, unexamined trust.

The Excel binary is a magnificent piece of software and a genuinely poor database. It has no schema, no referential integrity, no audit trail worth the name, and no concept of a required field. What it does have is a grid, and a grid is the most persuasive interface ever designed, because it looks like the answer. You open it, you see rows, you think: data. You do not think: untested, unversioned, unowned.

Two cases, one mechanism

You already know the loud one. In autumn 2020, Public Health England's daily COVID case file was assembled through an .xls template whose row limit was 65,536. Once the count passed it, additional records were dropped — not flagged, not bounced, just gone. Around 15,841 cases went missing from the published totals over roughly a week before anyone noticed. The downstream consequences were dull and serious: contact tracing delayed, decisions made on a picture that was short of people.

That is the entire failure in one sentence. A format limit met a growth curve, and the system behaved exactly as designed by doing nothing at all.

The quieter one is the Post Office, where the Horizon Inquiry spent years demonstrating what happens when a system of record and a human's reconciliation spreadsheet disagree — and the human is assumed to be lying. I'm not going to relitigate Horizon here; the inquiry's final report did that at length. But the shape of it is worth holding onto. Spreadsheets are not neutral artefacts. They carry the authority of whoever made them and whatever they wanted to be true.

Why nothing warns you

Software that fails loudly gets fixed. Software that fails silently gets trusted.

A truncated import doesn't throw. A VLOOKUP that finds no match doesn't throw either — it returns #N/A, which someone has already replaced with IFERROR(...,"") because the errors were untidy. A merged cell silently reassigns a value to the top-left corner of the merge. A CSV export from a system that's been quietly paginating at 10,000 records will hand you 10,000 records and a cheerful 200 OK. Google Sheets will refuse to grow past 10 million cells, which sounds generous until someone starts pasting timesheets into it.

None of these are bugs in the ordinary sense. They're all successful operations producing incomplete results, and they all leave a human looking at a total that is plausible. Plausible is the enemy. Implausible numbers get challenged; plausible ones get forwarded.

Rho is for row, and the row is a decision

The most useful thing about a row is that it is a boundary. Something above it is inside the system; something below it is not. Every spreadsheet-shaped process in the country is making that decision thousands of times a day and telling almost no one where the line fell.

So the practical question is not "are we still using spreadsheets?" — of course we are, and so are you, and so is your bank. The question is: where does this thing stop counting, and would we know?

A field kit for inherited spreadsheets

If you've just picked up a process, a service, or a supplier relationship and there's an .xlsx at the bottom of it, run these before you trust a single total.

• Count the rows against the source. Not the sheet's idea of its row count — Ctrl+End and the source system's own count. If they disagree by a round number, you've found a limit.
• Look for the round numbers. 65,536, 1,048,576, 10,000, 50,000, 1,000,000. Distressingly often, a figure that ends in 000 is an API default nobody tuned.
• Open every hidden sheet and every tab, including the ones named after people. Half the logic in British institutional life lives on a tab called Dave temp.
• Check for manual steps described as "just". "I just paste the new export in every Monday" is a pipeline with a human as the error handler.
• Search for IFERROR wrapping lookups. Each one is a place where the sheet has been taught to lie politely.
• Find the boundary and test it. If the sheet handles 5,000 records, feed it 5,001. You want to see it complain. If it doesn't complain, that's your finding.
• Ask what breaks if the file is open on someone's laptop. If the answer is "everything", you have a single point of failure with a person attached.
• Check who owns the macro. If nobody knows, the answer is: nobody, and it will still be running in 2031.
• Look for a date typed into a formula. Hardcoded cutoffs are how last year's policy quietly becomes this year's answer.

The goal isn't to shame anyone into abandoning the spreadsheet. It's excellent at what it's actually for: thinking in a grid, quickly, with a colleague on a call. The goal is to stop letting it be the last word on how many people are on a waiting list, how many tests were run, or how many invoices got paid.

Somewhere in Britain this evening, a file will accept its final row and go quiet. The least we can do is notice which row that is.