A Successful Backup Is Only A Claim About Writing

A backup job that finishes with a success status has told you one thing: the software believes it handed data to the storage. It has not told you the data is still there, still readable, or still restorable. Those are three different claims, and the only person who ever needs the second and third is someone having the worst day of their quarter. If you treat the green tick as proof of recoverability, you have confused the receipt with the goods.

Writing and keeping are different jobs

Most backup tooling splits its work into two phases that look like one. First it copies data to a repository. Later, often on a schedule of its own, it goes back and checks the stored files: do the blocks still match what was written, is the chain of increments intact, can the restore points actually be assembled. The first phase is quick, visible and cheerful. The second is quieter, and it is where the bad news lives.

That separation is why you can see a job report success while the health check on some of its files fails. Both statements are true at once. The write succeeded when it happened. The storage then did something storage does, whether that was a degrading disk, a controller fault, a filesystem hiccup or a volume quietly running out of room, and the stored copy stopped matching the original. Nothing was wrong at write time, so nothing was reported wrong at write time.

The uncomfortable part is that the dashboards most people watch are built around the first phase. A row of green jobs feels like safety. A single line saying some files failed verification is easy to read as a footnote when the headline says success. I think we sometimes confuse the status of the process with the state of the asset.

What the health check is really telling you

When a verification step flags damaged files, it is rarely telling you about one file. It is telling you about the thing underneath. Storage that corrupts one backup file is storage you can no longer vouch for, and every restore point that lives on it is now an unknown rather than an asset. The damaged files are the ones you happened to find. The honest reading is that your confidence in the repository has dropped, and you should act on that before you learn which files were the ones that mattered.

That changes the urgency. A failed job is loud and gets fixed the same day. A job that succeeds against a repository you no longer trust is silent, and it keeps producing new restore points that look healthy and may not be. Each night you carry on, you add more data to a place you have just been told is unreliable. The right immediate move is usually to stop treating that repository as the only copy, get a second, independent copy of the important data somewhere else, and then fix or replace the storage. In that order.

The decision is the easy part

Here is the commercial reality that makes this harder than it sounds. Replacement hardware has often already been approved, or even bought, by the time the problem shows up. Everyone agrees it should be fitted. And then it sits in the gap between a decision and a delivery: waiting on a supplier, waiting on someone's calendar, waiting on a ticket to be picked up. Everyone is acting reasonably, and the exposure continues anyway.

I have come to prefer treating that gap as a risk with an owner and an end date rather than a to-do item. Someone should be able to say who is chasing it, what the interim protection is while the gap exists, and what day we will check that the new thing is actually receiving data. Buying the replacement is a decision. Having it hold a verified copy is an outcome. The two are further apart than the purchase order suggests.

It also helps to be honest about cost. A second copy, an offsite or cloud tier, or a periodic restore test all cost money and time, and a small business can reasonably ask whether that is justified for data nobody has needed in years. The answer depends on what losing it costs. For anything touching patient records, financial data or customers' personal information, the cost of not being able to restore is not a line item. It is the business, and sometimes it is other people's trust in you. Where the data is entrusted to you, recoverability is part of the duty of care, not an optional extra.

Make recovery something you have done

The only reliable evidence that a backup works is a restore. Not a hope that one would work, not a report that says it should, but an actual recovery of something real into somewhere safe, with someone checking that what comes back is usable. It does not need to be dramatic. Restore a file. Restore a database to a scratch environment and run a query that only real data could answer. Occasionally restore a whole system and see how long it truly takes, because the time you assumed and the time you measure are usually strangers.

Do it on a schedule, and write down what happened. A practised restore has a second benefit that has nothing to do with the data: the people involved know where the credentials are, what the steps are and where the documentation is wrong. Discovering that during a drill is a mild inconvenience. Discovering it during an outage is a very long afternoon.

None of this requires distrust of the tools. The tools are doing what they were designed to do, and many of them do it well, including surfacing the health check failure honestly. The point is to read the whole report rather than the headline, to treat verification as the number that matters, and to keep the habit of proving recovery rather than assuming it. A backup is a promise to a future version of you. It is worth finding out, while that person is still calm, whether it can be kept.