Old Data Never Read The Release Notes

Adding a required field to stored data is a change to every record you have ever written, not just the ones you will write next. The new code is tested against new data, the tests pass, and the build is clean. Then it meets a record created last month that has never heard of the field, and the whole read path falls over. The fix is not clever. Before you ship, decide what the absence of the field means, and make the code say so.

I was reminded of this on a small side project this week. A feature went out that let a host define a custom set of cards for a session. The new property was modelled as required, which is the natural thing to do when every new session will have one. The feature's own tests passed. The problem was everything stored before the feature existed. Those documents had no such element, and the data layer, quite reasonably, refused to build an object out of a document missing something the model said was mandatory. It did not fail once, politely, on the page that used custom cards. It failed on every read of an older session, including in a piece of middleware that has nothing to do with cards at all. The evidence turned up in production logs, which is a slightly humbling place to learn about your own assumption.

Why this slips through

It slips through because the author's mental picture of the data is the picture after the change. When you build a feature you create fresh records, you click through them, and every one of them carries the new field. Nothing in that loop ever produces a record from before. Your test data is born under the new rules, so it cannot disagree with them.

Document stores make this easier to miss, not harder. A relational migration forces the question at deploy time: you add a column, and the database makes you say what existing rows get. A document store lets you add a property to your model with no ceremony at all, which feels like freedom right up to the moment the old documents are read back through the new model. The flexibility has not removed the migration. It has moved the migration to runtime and left it unwritten.

There was also a gap in how I verified the change. The tests that exercise the database were not something I could run in that environment, so the paths that read real stored documents went untested before merge. I would not call that a scandal. It is the ordinary way a risk gets through: the check that would have caught it was the one that was inconvenient. What I take from it is that when the honest answer to "did you run the tests that touch real data" is no, that is the moment to stop and ask what real data looks like.

Decide what absence means

The repair was small, and the smallness is the useful part. The property became optional, and a missing value was given a meaning: no custom cards, use the standard set. The code that already fell back to the standard set when the list was empty needed no change. Two places that accessed the property directly were made safe against it being missing. That was the whole fix, and it shipped within minutes of the problem being noticed.

The design decision hiding in that repair is the one I would make up front next time. For any field you add to something already stored, ask what a record without it should mean. Often the answer is obvious and cheap, and you are really just writing it down: absent means the default. Sometimes the answer is that absence means something meaningfully different from empty, and then you have found a genuine modelling question while it is still inexpensive to answer. Occasionally the answer is that old records genuinely need backfilling, and then you have a migration to write, test and schedule, and you have learned that before it became an incident.

Required is a strong claim. It says no valid record exists without this value. That is true of new records and false of old ones, so for any data that outlives a release the honest model is usually optional with a defined default, tightened later if and when you have actually migrated everything. Making a field required is something you earn by backfilling, not something you declare in the commit that introduces it.

What this costs, and what it does not

I do not want to turn this into a rule that everything must be nullable forever. That way lies a codebase where every value might be missing and nobody can tell which absences are real. Optional fields carry their own carrying cost: every consumer has to handle the missing case, and the missing case has to keep meaning the same thing. The aim is to make the decision deliberately, in one place, rather than leave each reader to discover it independently.

For a hobby tool with modest data, a few minutes of broken reads is an annoyance. The same oversight in something that holds people's records, or runs a clinical or financial workflow, is not an annoyance at all, and it is the sort of failure that arrives on a day nobody chose. The habit that scales is cheap. Keep a handful of genuinely old records around, or a snapshot of production shape, and read them through every change that touches the model. If you cannot do that, say so in the review and treat the change as less proven than the green tick suggests.

New records are the ones you are paying attention to. The old ones are the ones with customers attached, and they were never told the rules had changed.