gtmjosh
Improve the System
Chapters
  1. 01 · Establish a Review Rhythm
  2. 02 · Inspect Errors, Exceptions, and Overrides
  3. 03 · Compare Expected and Actual Outcomes
  4. 04 · Measure Adoption, Cost, Speed, and Quality
  5. 05 · Find Drift and Stale Definitions
  6. 06 · Test Changes Before Broad Release
  7. 07 · Roll Out and Roll Back Deliberately
  8. 08 · Update Owners, Documentation, and Training
  9. 09 · Capstone: Run the Quarterly Operating Review
Guide overview →

12 min · Interactive exercise

Chapter 5 of 90 complete

Find Drift and Stale Definitions

Scheduled checks that catch the failures where nothing fails.

The best evidence for this is a real document that says so about itself. A specification was re-read against the live implementation and two things it had claimed turned out to be wrong. Both had been believed, and acted on, for months.

Nothing failed. Nobody was careless. The implementation changed and the document did not, which is the normal behavior of documents.

So the check is: re-read the specification against the live behavior, on a schedule. Not review the document, which finds typos. Compare it to what the system actually does.

Staleness is per thing, and needs a sweep

Freshness is not one global setting. Different definitions, rules, and cached values go stale at different rates, so each gets its own threshold.

And a threshold on its own is a hope. What turns it into a guarantee is a scheduled sweep that finds everything past its threshold and produces a list. Without the sweep, the threshold is a number nobody evaluates.

Rules and definitions need review dates

Some things expire: a rule scoped to a fiscal period is only correct inside it. Most things do not expire and still need a review date, which is a freshness commitment about the definition itself, separate from the freshness of the data it describes.

A definition nobody has looked at in two years is not automatically wrong. It is automatically unverified, and the review list should say which.

Record the reason, so the decision can be revisited

A real system records that a particular vendor choice was made for cost, that quality was comparable, and that it can be reversed at any time. Read what that buys: when pricing moves, and it will, somebody can revisit the decision knowing exactly what it was based on.

A decision whose reason was recorded can be revisited when the reason changes. A decision whose reason was not recorded becomes permanent by default, and everybody assumes there was a good reason, which is how systems accumulate choices nobody would make again.

GTM Lab

Saved locally
Operating Review and Improvement Plan · 0 of 9 sections started

Saved locally to your browser.

Catch the quiet failures

Two claims had been believed and acted on for months and nothing had failed. Build the checks that would have caught them.

Fixture: document-that-drifted

Two claims that had been wrong for months

FIXTURE-DOCUMENT-DRIFTED
What was re-read
The specification, against live behavior
Claims found wrong
Two
How long they had been believed
Months
Anything that failed
Nothing
What changed
The implementation. The document did not.
Resets every run.
What makes a staleness threshold actually work?
A definition has not been touched in two years. What is it?
A vendor was chosen on price. What gets recorded?
Chapter 5 of 90 complete