gtmjosh
Improve the System
Chapters
  1. 01 · Establish a Review Rhythm
  2. 02 · Inspect Errors, Exceptions, and Overrides
  3. 03 · Compare Expected and Actual Outcomes
  4. 04 · Measure Adoption, Cost, Speed, and Quality
  5. 05 · Find Drift and Stale Definitions
  6. 06 · Test Changes Before Broad Release
  7. 07 · Roll Out and Roll Back Deliberately
  8. 08 · Update Owners, Documentation, and Training
  9. 09 · Capstone: Run the Quarterly Operating Review
Guide overview →

14 min · Interactive exercise

Chapter 6 of 90 complete

Test Changes Before Broad Release

A change that is checkable before it reaches anyone, against criteria written first.

A definition change silently alters every historical number computed from it. That is one of the more alarming things that can happen to a reporting system and it happens routinely, because changing a definition looks like editing a field.

So configuration changes get validation before they publish: what does this change, how many records move, and what does the historical series look like before and after. That is a preview, in exactly the sense Pillar 3 used the word, applied to a definition rather than to a bulk operation.

Test on a slice with no external dependency

The delivery sequence from a real technical guide doubles as a test method: map the decision and the source data, build a read-only slice, add review and exception paths, introduce automation, then monitor.

The read-only slice is the part worth extracting. Prove the change produces the right answer somewhere it cannot affect anything, before it can fail for reasons outside your control.

Write the acceptance criterion first

Before making the change, write what would count as it having worked, and what would count as it having failed. Both, in advance, in numbers where possible.

Written afterwards, the criterion is chosen to match whatever happened. That is not dishonesty, it is how people work, and writing it first is the only defence.

GTM Lab

Saved locally
Operating Review and Improvement Plan · 0 of 9 sections started

Saved locally to your browser.

Write down what would count as working

The change above met its acceptance criteria exactly, because they were written afterwards. Fix the order.

Fixture: criteria-written-after

A change that succeeded by definition

FIXTURE-CRITERIA-AFTER
Change shipped
Yes
Acceptance criteria written
After the results came in
Did it meet them?
Exactly
Was that dishonest?
No, it is how people work
The only defence
Writing them first
Resets every run.
How is a change checked before it reaches anyone?
When are the acceptance criteria written?
Chapter 6 of 90 complete