Test Changes Before Broad Release

A change that is checkable before it reaches anyone, against criteria written first.

See the situation

Start with the situation below. Next, we will work through an example together before you try the activity.

Elena · Reporting lead

The case

A change that succeeded by definition

Write acceptance and failure criteria before running a change against a bounded test set.

Change shipped: Yes
Acceptance criteria written: After the results came in
Did it meet them?: Exactly
What is the problem?: The criteria cannot independently test this result
One practical safeguard: Writing them first

Your task

Select the success criterion and freeze it. Then open the results and decide whether the change passes.

Need a little guidance?

You’re done when: The pass/fail decision uses criteria that existed before the outcome was visible.

Moving the goalposts after seeing results turns evaluation into storytelling.

Need a reminder? Reopen the worked example

Watch one example first

Before seeing results, this team requires overall performance not to fall and its protected segment to decline by no more than two percentage points. An overall gain cannot erase a segment failure.

Why this follows from the evidence

Freeze acceptance and failure criteria before evaluating the candidate. If the criteria need revision, start a new test rather than redefining success around revealed results.

GTM Lab

Saved locally

Operating Review and Improvement Plan · 0 of 9 sections started

Saved locally to your browser.

Case evidence

A change that succeeded by definition

Change shipped: Yes
Acceptance criteria written: After the results came in
Did it meet them?: Exactly
What is the problem?: The criteria cannot independently test this result
One practical safeguard: Writing them first

The change above met its acceptance criteria exactly, because they were written afterwards. Fix the order.

Check the case below to save your activity result and this chapter’s prepared example plan section. You can inspect that template in the saved plan above; no live records are changed.

Review the case

Freeze the guardrail before you open results

This team requires overall performance not to fall and its protected segment to decline by no more than two percentage points. Record that guardrail before opening the comparison. If you explore a different criterion, restart the test before changing it.

Acceptance and failure criterion

Restoring this working packet…

Optional: compare alternative reasoning

The case tool above saves your result and completes this activity. These comparisons are extra practice; they do not change your saved plan.

Work the case · 0 of 2 sections drafted

Write down what would count as working

Compare these alternatives one section at a time. Check a choice to see its consequence; the draft summary below keeps the proposed changes together.

Step 1 of 2

How is a change checked before it reaches anyone?

Against a read-only slice with no external dependency, and configuration validated before it publishes.Ship it to one small team and watch what happens.Review the proposed change with the people who own it.

This is a proposed draft until the chapter check runs. Selecting a choice alone does not complete the chapter.

4c · Proposed

Improvement plan draft

Improvement plan draft · current draftCompare the original state with the proposed change. This preview does not change live records.

Improvement plan draft · current draft
Item	Before	Proposed	Why
How is a change checked before it reaches anyone?	No saved change in this exercise draft	Not drafted yet	Choose a proposal to see what it would add to the draft.
When are the acceptance criteria written?	No saved change in this exercise draft	Not drafted yet	Choose a proposal to see what it would add to the draft.

How is a change checked before it reaches anyone?: BeforeNo saved change in this exercise draft; ProposedNot drafted yet; WhyChoose a proposal to see what it would add to the draft.
When are the acceptance criteria written?: BeforeNo saved change in this exercise draft; ProposedNot drafted yet; WhyChoose a proposal to see what it would add to the draft.

A choice appears here as a proposal, not as evidence that a live system changed.

Your place saves as you go.

Chapter 6 of 90 complete