gtmjosh
Report on What Matters
Chapters
  1. 01 · Ask a Business Question, Not for a Dashboard
  2. 02 · Choose the Right Population and Denominator
  3. 03 · Separate Volume, Speed, Quality, Cost, and Outcome
  4. 04 · Compare Trends and Useful Segments
  5. 05 · Use Leading and Lagging Indicators
  6. 06 · Show Sample Size, Missing Data, and Blind Spots
  7. 07 · Connect Summary Metrics to the Records
  8. 08 · Design for the Person Using the Report
  9. 09 · Capstone: Build a Reporting Brief
Guide overview →

12 min · Interactive exercise

Chapter 4 of 90 complete

Compare Trends and Useful Segments

Report the level and the direction, and cut only the segments somebody can act on.

A rate is a level. Whether it is rising or falling is a different fact, and a reader given only one of them will draw the wrong conclusion roughly half the time.

A segment at a low rate that has doubled over the period is a success story. The same segment at the same rate having halved is an emergency. The level is identical. Every action you would take is opposite.

So report both, together, always. This costs one column.

The trend can invert the headline

In a real analysis, one segment is presented with a low conversion rate and, in the same breath, the fact that the rate has been falling sharply across the period. Those two facts together support a conclusion neither supports alone: this is not a chronically weak segment, it is a deteriorating one, and whatever changed is recent and probably findable.

Without the trend, the reasonable reading is that the segment is simply poor and should be deprioritized, which is precisely the wrong response to something that used to work.

A blank segment is a finding

Carried from the previous guide and repeated here because it shows up first in a report: a large blank bucket in a grouping field is data quality surfacing, not a category. Show it, label it, do not analyze it as though it were a segment.

Which segments are worth cutting

Not all of them. A segment is worth reporting if somebody could act differently on it. Segments nobody can act on differently are decoration, and each one costs the reader attention that the actionable ones needed.

GTM Lab

Saved locally
Metric Dictionary and Reporting Brief · 0 of 9 sections started

Saved locally to your browser.

Level and direction, together

The same rate supports two opposite responses depending on a fact the report does not show. Add it, then choose segments somebody can act on.

Fixture: level-without-direction

The same rate, two opposite responses

FIXTURE-LEVEL-NO-DIRECTION
Segment rate
Low
If it has doubled over the period
A success story worth funding
If it has halved over the period
An emergency, and recent
What the report shows
The rate
Cost of showing both
One column
Resets every run.
A segment is at a low rate. What do you conclude?
Which segments belong in the brief?
Chapter 4 of 90 complete