gtmjosh | Connect Summary Metrics to the Records

Connect Summary Metrics to the Records

Every summary opens, and one of them gets read on purpose.

Drill-through is a first-class requirement, not a nice-to-have. Every aggregate in the brief names how to get from the number to the records underneath it, filtered the same way.

The reason is not curiosity. It is that this is the only way anybody ever finds out a number is wrong.

The field that disagrees with the notes

This is the payoff for the entire previous guide and it is the most useful thing in this one.

In a real analysis, a categorical reason field reported that competitors accounted for a small share of losses. Somebody read the free-text notes on those same records and found a competitor named several times more often than the field claimed.

The field was not broken. It was filled in by people, at a moment when categorising a loss was the least interesting part of their day, and the default was easier than the accurate answer. A field is only as good as the process that fills it, which the previous guide spent eight chapters on, and the only way to discover the gap is to go and read the records.

So the brief should name, for at least one metric, the record-level check that would falsify it. Not as a one-off audit. As a standing part of how the number is trusted.

GTM Lab

Saved locally

Metric Dictionary and Reporting Brief · 0 of 9 sections started

Saved locally to your browser.

Open the number

The field and the records disagree, and the only reason anybody knows is that somebody read them. Add a path from every number to its records, then write the check.

Fixture: field-versus-notes

The field and the record disagree

FIXTURE-FIELD-VS-NOTES

What the reason field reports: A small share
What the free-text notes say: Several times more often
Was the field broken?: No
Who fills it in: People, at the least interesting moment of their day
How anybody found out: By reading the records

What does drill-through actually buy you?

It is the only way anybody ever finds out a number is wrong.
A summary you cannot open is a summary you cannot check, and an unchecked number gets believed for exactly as long as it stays unchecked.It lets curious readers explore the detail behind a figure.
Self-service exploration is one of the most requested features in any reporting tool.It reduces the number of questions the analyst has to field.
Frees up real time, and readers get their answers faster.

A reason field reports one rate and the notes on those records tell a different story. What is the conclusion?

The field undercounts, so the metric carries a standing record-level check with a cadence rather than a one-off audit.
A field is only as good as the process that fills it. Making the check standing is what turns a discovery into a property of how the number is trusted.The team needs training on selecting the right reason.
Addresses the immediate cause, and the people concerned would genuinely rather be accurate.The metric is unreliable and should be dropped from the brief.
Removes a number nobody can trust, which is better than quoting one that is wrong.

Chapter 7 of 90 complete