Choose the Right Population and Denominator

This is where most reports are already wrong, and it happens before anybody opens a tool.

A rate is a fraction. Almost all the argument about a rate is actually an argument about its denominator, and almost nobody states the denominator, so the argument never resolves.

The denominator is a choice

Take a simple-sounding number: the share of qualified requests that reached a conversation.

Out of everything qualified this quarter? Including the ones qualified on the last day, which have had no chance? Out of everything qualified and now closed, which quietly excludes everything still working and biases the answer toward whatever resolved fastest? Out of everything the team actually worked, which excludes what nobody got to and therefore reports the effectiveness of effort rather than of the process?

Four denominators. Four different, correct numbers. They will differ by a lot, and two people quoting two of them will believe they are disagreeing about the business.

Say what is missing and which direction it bends

Here is the practice worth stealing, and it comes from a real analysis. A currency total in that analysis carries a footnote saying it counts only the records that had an amount entered, and that the true total is therefore higher.

Read what that does. It does not hide the incompleteness, it does not merely caveat it, and crucially it names the direction of the error.

That last part is what makes a partial number usable. “This is incomplete” tells a reader to distrust it, which is not actionable. “This is incomplete and the real figure is higher” tells them exactly how to use it: as a floor. A floor is a perfectly good thing to make a decision on.

Decompose a big bucket before concluding from it

When one category dominates a breakdown, split it before drawing any conclusion, because a large bucket is usually several different things and they point at different actions.

In that same analysis the largest loss category is broken out in a footnote into its constituent reasons, and the constituents demand completely different responses. Left whole, it would have supported one conclusion. Split, it supports a better one.