Define the Reporting Population

Every number is a number about something, and the something is almost never as obvious as the label suggests.

“Conversion rate” over which records? Created when? Including the ones still in flight? Including the ones that were disqualified two days in? Including the test records somebody made last Tuesday?

Four defensible answers to those four questions produce sixteen different conversion rates, all of which are correct, and any two of which will be quoted in the same meeting by people who think they are disagreeing about the business.

Three things to write

The population. Which records, by which rule, over which window. Written so somebody else could reproduce the set exactly.

The counting rule. What specifically counts as the event you are measuring, including at least one edge case that deliberately earns no credit. A good example from a real analysis: a meeting counts only when it happened after the record was created, so an event predating the record gets no credit even though it genuinely happened. That is a defensible choice, and its value is that it is written down, so the number means the same thing every time it is produced.

The exclusions. Not the logical complement, which is useless. The specific cases somebody will ask about: test records, internal submissions, records that were merged away, records belonging to a population with different semantics.

The population that behaves differently

If a population moves on a calendar and you measure it with a metric designed for momentum, that population will read as failing, permanently, for behaving correctly. And because it reads as failing consistently rather than randomly, nobody will suspect the metric. They will suspect the population.

The fix is to give it its own treatment and say so, or to exclude it and say so. Both are fine. What is not fine is including it silently, which is the default.