Test Plausible Edge Cases

The engagement score goes live across every open opportunity. Within a week, renewals look like a disaster: almost every one of them is deep in the danger zone.

The renewals are fine. They are behaving exactly as renewals behave. A renewal opportunity is created well before its due date and sits in one stage for months, because renewal stages track a calendar rather than momentum. The time-in-stage penalty, designed for new business where a deal sitting still is bad news, punishes a renewal for being a renewal. Nobody tested the design against a renewal, because the design was built while thinking about new business, and new business was what got tested.

A population that behaves differently needs its own calculation

The instinct is to patch: add an exception, subtract the penalty for renewals, special-case it. The better answer is usually a separate calculation. When a population's normal behavior differs from the population the rules were designed for, exceptions accumulate until nobody can predict what the rules do, and each exception is a place a future change will forget to look.

GTM Lab gives renewals their own version: no time-in-stage penalty at all, meeting decay at half the rate because renewal cadence is quarterly rather than weekly, an urgency term that only engages as the renewal date approaches, and a relevance window outside which the renewal is not scored at all. That last one matters: a renewal eighteen months out is not healthy and not dead. It is not in play, and blank is the only honest answer.

The test for whether a segment needs its own calculation: would a record behaving perfectly normally score badly? If yes, the rules are measuring the wrong thing for that population.

Caps exist because something was uncapped once

The other correction from the same period is smaller and worth stating because it generalizes. The time-in-stage penalty was originally unbounded. It grew for as long as a deal sat in a stage, which meant a deal with an old stage date and a genuinely recent meeting could have the meeting wiped out entirely by the penalty. The most informative fact on the record was cancelled by the least.

Capping the penalty fixes it. The general form: any term that can grow without bound will eventually dominate every other term, and when it does, the score stops measuring what it was built to measure. Cap the terms that can run away, and choose the cap by asking what you never want this term to be able to override.

Build the test set from failures, not from happy paths

A test set assembled by imagining what might go wrong will contain the things you already thought of, which are the things you already handled. Build it from cases that actually broke something. A renewal, because it exposed a whole population being scored wrongly. A deal with an old stage date and a recent meeting, because it exposed the uncapped term. A record with a required activity date missing, because it exposed absence read as evidence. The enormous company with the test message, because it exposed rule ordering. A genuinely interested company far below the size threshold, because it exposed the missing outcome. A record where two sources disagree on the deciding field. A record where the decision times out and never returns.

Each carries its expected outcome, and the set runs against the whole design after every change. This is the difference between a decision system you can modify and one everybody is afraid to touch.

Carry this into your business

Take the last time an automated decision surprised somebody at your company and turn that record into a permanent test case. That single habit, applied consistently, produces a better test set in six months than any amount of upfront design.