How mature does a CRM context layer need to be?

Locate yourself on a ladder rather than aiming for the top: raw schema, documented schema, authority and provenance, business rules, hard enforcement, regression testing, observability. Most teams get real value at stage 3 or 4 and never need the last two. The stage that changes the risk profile most is hard enforcement, because everything below it is advice the model can ignore.

What happens when a CRM field changes after you've written a rule about it?

Nothing, which is the problem — the rule keeps loading and keeps being wrong. Schema drift needs a scheduled diff between the live schema and the fields your rules reference, producing a review queue: fields that vanished, changed type, gained picklist values, or appeared with no guidance. A rule pointing at a field that no longer exists should fail loudly rather than silently.

What metrics matter for a CRM AI agent?

Rule hit rate, refusal rate, tool-boundary rejection rate, stale-rule rate, regression pass rate, cache hit rate, cost per question, and correction rate. The two people ignore are tool-boundary rejection rate — a flat zero usually means the boundary isn't wired up, not that nobody asks out of scope — and correction rate, which is the only one that measures whether users trust the answers.

Operating a CRM context layer

Q: Should context rules expire?

Some should. A fiscal-period rule is only correct inside its window, so it gets expiresOn. Everything else gets reviewAfter, which is a freshness SLA for the rule itself, separate from the freshness of the CRM field it describes. A rule nobody has looked at in two years is not automatically wrong, but it is automatically unverified.

Most of what's written about this stops at "and then it works." This is the part after that: knowing where you are, keeping rules honest as the CRM moves underneath them, and being able to tell whether the thing is healthy.

Where you actually are

You don't need all of this, and the top of the ladder isn't the goal. Find your rung and decide whether the next one is worth it.

Stage	You have	You still get burned by
0. Raw schema	The agent can see field names and types	Everything. Field labels read as meaning
1. Documented schema	Help text pulled in as baseline context	Stale descriptions, fields with no help text
2. Authority and provenance	Which fields to trust, who writes them, how fresh	Definitions, exclusions, junk records in every count
3. Business rules	Fiscal calendar, exclusions, attribution, anti-patterns	The agent reaching things it shouldn't
4. Hard enforcement	Object and field allowlists checked in code	Regressions you find out about from a user
5. Regression testing	Golden questions on both sides of every change	Not knowing why an answer changed
6. Observability	Run logs, metrics, attributable failures	Not much. This is where it becomes a system

Most teams get real value at 3 and can stop there for a long time. Stage 4 is the one that changes the risk profile, because everything below it is advice a model can decline to take. If you're going to jump a rung, jump that one.

Stages 5 and 6 earn their keep once more than one person is editing rules, or once the answers start feeding decisions nobody double-checks.

The rule lifecycle

A rule isn't a document, it's a small piece of production configuration. Give it states and make the transitions deliberate.

lifecycle

draft ──approve──► approved ──activate──► active ──supersede──► deprecated
  │                                          │                       │
  └──────────── discard ◄────────────────────┘                       │
                                                    replacedBy: R-xxx ┘

draft — written, not loaded by anything. Where auto-generated candidates land.

approved — someone who wasn't the author has read it. Still not loaded, which sounds pedantic and is the step that catches a rule saying the opposite of what its author meant.

active — loaded by agents. The only state that reaches production.

deprecated — retired without being deleted, with replacedBy pointing at the successor. Deleting a rule destroys the ability to explain an answer from six months ago.

Each rule should carry who owns it, who reviewed it, when it was created and last reviewed, why it's true, the production failure that caused it, the golden questions covering it, and its successor if any. The schema has a field for each.

The one people leave out is originFailure. A year in, it's the difference between a library you can prune and a pile of assertions nobody dares touch.

Expiry and review

Two different clocks, and conflating them is why stale rules survive.

expiresOn is for rules that stop being correct. A fiscal-period definition, a territory rule that only applies to this year's segmentation, a temporary exclusion during a data migration. Past that date the rule shouldn't load at all.

reviewAfter is a freshness SLA for the rule itself, separate from the freshness of the CRM field it describes. A rule nobody has looked at in two years isn't automatically wrong. It is automatically unverified, and that distinction should be visible.

A rule past reviewAfter should still load, since pulling it would be worse, but the payload should mark it, and the answer should be able to say the guidance is overdue for review. That's the known_stale resolution state.

The queue this produces is the useful artifact: twelve rules haven't been reviewed in a year, and four of them govern the objects people ask about most.

When the schema moves underneath you

The failure here is quiet: a field changes, the rule about it keeps loading, and nothing errors.

Five drift cases, and they need different handling:

Drift	What happens without a check	What should happen
Field deleted	Rule references a field that isn't there. Query errors, or worse, silently drops it	Rule fails loudly; goes to review queue
Field renamed	Same as deleted, plus a lookalike field may now answer instead	Detected by API-name diff; rule flagged
Type changed	A rule's thresholds stop meaning anything. Text-to-picklist is the classic	Flagged; interpretation needs rewriting
Picklist value added	Rule enumerates values and silently omits the new one	Flagged; `valueMap` needs the new entry
New field appears	No guidance at all, so the agent infers meaning from the name	Enters the draft queue as unreviewed

The mechanism is a scheduled diff between the live schema and the fields your rules reference. Not at query time. That's the Properties API anti-pattern. Run it nightly, produce a review queue, and let a human decide.

The last row matters more than it looks. A new field with no guidance is the default state of every field somebody adds, and the honest position is that a field with no approved guidance isn't authoritative. That's a rule you write once, globally.

What to watch

You're already logging guidance version, rule IDs loaded, planned versus actual objects and fields, tool calls, answer or refusal, latency, and cache state. These are the metrics worth deriving from that.

Metric	Reading it
Rule hit rate	Which rules ever load. Rules that never fire are either dead or badly titled
Refusal rate	Should be low but non-zero. Zero means it's answering things it shouldn't
Tool-boundary rejection rate	Off-allowlist requests caught in code
Stale-rule rate	Share of loaded rules past `reviewAfter`
Regression pass rate	Golden-set results over time, per release
Cache hit rate	Below expectations means something volatile is in your stable prefix
Cost per question	Tracked over time. The payload grows without anyone deciding it should
Correction rate	How often a human tells the agent it's wrong

Two get ignored and shouldn't.

Tool-boundary rejection rate at a flat zero almost never means users are perfectly well-behaved. It usually means the boundary isn't wired up. If you've never seen a rejection logged, go ask for something out of scope and check it appears.

Correction rate is the only metric here that measures trust. Everything else says whether the system did what it was told. This one says whether the people using it believe the output. A rising correction rate with a green golden set means your test questions have stopped resembling the real ones.

Rule hit rate is the quiet diagnostic. A rule that never loads is either genuinely dead or, more often, titled as a category so the agent never had a reason to fetch it. That's usually a one-line fix once you can see it.

A reasonable cadence

On every rule change: schema validation, targeted golden questions, deploy rule text separately from retrieval logic.
On every release: broad and adversarial golden tiers, regression diff against the previous guidance version.
Nightly: schema drift diff into the review queue.
Monthly: rules past reviewAfter, rules with zero hits, correction-rate trend.
Whenever something goes wrong: trace it, write the rule, add the golden question, record originFailure.

That last line is the whole loop. Everything else is scaffolding that keeps it working once the library is bigger than one person's memory.

How to build an AI context layer for your CRM — the concept and full build
The rule schema — every lifecycle and expiry field
Golden-question testing — the regression half of this
One CRM question, traced end to end — what a single run log looks like