Why does my CRM AI give confident wrong answers?

Almost always because it can read your data but not the rules around your data. It sees a field name and a value, and it cannot see that the field was abandoned last year, that your fiscal year starts in February, or that a third of those records are CPQ artifacts. So it fills the gap with a reasonable-sounding assumption, and reasonable-sounding assumptions produce answers that look right.

How do I tell whether the model is wrong or the data is wrong?

Ask it to state the exact query it ran, the date range it used, and the exclusions it applied, then run that query yourself. If your query returns the same number, the model reasoned correctly from bad or unexplained inputs, which is a context problem. If it returns a different number, the model got the plan wrong, which is a prompt or retrieval problem. These have completely different fixes.

Why do two people get different answers to the same question?

Because the connector is probably running as the asking user, so each query returns only the records that user can see. That's correct behavior and the model has no idea it's looking at a partial org. The fix isn't to run as an admin; it's to make the agent state whose permissions the query ran under, and never conclude that no records exist from an empty result on a sharing-restricted object.

The field is blank. Does that mean there's no value?

Not necessarily. If field-level security hides a field from the running user, the platform does not raise an error — it returns null, which is indistinguishable from a genuine blank. This is the nastiest failure on the list because there is no signal at all. Use WITH SECURITY_ENFORCED in Salesforce so the gap surfaces as an exception instead of an empty value.

Debugging wrong CRM AI answers

Something asked your CRM AI a question and got an answer that was wrong. Not obviously wrong, or you wouldn't be here. Wrong in the way where the number looked plausible, the prose was confident, and somebody almost put it in a board deck.

This page is the diagnostic. Find the symptom, get the cause, write the rule that stops it recurring.

The short version of why this happens: the model can read your data but not the rules around your data. It sees account_health = Green and a field name that sounds official. It cannot see the Slack thread where everyone agreed to stop maintaining that field. So it fills the gap with a reasonable assumption, and reasonable assumptions produce answers that look right.

First: is it wrong, or answering a different question?

Before hunting a cause, separate two failures that need completely different fixes.

Ask the agent to state the exact query it ran, the date range it used, and the exclusions it applied. Then run that query yourself.

Your query returns the same number. The model reasoned correctly from inputs nobody explained to it. That's a context problem, and the rest of this page applies.
Your query returns a different number. The model built the wrong plan. That's a prompt or retrieval problem. Check what guidance actually got loaded before you go writing new rules.

If the agent can't tell you what query it ran, fix that first. An agent that can't show its work can't be debugged, and disclosure should be an instruction, not a hope.

The failure-mode index

Twenty failures, the cause of each, and the rule category that prevents it. Symptoms first, because that's what you have.

Symptom	What's actually happening	Category	Fix
Quarter numbers are close but never match the finance report	Calendar quarters used instead of your fiscal calendar	`temporal`	One always-loaded rule with your fiscal boundaries and how the year is named
Year-over-year comparisons are subtly off	Same fiscal drift, compounded across two periods	`temporal`	Same rule; require the agent to state the exact date range
Pipeline total is meaningless but looks fine	Mixed currencies summed as one number	`currency`	Convert to reporting currency, or state which single currency the total covers
Counts are inflated and nobody knows why	Test, CPQ, and internal records counted as real	`exclusion`	Always-loaded exclusion carrying the literal filter predicate
Deal credit goes to the wrong person, consistently	Two roles collapsed onto one owner field	`attribution`	Name both roles and the field each comes from
A breakdown looks valid and every number is wrong	Grouped by a multi-select; buckets are stored combinations	`anti_pattern`	Split values before counting. Never group the raw field
Grouping fails, mis-groups, or gets worked around	Formula or calculated field used in an aggregate	`anti_pattern`	Aggregate the underlying stored field instead
A conclusion is built on a field nobody uses anymore	Dead field read as live because the name sounds official	`authority`	Deprecation prefix plus one global rule covering the whole tail
Stale status trusted over a maintained score	Field label sounded more important than the real signal	`authority`	Mark the stale field non-authoritative and name its replacement
A generated summary gets cited as verified fact	AI-written field treated as a source	`authority`	`authoritative: false` on every generated field, with instructions to verify
"Next step: send proposal" resurfaces later as "proposal sent"	Generator promoted intent to completed fact	`authority`	Instruct that statements about completed actions are least reliable
Two people get different totals for the same question	Query runs as the asking user; each sees a partial org	`security`	State whose permissions ran. Never conclude "none exist" from an empty result
A field is blank and the agent reasons from the blank	FLS suppressed the field; null is indistinguishable from empty	`security`	`WITH SECURITY_ENFORCED` so it errors instead of returning null
The agent answered from an object you didn't expect	Couldn't reach the right object, substituted a similar name	`security`	Enforce the object whitelist in code, not in the prompt
Funnel counts drop records that clearly exist	Population moved objects mid-lifecycle; query lives on one	`cross_object_dependency`	Ship the join, not just the observation that conversion happens
A rule applies on one object but not the related one	Plan grew a second object after guidance loaded	`cross_object_dependency`	Fetch rules mid-plan, before querying any newly introduced object
The same question returns a different shape each time	No canonical structure for a recurring answer	`output_shape`	Encode the standard view as numbered rules plus taxonomy
An update "worked" but the value reverted	Field is force-set by automation on every save	`write_policy`	`writePolicy: force_set_by_automation`; don't recommend writing it
The AI overwrote a rep's qualification score	Human-owned field treated as agent-writable	`write_policy`	`authoritative: human_owned` — compare and report, never overwrite
Answers changed after a "small" guidance edit	Two active rules now contradict; retrieval order decides	`definition`	Set `priority` explicitly and validate for conflicts

The four that leave no trace

Most of the table gives you something to notice. These four don't, which is why they deserve separate attention.

A permission-suppressed null. Field-level security hides a field and the platform returns null rather than an error. There is no signal. The model reads "this account has no renewal date" and reasons from there. My favorite gotcha on the whole list, and the reason WITH SECURITY_ENFORCED is worth the noise.

A missing exclusion. An agent that doesn't know an exclusion exists cannot decide to fetch it. There's no gap for it to notice. It runs the query, gets a number full of test records, and reports it confidently. This is why exclusions always load and are never lazy-fetched.

A silently reverted write. The API returns 200. The value reverts minutes later when automation fires. Everyone downstream believes the recommendation was applied. Worse than a refusal, because a refusal is visible.

A field-history entry that never existed. Debugging note that costs people a day: the absence of a field-history row does not prove a field was never written. An async path can write and revert inside the same transaction window and leave no trace. Don't use "no history row" as evidence of anything.

Working one through

Take the most common report: "the agent said we created 214 opportunities last quarter, and the real number is 186."

Get the query. Ask what date range and filters it used. Say it reports 1 April to 30 June and no exclusions.
Two causes are already visible. Your fiscal Q1 doesn't run April to June, and it applied no exclusions at all.
Confirm before writing rules. Run the count for your real fiscal range with your standard exclusions. If you get 186, both causes are confirmed.
Write the rules. One temporal rule with your fiscal boundaries, one exclusion rule carrying the literal predicate. Both alwaysLoad: true.
Record why they exist. Set originFailure on each to this report. In six months, that's the difference between a reviewable library and a pile of assertions.
Add a golden question. "How many opportunities did we create last quarter?" with pass criteria: states the exact date range, names the exclusions applied, and excludes system-generated records.

Step 6 is the one people skip, and it's the one that makes the fix permanent. The rule stops this failure today. The golden question stops it coming back when somebody changes retrieval logic four months from now.

When the answer is right but you can't tell

Sometimes nothing is wrong and you have no way to know that, which is its own problem.

Make the agent show its work by instruction, not by request. Every answer should state the rules it applied, the exclusions it applied, the exact date range, and whose permissions the query ran under. That turns an unverifiable paragraph into something a skeptical person can check in two minutes.

You'll catch more bugs from that habit than from any amount of prompt tuning.

Stopping the next one

Debugging one answer is worth doing. Building the loop is worth more:

Log the guidance version and rule IDs with every run, alongside the objects and fields the plan actually queried. You cannot write a corrective rule for a failure you can't reconstruct.
Turn every fix into a golden question, so the suite grows the same way the library does.
Deploy rule changes separately from retrieval changes. Ship both at once and you won't know which caused the next regression.

That's the whole mechanism: mistake, trace, rule, golden question, protected forever.

How to build an AI context layer for your CRM — the concept and full build
The rule schema — every category in the index above, defined
The minimum viable context layer — the seven rules that prevent most of this
MCP vs. CRM context layer — why a working connection doesn't fix any of it