CRM AI context layer anti-patterns

Everything here is a real way people build this, usually because it looked reasonable at the time. Ordered roughly by how often I see it.

1. Dumping the CRM schema into the prompt

What it looks like. Pull every object and field from the metadata API, serialize it, paste it in. Now the model "knows the CRM."

Why it fails. A schema is names and types. The problem is meaning. The dump tells the model that renewal_risk_score is a number from 0 to 100. It does not say whether 82 is good, that account_health was abandoned last year, that the fiscal year starts in February, or that a third of those opportunities are CPQ artifacts.

You've spent most of your context window making the model more confident about field names, and changed none of the answers. Often it's worse than nothing: a model that can see 400 fields will use them, including the eleven that look authoritative and aren't.

Instead. Define the few fields that change the answer. Most fields need nothing; their help text is already fine. Spend the effort where the description is missing, stale, or doesn't carry the rule that matters.

2. The giant system prompt

What it looks like. All the rules, in one prompt, growing by a paragraph every time something goes wrong.

Why it fails. It works to about five fields. Then you add objects, scoring fields, lifecycle fields, regional exceptions, and the prompt is a junk drawer nobody will edit for fear of breaking something. You can't review a change, version it, test which rules fired, or reuse any of it in a second workflow.

Instead. Rules as data, prompt as delivery. The prompt still matters. It's how the model receives the rules for this answer. It just isn't where the library lives.

3. Vectorizing everything

What it looks like. Embed the field descriptions and the rules, retrieve by similarity, hand over the top k.

Why it fails. Rules need exact retrieval by scope and exact application. "Is this field authoritative" is a lookup, not a similarity score. "What predicate goes in the query" needs the literal predicate. "Does this request cross a boundary" needs a deterministic yes or no.

Worse, similarity retrieval will confidently return the four most semantically similar rules and silently omit the exclusion that wasn't worded like the question.

Instead. Index by id, title, and scope. Fetch by id. Keep RAG for the documents it's good at: contracts, notes, tickets.

4. Permissions in the prompt

What it looks like. "If the object isn't allowlisted, refuse and say what's missing," written into the system prompt.

Why it fails. It's a polite request to a model optimizing for a helpful answer. It holds most of the time, and the failures are invisible: the agent can't reach Contract, so it answers from Opportunity. No error. Just a different number.

Instead. Enforce at the tool boundary in code, against the finished query plan, after the model has stopped reasoning. Guidance text is for interpretation. Prompt instructions are not authorization.

5. One integration user with full access

What it looks like. The agent connects as a service account with a broad permission set, because it's simpler and everything works.

Why it fails. Every agent answer becomes a data-exfiltration path. A rep asks a question and gets records they could never open in the UI, and nothing about the answer signals it. You've also lost the only mechanism that was correctly limiting scope for free.

Instead. Run as the asking user. Accept that the same question returns different answers for different people, and make the agent say so.

6. Auto-generating guidance without review

What it looks like. A script reads field metadata and help text and writes rules automatically. The library is 300 rules by Friday.

Why it fails. It inherits whatever was wrong in the source, including help text that's been stale for two years, and produces confident-sounding rules nobody has verified. Now the wrongness is laundered: it looks like a reviewed rule.

Instead. Auto-generate into status: draft as a queue. A human approves before anything reaches an agent. Automation is good at finding fields that need attention and bad at deciding what they mean.

7. Treating field labels as semantics

What it looks like. No rule at all, on the theory that Account Health is self-explanatory.

Why it fails. Field labels are the single most reliable source of false confidence in a CRM. account_health sounds more authoritative than renewal_risk_score and is the one nobody has maintained since 2024. The model has no way to know which is which, and the label actively misleads it.

Instead. Mark the abandoned one non-authoritative and name its replacement. This is the cheapest high-value rule in the whole library.

8. Lazy-loading exclusions

What it looks like. Sensible phased retrieval, with exclusions fetched on demand like everything else.

Why it fails. An agent that doesn't know an exclusion exists cannot decide to fetch it. There's no gap for it to notice. It runs the query, gets a number that includes every test record in the org, and reports it with complete confidence.

Interpretation rules are safe to lazy-load. The worst case is a naive read the model usually flags. Exclusions are the asymmetric case.

Instead. Exclusions, anti-patterns, temporal, currency, and security rules always load. Every question, whatever the scope.

9. Scope-filtering as the whole retrieval strategy

What it looks like. "Load rules where scope == Opportunity." Reasonable, and it's what most first builds do.

Why it fails. Two ways. It loads the full body of every rule on that object, and rules pile up on exactly the objects people ask about most, so at sixty Opportunity rules you're back to the giant-payload problem. And questions don't stay on one object: someone asks about pipeline, the plan grows an Account join halfway through, and the Account rules were never loaded.

Instead. Index everything, fetch bodies on demand, and re-fetch mid-plan before querying any object the plan didn't start with.

10. Titles written as categories

What it looks like. Opportunity exclusions. Amount notes. Fiscal stuff.

Why it fails. In a phased design the agent decides whether to fetch a rule's body from its title alone. A category label loses that decision every time, so the rule sits there, active and correct and never applied.

Instead. State the condition under which the rule matters. Exclude system-generated and test opportunities from all pipeline and win-rate reporting. If a title could sit on a folder, rewrite it.

11. Shipping rule changes and retrieval changes together

What it looks like. One deploy: three rule edits and a tweak to how phase 3 selects bodies.

Why it fails. Something regresses and you have no idea which half caused it. Rule text and retrieval logic fail in similar-looking ways. An answer that's missing a consideration could be a reworded rule or a rule that never got fetched.

Instead. Separate deploys, golden set on both sides of each. Keep a one-click rollback to full, unphased guidance so you always have a known-good fallback that trades cost for correctness.

The pattern behind the pattern

Nine of these eleven share a shape: the failure returns a result.

Not an error, not a refusal, not a timeout. A number, in a sentence, with the right units and a plausible magnitude. An error gets fixed in five minutes because someone sees it. A plausible number gets forwarded to a VP.

That's why so much of this architecture is about making things fail loudly on purpose: throwing at the boundary instead of warning, erroring on suppressed nulls instead of returning them, stopping when guidance doesn't load instead of proceeding. Every one of those trades a small amount of availability for the ability to notice.

• How to build an AI context layer for your CRM — the concept and full build
• Debugging wrong CRM AI answers — the symptoms these produce at runtime
• One CRM question, traced end to end — what the correct version looks like in motion
• Permissions and safety for CRM AI agents — anti-patterns 4 and 5 in full