Context layer vs. RAG vs. semantic layer
Three things that get substituted for each other and answer different questions. Why embedding your field descriptions doesn't work, and where a semantic layer genuinely overlaps.
These three get substituted for each other constantly, usually by someone who has one of them working and is wondering why the CRM answers are still wrong.
They answer different questions.
| Context layer | RAG | Semantic layer | |
|---|---|---|---|
| Question | What does this field mean, and may the agent use it? | Which documents are relevant? | What does this metric mean? |
| Operates on | Fields, objects, rules | Unstructured text | Metrics and dimensions |
| Retrieval | Exact, by scope and id | Approximate, by similarity | Exact, by metric name |
| Output | Rules to apply, and a boundary | Passages as background | A definition, often SQL |
| Fails by | Missing a rule nobody knew existed | Returning plausible but irrelevant text | Being right about a metric the agent shouldn't read |
Context layer vs. RAG
This is the substitution I see most, and it usually arrives as: can't we just embed the field descriptions and retrieve them?
You can. What you get is a fuzzy paraphrase of a field description, ranked by cosine similarity. What a rule needs is none of those things.
Rules need exact retrieval by scope. "What applies to Opportunity" is a filter, not a ranking. Similarity will return the four most semantically similar rules, which is not the same set.
Rules need a literal artifact. A filter predicate you can put in a query. A boolean for whether a field is authoritative. A yes or no on whether this request crosses a boundary. Embedding retrieval hands back prose that resembles the answer.
Rules need to load whether or not the question resembles them. This is the fatal one. An exclusion rule about test opportunities is not lexically or semantically similar to "how much pipeline did we create last quarter." A similarity search correctly ranks it low and drops it, and an agent that doesn't know an exclusion exists cannot decide to fetch it. The number comes back inflated, and nothing anywhere errors.
That's the structural argument. Similarity retrieval optimizes for relevance to the question, and the rules most likely to prevent a wrong answer are the ones the question gives no hint about.
RAG over contracts, call notes, and tickets is a genuinely good idea. It's answering a different question, and the two coexist fine: retrieve documents by similarity, retrieve rules by scope.
Context layer vs. semantic layer
The overlap here is real, and narrower than it looks.
A semantic layer defines ARR. That's a calculation, versioned, usually consumed by BI tools, and it's the right home for the definition.
A context layer says:
- which ARR field this agent may read, out of the four that exist;
- which population to exclude before the number means anything;
- which fiscal period applies, and how your year is numbered;
- whether the asking user is permitted to see it at all;
- that the field is a formula and therefore can't be grouped.
Different question, same noun. If you have a semantic layer, keep it and point the context layer at it rather than restating the definition, so ARR doesn't end up defined in two places that drift.
Where teams get burned is assuming the semantic layer's correctness carries over. A metric can be defined perfectly and still be the wrong field for an agent to read, still include records that shouldn't count, still be invisible to the person asking.
What about a knowledge graph
Adjacent, and optional. A knowledge graph maps entities and relationships: accounts to contacts to opportunities to tickets. That's structure.
A context layer is about interpretation, which is orthogonal. You can build one on top of a graph, and you don't need a graph to solve field meaning. Most CRM AI failures are interpretive rather than relational: the agent found the right records and misunderstood them.
Which do you need
Most CRM AI builds need a context layer and nothing else, because the failures are about field meaning and permission rather than document retrieval.
- Add RAG when questions genuinely require unstructured sources. "What did we agree to in their contract" is a RAG question. "How much qualified pipeline did we create" is not.
- Reuse a semantic layer when metric definitions already live somewhere authoritative. Don't create a second source of truth for ARR.
- Add a knowledge graph when relationship traversal is the hard part, which is uncommon in GTM CRM work.
The failure to avoid is reaching for RAG because it's the familiar tool, and ending up with an agent that retrieves four passages about a field and still doesn't know whether to trust it.
- How to build a CRM AI context layer — the concept and full build
- MCP vs. CRM context layer — the other thing this gets confused with
- Anti-patterns — "vectorize everything" in full
- Glossary — every term on this site, defined once
FAQ
- What's the difference between a context layer and RAG?
- RAG retrieves relevant documents by similarity and hands them to the model as background. A context layer retrieves specific rules by scope and applies them deterministically. The distinction is the retrieval model: 'which text is most similar to this question' versus 'is this field authoritative, what predicate goes in the query, and does this request cross a boundary.' Similarity ranking cannot answer the second set reliably.
- Can I just embed my CRM field descriptions and retrieve them?
- You'll get a fuzzy paraphrase of a field description ranked by cosine similarity, which is the wrong shape for a rule. Worse, similarity retrieval returns the top k most semantically similar rules and silently omits the exclusion that wasn't worded like the question — and an exclusion the agent doesn't know exists is one it can't decide to fetch.
- Is a context layer the same as a semantic layer?
- No, though they overlap on definitions. A semantic layer defines what a metric means, usually for analytics: this is how ARR is calculated. A context layer says which ARR field this agent may read, which population to exclude, which fiscal period applies, and whether the asking user is allowed to see it. If you have a semantic layer, keep it and point the context layer at it.
- Do I need all three?
- Rarely. Most CRM AI builds need a context layer and nothing else, because the failures are about field meaning and permission rather than document retrieval. Add RAG when questions genuinely need contracts, call notes, or tickets. Add or reuse a semantic layer when metric definitions already live somewhere authoritative and you don't want two sources of truth for ARR.
Get the next guide
New guides and the occasional note on GTM tooling. Don't worry, I won't drop you into a three-month nurture.