Retrieval-augmented generation supplies relevant external information at answer time. Agent memory preserves selected context across work. Both need provenance, freshness and access controls; storing more text does not automatically produce a better answer.
Begin with the information boundary
A knowledge assistant is only as useful as the material it can access and interpret. Decide which sources are authoritative, who can see them and how updates enter the index. A search result should carry the document identity and enough context to understand the passage. Without that, a plausible answer is difficult to inspect.
Apply permissions when retrieving information, not only when displaying the final answer. A model that has already seen an unauthorized passage has crossed the information boundary even if the interface later hides the citation. This is a systems design concern, not something a prompt alone can solve.
Chunking is a meaning problem
Breaking a document into pieces makes retrieval practical, but can separate a sentence from its qualification. A policy exception might sit under a heading that is lost during ingestion. A table cell might be meaningless without its column labels. Inspect actual retrieved passages before tuning an embedding model.
Anthropic's Contextual Retrieval describes adding contextual information to chunks before indexing. It is one published technique worth evaluating, not a universal prescription. Compare approaches using questions drawn from your documents, including questions where the right answer is that the available material does not establish a conclusion.
Memory needs a lifecycle
An assistant may need to remember an agreed decision, a project constraint or the last verified state of a task. Those records are different from a complete conversation transcript. A concise memory with its source, date and scope can be easier to use and correct than an undifferentiated history.
Decide how a record is created, reviewed, updated and removed. Make a correction append or supersede earlier evidence in a way the application can explain. Distinguish a user's current instruction from an old preference and a saved observation from a verified present fact. Otherwise, memory can quietly turn yesterday's context into today's mistake.
Evaluate retrieval and answers separately
When an answer fails, first ask whether the system retrieved the required information. If not, examine source coverage, parsing, query formation and ranking. If the evidence was present, inspect how the model used it. Separating these stages makes the next change more precise.
A useful review set includes exact-name lookups, paraphrased questions, ambiguous requests and questions outside the corpus. Check citation support, not simply whether a URL appears. In the interface, give people a direct route to the passage and a way to report missing or outdated material. A good knowledge product makes disagreement and correction straightforward.
Questions, answered.
Does RAG eliminate hallucinations?
No. Retrieval can provide evidence, but the system can retrieve the wrong material or misinterpret it. Evaluate source support and handle uncertainty explicitly.
Is agent memory the same as a vector database?
No. A vector database can help retrieve records. Memory also requires decisions about what to retain, provenance, updates, permissions and deletion.
Sources & further reading
Source-linked explanation and engineering perspective. Programme details and product documentation can change; consult the original sources for their current terms and capabilities.
Anthropic — Contextual Retrieval ↗