Agent Memory: When It Helps and When It Creates Risk

Agent memory helps the moment it saves a user from repeating context they’ve already given. It creates risk the moment it keeps something that was only ever true temporarily, or something nobody meant to make permanent. Both of those happen through the exact same mechanism: writing a fact outside the current conversation so a future one can use it. The mechanism isn’t the problem. What gets written into it, and for how long, is.

Three kinds of memory, three different risk profiles

Working memory lives inside one session and disappears when it ends. It’s the safest kind, since nothing it holds outlives the conversation that produced it.

Persistent memory survives across sessions on purpose: a remembered preference, a running project context, a fact the agent is meant to carry forward. This is where most of the real risk lives, because anything written here is now a standing claim the agent will treat as true until something explicitly updates it.

Retrieved memory, the kind a RAG system provides, is really an external, versioned record rather than the agent’s own memory at all. It’s worth naming separately because it has a property neither of the other two has: it can be corrected at the source, the way Nexus treats a synced Slack thread or wiki page. Persistent memory usually can’t be corrected that cleanly; it can only be overwritten or deleted.

Where memory genuinely helps

Inside one long task, memory cuts the back-and-forth that would otherwise repeat the same context every step. A multi-step workflow that remembers the customer ID it was given three turns ago doesn’t need to ask for it again. That’s working memory doing exactly its job, bounded to a task that has a clear end.

Low-stakes, easily corrected persistent memory earns its place too: a preferred name, a formatting preference, a default time zone. If it’s wrong, the cost of being wrong is small and the fix is a one-line correction.

Where it creates risk

Stale facts treated as current. A memory written in March saying a deadline is in June doesn’t know the deadline moved in April. Unlike a RAG citation, which points at a source that can be checked and resynced, a plain memory entry just sits there being confidently wrong until something notices.

Context leakage. Memory written in one conversation, with one user, in one context, can surface in a different one if it isn’t scoped tightly. An agent that remembers something from a private conversation and brings it up in a shared or lower-trust context has turned a convenience feature into a privacy incident.

Memory poisoning. If anything a user says can get written into long-term memory without review, an adversarial input doesn’t need to jailbreak the model in the moment. It just needs to get a false “fact” stored once, and that fact will keep influencing answers long after the original conversation is gone. This is a slower, quieter version of prompt injection, and it’s easy to miss because the damage shows up in a later, unrelated session.

Permission drift. Something remembered while an agent had broad access can get recalled later in a context where that access should no longer apply. Memory doesn’t automatically inherit the permission boundary of the moment it was captured in.

flowchart LR
  accTitle: Where agent memory risk actually comes from
  accDescr: Working memory is scoped to one session and disappears with it, which makes it low risk. Persistent memory survives across sessions and carries the real risk: staleness, context leakage, poisoning, and permission drift. Retrieved memory from a RAG system is external and correctable at the source, which is a different and generally safer category.

  WORK[Working memory<br/>one session, then gone]
  PERSIST[Persistent memory<br/>survives across sessions]
  RISK[Staleness &middot; leakage &middot;<br/>poisoning &middot; permission drift]
  RETRIEVE[Retrieved memory<br/>external, correctable at the source]

  WORK -->|low risk| SAFE[Bounded by design]
  PERSIST --> RISK
  RETRIEVE -->|correct the source, not the memory| SAFE

What to do about it

• Treat a memory write the way you’d treat any other data write: reviewable, deletable, scoped to a user or project, timestamped.
• Default to retrieval over memory for anything that’s actually knowledge rather than preference. If it can change and someone needs to be able to fix it, it belongs in a source you can correct, not in an opaque memory entry.
• Give persistent memory an expiry or a revalidation step, rather than assuming it stays true forever just because nothing deleted it.
• Scope memory per user and per project by default. Global memory shared across every context is exactly how leakage and permission drift happen.

Key takeaways

• Working memory is low risk because it’s bounded to a session. Persistent memory is where staleness, leakage, and poisoning actually live.
• A memory that can’t be corrected at a source is a memory that can only get more wrong over time, not less.
• Scope and expiry aren’t optional extras on a memory system. They’re the two things that keep it from becoming a liability.

FAQ

Is a RAG knowledge base the same thing as agent memory?

No, and the distinction matters. A knowledge base holds facts that exist independently of any one conversation and can be corrected at the source. Agent memory usually holds facts about the conversation or the user themselves, like preferences or running context, that don’t have an external source to correct against.

How do you prevent memory poisoning from a single conversation?

Don’t write to persistent memory automatically from raw user input. Require an explicit confirmation step, or restrict what can be written to a narrow, validated set of fields, the same way you’d restrict any other write path an untrusted input can reach.

Should every agent have persistent memory by default?

No. Add it when there’s a concrete, low-stakes use for it, like a preference that’s genuinely annoying to repeat. An agent that doesn’t need to remember anything across sessions is simpler and has one less place for stale or leaked data to hide.

Related resources

• RAG Versus Fine-Tuning for Operational Knowledge
• Domain-Specific AI Agents: Why Composition Beats One Big General Agent
• Nexus: I Built a Self-Hosted Knowledge Base That Cites Its Sources