Agent Memory Architecture: The 3 Tiers Every Production AI Agent Needs
By Delos Workers team — 2026-11-01
Your agent forgets what you told it last week. A bigger context window will not fix it. Here is the architecture that does.
Your agent handled a supplier dispute perfectly on Monday. On Thursday, it asks the same questions again. That is not a model problem. It is an architecture problem: the agent has no memory worth the name. In 2026, memory is treated as a dedicated component that sits beside the context window, not as a longer prompt.
Why a bigger context window is not memory
The intuitive fix for an agent that forgets is a longer context window. It does not work. The memory problem is not storage capacity: it is deciding what is worth keeping, what has stopped being true, and what should surface right now. A million-token window holds more of the conversation, yet does nothing about a preference expressed in March and contradicted in July. Practitioners also report that retrieval degrades for a counterintuitive reason: search is rarely bad, but irrelevant past state keeps outranking fresh context. That is a ranking problem, not an embedding problem (GMI Cloud, Agent Memory Architecture for Production, Oct 2026).
The cost of ignoring memory is concrete. On the LOCOMO benchmark, the Mem0 team measured a p95 latency of 17.1 seconds when the full conversation history is passed in context, against 1.44 seconds with a dedicated memory layer (Mem0, arXiv:2504.19413).
The three tiers of agent memory
1. Working memory
This is the context window itself: the instructions, the current task and the few facts the agent needs right now. It is fast and volatile. Rule: keep it small and curated, and never use it as an archive.
2. Session memory
Per-thread state that survives a turn: the ticket being handled, the documents already opened, the decisions made so far. It lets a multi-step task resume after an interruption. Rule: store it outside the prompt, keyed by thread, and expire it when the task closes.
3. Long-term memory
Knowledge that persists across sessions: client preferences, past incidents, validated procedures. This is the only tier that is genuinely hard. It splits into episodic memory (time-stamped events: 'on 12 March, the client refused the quote') and derived or semantic memory (distilled facts: 'this client negotiates on payment terms, not price'). Databricks' MemCurrent design (Oct 2026) combines exactly these two ideas: raw episodic records plus derived memory built from them.
Write policies: the decision that makes or breaks it
Three rules that work in production:
- Read on the hot path, write in the background. A user waiting on a memory extraction call is waiting for work that does not improve the current answer. Write at session boundaries.
- Batch consolidation. Vector storage is cheap. The LLM calls used for extraction and consolidation can cost 10 to 100 times more than storage (GMI Cloud). Consolidating per session rather than per message is usually the single most effective cost decision.
- Handle contradiction and staleness. Every memory needs a date and a source. When a new fact contradicts an old one, update or retire the old one instead of stacking both.
A concrete enterprise example
Take a sales follow-up agent (see our article on why sales follow-ups go unanswered). Working memory holds the email being drafted. Session memory holds the open deal and the last three exchanges. Long-term memory holds that this buyer replies on Tuesday mornings, dislikes attachments, and was promised a discount cap in the last negotiation. Without tier 3, every email starts from zero. With it, the agent behaves like a colleague who remembers.
Governance: memory is data
What an agent remembers is personal and business data. Define retention periods, let users view and delete what is stored, and keep memory per tenant or per team. Research on agent memory increasingly lists security, privacy and safe deletion as a core topic (Memory Papers, memorypapers.org).
Checklist before you ship
- Which facts must survive a session, and which must not?
- Where does each tier live, and who can read it?
- Are writes asynchronous and batched?
- How are contradictions resolved and old facts retired?
- Can a user inspect and delete their memory?
- Do you measure retrieval quality on your own cases, not only on public benchmarks?
Conclusion
Memory is what turns a clever model into a reliable colleague. Start with the three tiers, write in the background, date everything, and keep it deletable. To go further, read our guide on agentic AI enterprise workflows and our comparison of agent vs RPA vs script.