Agent memory

Agent memory is the set of mechanisms by which an AI agent keeps information beyond a single call to the language model. It covers three different things: the state of the case in progress, stable knowledge about your company and its systems, and what the agent has learned from earlier cases. A model on its own has none of the three.

Why does an agent need memory at all?

A large language model is stateless. Every call starts from nothing, and whatever the model appears to know about your company arrived in the context window with the request. That is the mechanical reason why teams end up pasting the same background into every chat: the tool has no memory, so a person acts as one.

Memory is the engineering that ends the pasting. The system puts the right information in front of the model for each case. Nobody types it in, and nobody forgets to.

What kinds of memory are there?

Working memory is the state of the current case: what the agent has read, which tools it has called, what it has decided so far. It lives in the context window and in the state of the agent graph, and it is discarded when the case ends.

Long-term knowledge is what stays true across cases: product documentation, price lists, policies, the structure of your systems. It is stored outside the model and retrieved on demand, usually through retrieval-augmented generation over a document store.

Experience memory is what the agent has learned from earlier cases: outcomes, corrections a person made, preferences of a particular customer. It is stored as structured records and retrieved when a similar case appears.

Separate from all three is the model's training knowledge. You do not control it, it has a cut-off date, and it should not be relied on for anything about your company.

Why does this matter for a company?

"The agent forgets" is the complaint behind many pilots that did not fail because of the model. It fails because nobody designed what the agent knows at the start of a case, where that knowledge comes from, who maintains it, and what happens when a person corrects the agent.

That last point decides whether an agent gets better in operation. If a correction goes nowhere, the same mistake returns tomorrow. If it lands in experience memory, the agent improves without anyone retraining anything.

Fine-tuning is often reached for here and is usually the wrong tool. It bakes knowledge into model weights that you cannot inspect, cannot update on Tuesday afternoon, and cannot delete when a fact changes. Memory outside the model is a file or a record. You can read it, edit it and remove it.

Example and limits

Take a support-triage agent. Its working memory is the ticket, its attachments and the customer's last three tickets, fetched from the helpdesk when the case opens. Its long-term knowledge is the product documentation, retrieved per question. Its experience memory holds entries such as "when this customer writes about invoices, route to accounting, not support", written after a person corrected the routing twice.

The limit is that memory is data, and data has an owner. A wrong fact in memory is a hallucination with a paper trail: the agent will repeat it confidently until someone removes it. Every memory store therefore needs three answers. Who may write to it. When entries expire or get reviewed. Where personal data in it falls under your privacy rules.

Those answers belong in AI governance, and they are worth settling before the first agent goes live rather than after the first complaint. In the agents we build, the memory stores are part of the delivered system and stay in the client's own stack; see agents.