Adding Memory to an AI Agent
In the previous article, Adding Tools to an AI Agent, we gave our Agent the ability to use external tools.
Our Agent can now receive a request, decide whether a tool is needed, use the tool, and return the result to the user.
However, our Agent still has an important limitation:
It does not automatically remember everything from previous interactions.
For example, suppose a user tells the Agent:
My name is Mehrdad and I am a software developer.
Later, the user asks:
What programming language should I use for my next project?
A useful Agent may need to know the user’s background and preferences.
This is where memory becomes important.
What Is Memory in an AI Agent?
Memory allows an AI Agent to retain and use information from previous interactions or previous tasks.
A simplified Agent architecture can now look like this:
AI Agent
|
+--------------+--------------+
| | |
v v v
LLM Memory Tools
| | |
+--------------+--------------+
|
v
Result
Memory gives the Agent access to information that may have been obtained earlier.
Why Does an Agent Need Memory?
Without memory, each interaction can be treated as a new interaction.
For example:
User:
My name is Anna.
Agent:
Nice to meet you, Anna.
--- New interaction ---
User:
What is my name?
Agent:
I don't know.
With appropriate memory:
User:
My name is Anna.
Agent:
Nice to meet you, Anna.
--- New interaction ---
User:
What is my name?
Agent:
Your name is Anna.
The second example is more useful because the Agent can use information from a previous interaction.
Memory Is More Than Conversation History
It is important to distinguish conversation history from memory.
Conversation history contains previous messages from a conversation.
For example:
User:
My name is Anna.
Agent:
Nice to meet you.
User:
I work with C#.
Agent:
That's useful to know.
The complete conversation can be provided to the model again.
Memory is a broader concept.
Memory can contain selected information that should remain useful beyond the immediate conversation.
For example:
User Profile
Name: Anna
Profession: Software Developer
Preferred Language: C#
Experience: 8 years
The Agent can use this information when appropriate.
Short-Term Memory
Short-term memory is information that is useful during the current interaction.
For example:
User:
I am working on a web application.
Agent:
What technology are you using?
User:
ASP.NET Core.
Agent:
Then I recommend...
The Agent needs the earlier information to understand the current conversation.
This type of memory is often closely related to conversation state or context.
Long-Term Memory
Long-term memory contains information that may be useful across multiple interactions.
For example:
User Profile
Name: Anna
Role: Software Developer
Preferred language: C#
Preferred database: SQL Server
The Agent can use this information in future conversations.
Long-term memory usually requires some form of persistent storage.
A Simple Memory Architecture
A more complete architecture might look like this:
User
|
v
AI Agent
|
+----------------+----------------+
| | |
v v v
LLM Memory Tools
|
+------+------+
| |
v v
Short-Term Long-Term
Memory Memory
| |
v v
Conversation Database
Context
The exact architecture depends on the application.
What Information Should an Agent Remember?
An Agent should not necessarily remember everything.
For example, useful information might include:
- User name
- User preferences
- Programming language preference
- Project information
- Previous decisions
- Important instructions
- Customer preferences
- Frequently used information
But sensitive or unnecessary information should not automatically be stored.
A good memory system should have a clear purpose.
Our Practical Example
Let’s return to the Software Helper Agent that we created in the previous articles.
Our Agent currently has:
Software Helper Agent
|
+-- Instructions
|
+-- LLM
|
+-- Calculator Tool
We will now add memory:
Software Helper Agent
|
+-- Instructions
|
+-- LLM
|
+-- Calculator Tool
|
+-- Memory
The goal is simple:
Allow the Agent to remember information during our interaction.
Step 1 – Start With Conversation Memory
The simplest form of memory is maintaining the conversation history.
Imagine that our user says:
My name is David.
The Agent responds:
Nice to meet you, David.
Then the user asks:
What is my name?
If the previous interaction is available to the Agent, it can answer:
Your name is David.
The important point is that the Agent has access to the previous conversation.
Step 2 – Understand the Agent Session
An Agent interaction can be thought of as a session.
For example:
Session
|
+-- User message 1
|
+-- Agent response 1
|
+-- User message 2
|
+-- Agent response 2
|
+-- User message 3
|
+-- Agent response 3
The session contains the information necessary for the Agent to continue the conversation.
Modern Agent frameworks provide mechanisms for managing this state. The OpenAI Agents SDK has continued to add capabilities around agent execution and memory as part of its evolving architecture. (OpenAI)
Step 3 – Test Conversation Memory
Let’s test our Agent.
Instead of asking only one question, give it several messages.
For example:
User:
My name is David.
Agent:
Nice to meet you, David.
User:
I am a C# developer.
Agent:
That's useful to know.
User:
What programming language do I use?
Agent:
You use C#.
The Agent can answer the last question because the previous messages are part of the current interaction.
Screenshot
Take a screenshot showing the conversation where the Agent remembers information from earlier in the interaction.
Suggested caption:
The Agent uses information from earlier messages in the conversation.
Step 4 – Conversation Memory vs Persistent Memory
Now we need to understand an important difference.
Suppose we close our application.
Then we start it again.
If the previous conversation was only stored in memory during the application session, the Agent may no longer know what was discussed.
This gives us two different situations.
Session memory:
Application starts
|
Conversation
|
Application closes
|
Memory may disappear
Persistent memory:
Application starts
|
Conversation
|
Save information
|
Database
|
Application closes
|
Application starts again
|
Load information
|
Agent can use previous information
Persistent memory is much more powerful.
Step 5 – Store Important Information
Instead of storing every message forever, an application can extract important information.
For example:
Conversation:
User:
My name is David and I prefer C#.
Agent:
Understood.
The application could store:
Name = David
PreferredLanguage = C#
This information can then be used in future interactions.
Where Can Memory Be Stored?
There are many possible storage technologies.
For example:
- JSON file
- SQLite
- SQL Server
- PostgreSQL
- Cloud database
- Vector database
- Dedicated memory service
For a simple demonstration, a local file or small database may be enough.
For a production system, the choice depends on the application’s requirements.
Memory and Databases
A database can store structured information about the user.
For example:
User
--------------------------------
ID: 1001
Name: David
Role: Software Developer
Language: C#
Database: SQL Server
When the Agent needs this information, the application can retrieve it and provide the relevant information to the Agent.
The architecture becomes:
User
|
v
AI Agent
|
+---- LLM
|
+---- Tools
|
+---- Memory
|
v
Database
Memory Does Not Mean Remembering Everything
This is an important design principle.
An Agent should not automatically store every conversation and every piece of information.
Instead, the application should decide:
What should be remembered?
How long should it be remembered?
Where should it be stored?
Who can access it?
When should it be deleted?
For example, a customer’s preferred language might be useful for months.
A temporary calculation may only be useful for a few minutes.
Memory and Privacy
Memory also introduces privacy and security considerations.
If an Agent stores information about users, the application must protect that information.
Developers should consider:
- What data is stored?
- Why is it stored?
- Where is it stored?
- Who can access it?
- How long is it retained?
- Can the user delete it?
- Is sensitive information being stored unnecessarily?
The safest approach is to store only information that the application actually needs.
Memory + Tools
Now we can combine the concepts from our previous articles.
Our Agent has both memory and tools:
AI Agent
|
+---------------+---------------+
| | |
v v v
LLM Memory Tools
| |
v v
Database Calculator
The Agent can use both.
For example:
My name is David. Calculate the cost of 20 products at €15 each and remember that I prefer C#.
The Agent may:
- Understand the request.
- Store the user’s preference.
- Use the calculator tool.
- Receive the calculation result.
- Return the final answer.
The Agent is now becoming more capable.
The Complete Agent Workflow
We now have a more advanced workflow:
User
|
v
AI Agent
|
Understand Goal
|
+------------+------------+
| |
v v
Check Memory Select Tool
| |
v v
Retrieve Information Execute Tool
| |
+------------+------------+
|
v
LLM
|
v
Final Answer
This is becoming a real Agent architecture.
What Have We Added?
Our original Agent looked like:
Agent
|
+-- LLM
Then we added a tool:
Agent
|
+-- LLM
|
+-- Calculator Tool
Now we have:
Agent
|
+-- LLM
|
+-- Calculator Tool
|
+-- Memory
Each new capability makes the Agent more useful.
Why Memory Is Important for AI Agents
Memory allows Agents to work with information that extends beyond a single request.
This is especially useful for:
- Personal assistants
- Customer support
- Software development
- Education
- Business applications
- Long-running workflows
- Personalized recommendations
For example, a software-development Agent could remember:
Project: Customer Management System
Language: C#
Framework: ASP.NET Core
Database: SQL Server
Architecture: Clean Architecture
Testing: xUnit
The Agent can then use this information when helping the developer with future tasks.
Memory Is Not the Same as RAG
Memory and RAG are related but different concepts.
Memory is generally about information associated with the Agent, user, session, or previous interactions.
RAG (Retrieval-Augmented Generation) is primarily about retrieving relevant information from an external knowledge source and providing it to the model when answering a question.
For example:
Memory:
The user prefers C#.
RAG:
Search the company’s 500-page technical documentation and retrieve the section about authentication.
We will explore RAG in a later article.
What We Have Learned
In this article, we added another important capability to our AI Agent:
Memory.
We learned about:
- Conversation history
- Short-term memory
- Long-term memory
- Persistent memory
- Databases
- User preferences
- Privacy considerations
- Memory combined with tools
Our Agent can now combine:
LLM
+
Instructions
+
Tools
+
Memory
This is a significant step toward building more capable AI Agents.
Conclusion
Memory allows an AI Agent to use information beyond a single request.
Without memory, an Agent may treat every interaction as a new task.
With memory, an Agent can maintain context, remember useful information, and provide more personalized assistance.
However, memory should be designed carefully.
A good Agent should not remember everything simply because it can.
Instead, developers should decide what information is useful, how it should be stored, how long it should be retained, and how it should be protected.
In the next article, we will introduce another important technology:
RAG – Retrieval-Augmented Generation.
We will learn how to connect an AI Agent to external documents and knowledge so that it can retrieve relevant information and use it when answering questions.