Planning and Reasoning in AI Agents
In the previous articles, we gradually increased the capabilities of our AI Agent.
Our Agent can now:
- Understand user requests
- Use an AI model
- Use tools
- Maintain conversation context
- Use memory
- Retrieve information using RAG
- Connect to external capabilities through MCP
Our Agent is becoming more powerful.
However, there is another important capability that makes an Agent different from a simple chatbot:
Planning and reasoning.
An AI Agent should not only answer a question. For more complex tasks, it should be able to determine what needs to be done, which steps are required, and which tools or resources should be used.
The OpenAI Agents SDK describes agent orchestration as the flow of agents and the decisions about which agents run, in what order, and what happens next. An LLM can make these decisions, while developers can also control the workflow through code. (OpenAI GitHub)
What Is Planning?
Planning means breaking a larger goal into smaller tasks or steps.
For example, a user might say:
Find information about AI Agents, summarize the important points, and create a report.
This is not necessarily one simple operation.
The Agent could plan:
Goal:
Create a report about AI Agents
Step 1:
Find relevant information
Step 2:
Read and analyze the information
Step 3:
Identify the important points
Step 4:
Create a summary
Step 5:
Create the final report
The Agent can then work through these steps.
What Is Reasoning?
Reasoning is the process of using the available information to decide what to do.
For example:
User:
Calculate the total cost of 10 products
at €25 each and add 25% VAT.
The Agent needs to determine:
1. Calculate 10 × €25
2. Calculate 25% VAT
3. Add VAT to the original amount
4. Return the final result
The Agent may use a calculator tool rather than trying to perform every operation itself.
Planning and Reasoning Together
Planning and reasoning are closely related.
A simplified workflow is:
User Request
|
v
Understand Goal
|
v
Create Plan
|
v
Execute Step
|
v
Evaluate Result
|
+---- More work? ---- Yes ----+
| |
No |
| |
v |
Final Answer <---------------------+
This is one of the important characteristics of an Agent.
A Simple Example
Consider this request:
I need a report about Python, including its advantages, disadvantages, and common applications.
A simple chatbot might immediately generate an answer.
An Agent can approach the task more systematically:
Goal:
Create a Python report
Plan:
1. Identify important topics
2. Research Python advantages
3. Research disadvantages
4. Research common applications
5. Organize the information
6. Write the report
The exact planning process depends on the Agent and the application.
Why Is Planning Important?
Planning becomes particularly useful when the user gives the Agent a complex goal.
For example:
Analyze these documents, identify the important problems, compare the results, and prepare a report.
This could involve several operations:
Documents
|
v
Read
|
v
Analyze
|
v
Extract Information
|
v
Compare
|
v
Generate Report
Without some form of orchestration, it would be difficult to manage such a workflow reliably.
Simple Tasks vs Complex Tasks
A useful distinction is:
Simple task
User:
What is Python?
Agent:
Generate answer.
Complex task
User:
Analyze these three documents,
compare their recommendations,
identify differences,
and prepare a report.
The second task requires multiple operations.
This is where planning becomes more valuable.
The Agent Loop
An Agent can operate through a loop.
Conceptually:
+------------------+
| User Request |
+--------+---------+
|
v
+------------------+
| Understand Goal |
+--------+---------+
|
v
+------------------+
| Plan |
+--------+---------+
|
v
+------------------+
| Execute an Step |
+--------+---------+
|
v
+------------------+
| Evaluate Result |
+--------+---------+
|
+------+------+
| |
Complete Continue
| |
v |
Answer <--------+
The OpenAI Agents SDK includes an agent loop that manages turns, tool invocation, and continuing execution until the task is complete. (OpenAI GitHub)
Planning Does Not Always Mean Writing a Visible Plan
When we talk about an Agent planning, this does not necessarily mean that the Agent must display a detailed internal reasoning process to the user.
For example, the user might simply see:
I will analyze the documents and prepare the report.
Behind the scenes, the application may coordinate several steps.
The important concept for developers is the workflow, not exposing private chain-of-thought.
Planning With Tools
Planning becomes particularly useful when an Agent has tools.
Suppose our Agent has:
Tools:
Calculator
File Search
Web Search
Database
The user asks:
Find the sales figures, calculate the yearly total, and prepare a summary.
The Agent may determine:
1. Search the sales data
2. Retrieve the relevant figures
3. Calculate the total
4. Analyze the result
5. Create the summary
The Agent can select the appropriate tool for each operation.
Planning With RAG
We can also combine planning with RAG.
Suppose the user asks:
Compare our company’s security policy with the new development guidelines.
The Agent might need to:
1. Search the security documentation
2. Search the development documentation
3. Retrieve relevant sections
4. Compare the information
5. Identify differences
6. Prepare the answer
The architecture becomes:
AI Agent
|
Planning
|
+---------------+---------------+
| | |
v v v
RAG Tools Memory
| | |
v v v
Documents External Previous
Systems Information
Planning With MCP
We can also combine planning with MCP.
For example, an Agent might have access to:
MCP Server
|
+-- Search Documents
+-- Read Files
+-- Access Database
+-- Create Report
The Agent can determine which capability is required for each step.
User
|
v
AI Agent
|
v
Create Plan
|
+---- Search Documents
|
+---- Read Information
|
+---- Analyze
|
+---- Create Report
MCP therefore provides the external capabilities, while the Agent determines when those capabilities are useful.
Replanning
Sometimes the first plan does not work.
For example:
Step 1:
Search for the document.
Result:
Document not found.
The Agent may need to change its approach:
Original Plan
|
v
Document not found
|
v
New Plan
|
+---- Search another location
|
+---- Ask user for document
|
+---- Continue with available information
This ability to adapt is important in real-world Agent systems.
Example: Software Development Agent
Let’s imagine a software-development Agent.
The user says:
Add a login feature to my application.
A simple response might provide sample code.
A more capable Agent could approach the task as a workflow:
Goal:
Add login functionality
Possible plan:
1. Inspect the project
2. Identify the framework
3. Inspect the existing authentication structure
4. Determine what is missing
5. Modify the required files
6. Run tests
7. Check for errors
8. Report the changes
This is much closer to an Agent performing a real task.
Planning Does Not Mean Unlimited Autonomy
An important point for developers is that an Agent should not automatically be allowed to perform every possible action.
For example:
Read files ✓
Analyze code ✓
Run tests ✓
Modify code Maybe
Delete files No
Deploy application Human approval
The permissions should depend on the application.
Human Approval
For important actions, we may want a human to approve the Agent’s plan before execution.
For example:
User
|
v
Agent
|
v
Create Plan
|
v
Human Review
|
+---- Reject ----> Stop
|
+---- Approve ---> Execute
This is especially useful for:
- Financial transactions
- Sending emails
- Deleting information
- Production deployments
- Changing databases
- Security-sensitive operations
The OpenAI Agents SDK includes mechanisms for human-in-the-loop workflows as part of its broader Agent capabilities. (OpenAI GitHub)
Planning in a Multi-Agent System
Planning becomes even more interesting when several Agents are involved.
For example:
Manager Agent
|
+--------------+--------------+
| | |
v v v
Research Agent Coding Agent Review Agent
The Manager Agent can decide which specialist should perform a particular task.
For example:
User:
Create a technical report.
Manager Agent
|
+---- Research Agent
|
+---- Writing Agent
|
+---- Review Agent
|
v
Final Report
This is called Agent orchestration.
The OpenAI Agents SDK supports two important orchestration patterns: agents can be used as tools while a manager remains in control, or a task can be handed off to a specialist Agent that becomes responsible for the conversation. (OpenAI GitHub)
Manager Agent vs Handoff
There are two useful patterns.
Manager pattern
The main Agent remains responsible for the final answer.
Manager Agent
|
+---------+---------+
| |
v v
Research Agent Review Agent
| |
+---------+---------+
|
v
Manager Agent
|
v
Final Answer
The specialist Agents provide assistance to the manager.
Handoff pattern
The first Agent transfers the task to a specialist.
User
|
v
Triage Agent
|
+---- Technical Question ---> Technical Agent
|
+---- Billing Question -----> Billing Agent
|
+---- General Question -----> Support Agent
With a handoff, the selected specialist becomes the active Agent for the next part of the interaction. (OpenAI GitHub)
A Practical Example
Imagine we build a customer-support system.
The user asks:
My payment was rejected. Can you help me?
A triage Agent can recognize that this is a billing issue.
It can then route the request:
User
|
v
Triage Agent
|
| Billing problem
v
Billing Agent
|
v
Billing System
|
v
Answer
If the user instead asks:
How do I reset my password?
the Agent could route the request to a technical-support Agent.
Planning vs Orchestration
These concepts are related but not identical.
Planning
Determines what steps should be performed to achieve a goal.
Goal
|
v
Step 1
|
v
Step 2
|
v
Step 3
Orchestration
Determines which Agents or components should perform those steps.
Manager
|
+---- Research Agent
|
+---- Coding Agent
|
+---- Review Agent
An Agent system can use both.
Planning and Memory
Memory can also help planning.
Suppose the Agent remembers:
Project:
Customer Management System
Technology:
C#
Framework:
ASP.NET Core
Database:
SQL Server
The user then asks:
Add a customer search feature.
The Agent can use the remembered project information when deciding how to approach the task.
The workflow might be:
User Request
|
v
Retrieve Relevant Memory
|
v
Understand Project
|
v
Create Plan
|
v
Use Tools
|
v
Complete Task
Planning and RAG
RAG can provide the Agent with additional knowledge before it creates or executes a plan.
For example:
User Request
|
v
Search Documentation
|
v
Retrieve Relevant Information
|
v
Create Plan
|
v
Execute Steps
This is useful when the Agent needs to follow company procedures or technical documentation.
Planning and MCP
MCP can provide the tools and external capabilities required by the plan.
For example:
Goal
|
v
Planning
|
+---- MCP: Search Documents
|
+---- MCP: Read File
|
+---- MCP: Database Query
|
+---- MCP: Create Report
|
v
Final Result
Now our Agent is combining several technologies:
AI Agent
|
+------------------+------------------+
| | |
v v v
Planning Memory LLM
|
+--------+---------+---------+
| |
v v
RAG MCP
|
v
Tools
Planning With Code
Not every workflow should be controlled entirely by the LLM.
Sometimes the developer should define the workflow explicitly.
For example:
result1 = search_documents()
result2 = analyze_documents(result1)
result3 = create_report(result2)
Here, the developer controls the sequence.
This is called code-based orchestration.
The OpenAI Agents SDK supports both LLM-driven orchestration and orchestration controlled through application code, and the two approaches can be combined. (OpenAI GitHub)
LLM-Driven Planning
In other situations, the Agent can decide what to do based on the user’s request.
Conceptually:
User Request
|
v
LLM
|
+---- Decide: Search?
|
+---- Decide: Use Calculator?
|
+---- Decide: Ask User?
|
+---- Decide: Finish?
This gives the Agent more flexibility.
However, it also means developers need to carefully control permissions, validation, and failure handling.
When Should We Use Code?
A useful rule is:
Use code when the workflow is predictable.
For example:
Step 1 → Step 2 → Step 3 → Step 4
Use Agent-driven planning when the workflow is variable.
For example:
User Request
|
v
Determine what information is needed
|
+---- Search?
+---- Database?
+---- Calculator?
+---- Ask user?
In real applications, a combination of both approaches is often useful.
Planning Does Not Guarantee Correct Results
An Agent can create a reasonable plan and still make mistakes.
For example:
- It may choose the wrong tool.
- It may misunderstand the user’s goal.
- It may retrieve incorrect information.
- A tool may fail.
- The data may be incomplete.
- The Agent may make an incorrect decision.
Therefore, good Agent design requires:
- Validation
- Error handling
- Tool permissions
- Monitoring
- Testing
- Human approval where appropriate
Our Agent Architecture
Our Agent has now evolved significantly.
We started with:
Agent
|
+-- LLM
Then we added:
Agent
|
+-- LLM
+-- Tools
+-- Memory
+-- RAG
+-- MCP
Now we add planning and reasoning:
AI Agent
|
Planning
|
+-------------------+-------------------+
| | |
v v v
LLM Memory Tools
| |
v v
RAG MCP
|
v
External Services
The Agent can now:
- Understand a goal.
- Plan a workflow.
- Retrieve information.
- Remember relevant context.
- Select tools.
- Execute actions.
- Evaluate results.
- Adapt when necessary.
- Produce a final answer.
This is much closer to a practical AI Agent.
What We Have Learned
In this article, we introduced planning and reasoning in AI Agents.
We learned that planning allows an Agent to break a complex goal into smaller steps.
We also learned that reasoning allows the Agent to decide what information, tools, or actions may be required.
We explored:
- Planning
- Reasoning
- Agent loops
- Tool selection
- RAG and planning
- MCP and planning
- Memory and planning
- Human approval
- Multi-Agent orchestration
- Manager Agents
- Handoffs
- Code-based orchestration
- LLM-driven orchestration
The basic concept is:
User Goal
|
v
Understand
|
v
Plan
|
v
Execute
|
v
Evaluate
|
+---- Continue
|
v
Complete
Conclusion
Planning and reasoning are important capabilities for AI Agents because complex tasks usually require more than one operation.
A simple chatbot can answer a question directly.
An AI Agent can potentially take a goal, determine the steps required, use appropriate tools, retrieve information, remember relevant context, and adapt its approach as it works.
Our Agent has now developed from a simple LLM-based application into a much more complete architecture:
AI Agent
|
+-----------------+-----------------+
| | |
v v v
LLM Memory Tools
| |
v v
RAG MCP
|
v
Knowledge
|
v
Planning
|
v
Actions
In the next article, we will take another important step and explore Multi-Agent Systems.
We will see how several specialized AI Agents can work together, how a manager Agent can coordinate them, and when it is better to use multiple Agents instead of one large Agent.
→ Building a Multi-Agent System
← Back to AI Agents – Step-by-Step
Note for your series
For your current series, I recommend keeping this article conceptual rather than adding another Python project. We already demonstrated a simple Agent, tools, memory, RAG, and MCP concepts. The next natural step is to show your readers several Agents working together in practice. The OpenAI Agents SDK specifically supports both agents-as-tools and handoffs, so that practical example will fit very well with what we have explained here. (OpenAI GitHub)