We’ve spent considerable time in this series exploring what AI agents can do – how they use tools, access information through RAG, and coordinate everything through protocols like MCP. But we’ve been operating under a crucial assumption that it’s time to examine directly.
We’ve assumed that AI agents actually know what to do next.
This assumption breaks down more often than you might expect, and when it does, it’s usually the difference between an impressive demo and a production system that reliably delivers business value.
Today, we’re diving into one of the most critical – and often overlooked – components of agentic AI: how these systems actually think, plan, and reason their way through complex tasks.
The Planning Challenge: Beyond Simple Commands
Let’s start with a simple example. If you ask an AI agent “What’s $2,000 divided by $15?” it can either calculate the answer directly or use a calculator tool. This is straightforward – one clear request, one clear response, minimal decision-making required.
Now consider this request: “Find all our Q2 clients in the healthcare sector, identify which ones haven’t paid invoices in the last 60 days, and create personalized follow-up emails with updated payment links.”
Suddenly, the agent needs to:
- Understand the multi-part instruction
- Break it down into logical sequence of steps
- Determine which tools and data sources to access
- Execute actions in the correct order
- Handle any errors or missing information along the way
- Know when the entire task is actually complete
This process of interpreting, sequencing, and coordinating actions is what we mean by planning – and it’s where many agentic AI implementations struggle or fail entirely.
Why Traditional AI Models Struggle with Planning
To understand this limitation, we need to remember how most AI models are actually trained. At their core, large language models (LLMs) are trained to predict the next piece of text based on what came before. They excel at completing sentences, generating summaries, and answering direct questions because these tasks align with their training objective.
But planning requires something different – the ability to think ahead, consider multiple possible paths, and coordinate actions over time. Traditional language models are essentially “short-sighted” in this regard. They’re brilliant at responding to what’s immediately in front of them, but they weren’t designed to maintain long-term goals or adapt strategies based on changing circumstances.
This creates several common failure patterns in agentic systems:
- Skipping crucial steps in multi-part processes
- Repeating actions unnecessarily or getting stuck in loops
- Overcomplicating simple tasks with unnecessary steps
- Losing track of the original goal partway through execution
- Failing to handle unexpected responses from tools or systems
Early Solutions: Prompting for Better Behavior
The initial approach to addressing these planning limitations involved clever prompting techniques. Developers discovered that phrases like “let’s think step by step” or “break this down into manageable parts” could nudge models to exhibit more planning-like behavior.
Chain-of-thought prompting became particularly popular, where models would explicitly work through their reasoning process before providing answers. This worked well for logic puzzles, math problems, and structured question-answering scenarios.
But these prompting techniques had significant limitations when applied to real-world agentic tasks. They were essentially tricks that made models appear to plan, without fundamentally changing how the models processed information. When faced with dynamic environments, unexpected tool responses, or complex state management, these prompted planning behaviors often broke down.
The Rise of Reasoning Models
Recognition of these limitations led to a fundamental shift in how AI models are designed and trained. Instead of trying to prompt existing models to behave like planners, researchers began training models specifically for reasoning and planning tasks.
This gave birth to a new category: Large Reasoning Models (LRMs).
The key difference is architectural:
Traditional LLMs follow this pattern: Input → Processing → Output
Reasoning models follow this pattern: Input → Planning/Reasoning → Action + Justification
Both still operate with text, but reasoning models are explicitly trained to think before acting, to consider multiple approaches, and to maintain awareness of their goals throughout task execution.
OpenAI‘s o-series models (like o1 and o3) were among the first widely available examples. These were followed by DeepSeek’s R1 model and reasoning variants from other major providers. Today, most foundation model companies offer reasoning-enabled versions of their models, with some implementing hybrid approaches that activate reasoning capabilities only when needed.
How Reasoning Models Change Agentic Design
The primary value of reasoning models lies in improving the planning component of agentic systems – the part that determines “what should I do next and why?” This is often the weakest link in enterprise AI implementations.
Enhanced Planning Capabilities:
- Better task decomposition for complex, multi-step workflows
- Improved error handling and recovery strategies
- More sophisticated tool selection and sequencing
- Better adaptation to changing circumstances during task execution
Strategic Thinking:
- Ability to consider multiple approaches before acting
- More sophisticated goal prioritization
- Better understanding of task dependencies and constraints
Different Approaches to Agentic Reasoning
The field has developed several distinct strategies for implementing reasoning in agentic systems:
Conditional Logic Approaches. These systems use programmed if-then rules to guide decision-making. While less flexible, they’re highly reliable for well-defined scenarios like fraud detection or automated approval workflows.
Goal-Based Planning. Using heuristics, these agents are given specific objectives and use search algorithms to find optimal sequences of actions. They’re particularly effective for navigation, resource optimization, and workflow automation tasks.
Iterative Reasoning (ReAct Pattern). The ReAct (reason + act) approach implements a think-act-observe loop. The agent reasons about what to do, takes action, observes the results, and uses those observations to inform the next reasoning cycle. This works well for exploratory tasks but can sometimes get stuck in repetitive loops.
Plan-Ahead Strategies (ReWOO Pattern). Instead of reasoning after each action, these systems plan multiple steps in advance, gaining its name ReWOO (Reasoning WithOut Observation). A planner breaks down the task, workers execute individual components using available tools, and a solver synthesizes everything into a final result. This approach is more efficient but less adaptable to unexpected situations.
Self-Reflective Systems. Advanced reasoning agents can evaluate their own performance, identify errors in their reasoning, and adjust their approaches accordingly. This creates more robust systems but requires additional computational resources.
The Reality Check: When to Use Reasoning Models
While reasoning models represent a significant advancement, they come with important trade-offs that organisations need to consider:
- Increased Complexity and Cost. Reasoning models typically require more computational resources and time to generate responses. They often produce longer, more detailed outputs that may be unnecessary for simple tasks.
- Potential for Over-Engineering. These models can overthink straightforward problems, adding unnecessary complexity to simple workflows.
- Higher Risk of Sophisticated Mistakes. When reasoning models hallucinate, they often generate very convincing but completely incorrect reasoning chains that can be harder to detect than simple factual errors.
A Practical Approach to Implementation
The key to successfully implementing reasoning capabilities is matching the complexity of your solution to the complexity of your problem.
- Start Simple: Begin with standard models for your initial implementation. Many tasks that seem like they require sophisticated planning can actually be handled effectively with well-designed prompts and clear tool interfaces.
- Identify Planning Failures: Only consider reasoning models when you observe clear evidence that planning limitations are preventing your system from working effectively.
- Evaluate the Value: When you do implement reasoning capabilities, carefully measure whether the improved planning actually translates to better business outcomes, and whether those improvements justify the additional complexity and cost.
- Design for Your Use Case: Different reasoning approaches work better for different types of problems. Task-specific optimization often beats general-purpose reasoning capabilities.
The Strategic Importance of Planning
Understanding planning and reasoning capabilities is crucial for organisations implementing agentic AI because these capabilities often determine the difference between systems that work in controlled demos and systems that deliver reliable value in production environments.
The most successful enterprise AI implementations we see combine sophisticated planning capabilities with practical constraints and human oversight. They use reasoning models where the complexity is justified, but they don’t assume that more sophisticated reasoning is always better.
Looking Forward: The Evolution of AI Decision-Making
As reasoning models continue to evolve, we’re likely to see more nuanced approaches that can dynamically adjust their planning sophistication based on task complexity. The goal isn’t to make every AI interaction more complex, but to ensure that AI systems can handle complexity when it’s actually necessary.
The organisations that succeed with agentic AI will be those that understand not just what their AI can do, but how it thinks about what to do next. This understanding informs everything from system architecture decisions to user experience design to risk management strategies.
Ready to Think Through Your Planning Strategy?
If you’re implementing or considering agentic AI systems, understanding how these systems approach planning and reasoning is essential for making informed architecture decisions.
The questions to ask aren’t just about what tasks you want to automate, but about how complex the decision-making processes are within those tasks, how much flexibility your systems need, and what level of planning sophistication is actually justified by your business requirements.
Whether you’re dealing with straightforward workflow automation or complex, multi-step business processes that require sophisticated planning, we’d love to help you think through the reasoning requirements for your specific use cases. Contact us to explore how different planning and reasoning approaches might fit into your agentic AI strategy.
Sometimes the most important decision about AI planning is knowing when you don’t need it. Other times, sophisticated reasoning capabilities are exactly what transform a limited tool into a powerful business asset.
In our next post, we’ll explore another critical component of agentic systems: memory – how AI agents remember previous interactions and use that context to improve their performance over time.
