Building a ReAct Agent from Scratch (Part 6): Plan-Exec Mode and Beyond

Agent

Plan-Exec

Multi-Agent

Self-Reflection

ReAct

17 hours ago2024 words·11 min read

What You'll Learn

Understand ReAct's limitations: why "think-and-act" isn't always the optimal strategy
Master the Plan-Exec two-phase pattern: plan first, then execute
Build judgment on what strategy fits what task
Explore the frontiers of Agent technology: multi-agent collaboration, self-reflection, code execution, human-in-the-loop

Prerequisites

You've read Parts 1–5 of this series and have a complete understanding of the ReAct loop, tool system, and memory system

ReAct's Boundary: When "Think-and-Act" Isn't Enough

The ReAct Agent we built across five articles handles plenty of tasks: calculations, file reading, directory listing, shell execution. But consider this scenario:

txt

User: Deploy this Next.js project to the server.
      First check for TypeScript errors,
      then run the test suite. If everything passes,
      build a Docker image, push it to the registry,
      then SSH into the server, pull the new image, and restart the container.

Agent:
  Round 1: Action: runShell, "npx tsc --noEmit" → 0 errors
  Round 2: Action: runShell, "npm test" → all passing
  Round 3: Action: runShell, "docker build..." → build successful
  Round 4: Action: runShell, "docker push..." → push successful

  Round 5: Action: runShell, "ssh server 'docker pull...'"
           → Observation: Connection refused

  Round 6: Uh... the server is unreachable. Did I forget to set up SSH keys?
           Let me check...
           → Wait, what was the deployment workflow again? Where am I in the process?

What went wrong? The task is too long. A 12-step deployment flow, and by step 6 when an error occurs, the first five rounds of Observations have filled the context window. The Agent can't see the original deployment requirements, can't recall everything it's already done, and can't replan the remaining steps.

ReAct excels at clear, short, exploratory tasks where you adjust as you go. But for multi-step tasks with dependencies, where errors require replanning, you need a different pattern — Plan-Exec.

Plan-Exec: Plan First, Then Execute

The idea behind Plan-Exec is intuitive — it matches how humans handle complex tasks:

txt

ReAct: Think → Act → Observe → Think again
       Best for: exploratory tasks, incomplete information

Plan-Exec: Plan globally → Execute step by step
       Best for: structured tasks with clear steps and dependencies

The Two-Phase Workflow

Phase 1: Plan

Give the LLM a planning-specific System Prompt that asks it to decompose the user's problem into an ordered list of steps:

txt

You are a planning expert. Decompose the user's problem into clear, executable steps.

## Available Tools
(tool list)

## Output Format
PLAN:
1. <Step 1: what to do, which tool to use>
2. <Step 2: what to do, which tool to use>
...

## Rules
1. Each step should be clear, specific, and executable
2. Consider dependencies between steps
3. Do NOT execute — only plan

The LLM outputs a Plan, not an Action:

txt

PLAN:
Use runShell to run tsc --noEmit to check type errors
Use runShell to run npm test
If steps 1 and 2 pass, use runShell to build the Docker image
Use runShell to push the image to the registry
Use runShell to SSH into the server and pull the image
Use runShell to restart the container

Phase 2: Execute

Inject the Plan into the ReAct System Prompt as execution-guiding context:

txt

You are a Plan-Execute intelligent assistant.
You already have an execution plan — follow it step by step.

## Execution Plan
1. Use runShell to run tsc --noEmit to check type errors
2. Use runShell to run npm test
...

## Output Format
(same as ReAct: Action / Action Input / Final Answer)

## Rules
1. Follow the plan step by step; only one tool call at a time
2. If a step fails, assess whether to skip it or replan
3. Output Final Answer after all steps are complete

Then run the standard ReAct loop with this context. This brings three key benefits:

1. Global Perspective

The LLM can see the complete plan at every execution step. Even if the context window truncates earlier Observations, the Plan itself (usually just a few hundred tokens) remains in the System Prompt — the Agent never "gets lost."

2. Error Recovery

Plan-Exec naturally supports error handling. Step 4 failed? The LLM can assess: is this blocking (subsequent steps depend on its result), or can it be skipped? This determines whether to retry, skip, or abort.

3. Transparent Progress

The user sees what the Agent "intends to do" — the Plan is an auditable commitment. If the plan is wrong, the user can correct it before execution begins.

Implementation Notes

Plan-Exec requires only a few dozen lines beyond ReAct:

async function runPlanExec(query: string): Promise<AgentResult> {
  // Phase 1: Planning
  const planPrompt = buildPlanPrompt(tools);
  const planResponse = await llm.chat([
    { role: "system", content: planPrompt },
    { role: "user", content: query },
  ]);
  const plan = parsePlan(planResponse);  // Extract step list

  // Phase 2: Execution (reuse ReAct loop with plan context)
  const systemPrompt = buildPlanExecSystemPrompt(tools, plan);
  const messages = [
    { role: "system", content: systemPrompt },
    { role: "user", content: query },
  ];

  return executeReAct(messages);
}

parsePlan is straightforward — extract numbered lines after the PLAN: marker:

function parsePlan(text: string): string[] {
  const match = text.match(/PLAN:\s*([\s\S]*?)$/);
  const body = match?.[1] ?? text;
  return body
    .trim()
    .split("\n")
    .map((l) => l.replace(/^\d+[.)]\s*/, "").trim())
    .filter((l) => l.length > 0);
}

When to Use Which Pattern

Scenario	Recommended Pattern	Why
"Explore this repo's structure"	ReAct	Exploratory, steps unclear
"Deploy this project to the server"	Plan-Exec	Clear steps with dependencies
"Analyze and fix this bug"	ReAct	Need to discover as you go
"Initialize a new project with ESLint/Prettier/TS"	Plan-Exec	Known, repeatable workflow
"Write a technical blog post"	ReAct	Creative work, resists rigid planning

In practice, many Agent frameworks (LangGraph, OpenAI Agents SDK) allow dynamic switching at runtime — explore with ReAct first, then switch to Plan-Exec once the path is clear.

Series Retrospective: What We Actually Built

This is the final article. Let's look back at the complete knowledge graph:

txt

Agent = LLM + Tools + Loop
                              │
        ┌─────────────────────┼─────────────────────┐
        │                     │                     │
   ┌────▼────┐          ┌────▼────┐          ┌─────▼─────┐
   │   LLM   │          │  Tools  │          │   Loop    │
   │  Brain  │          │  Hands  │          │   Cycle   │
   └────┬────┘          └────┬────┘          └─────┬─────┘
        │                    │                      │
   ┌────▼────────┐    ┌──────▼───────┐    ┌────────▼────────┐
   │ System      │    │ Tool         │    │ executeReAct()  │
   │ Prompt      │    │ Interface    │    │ 7-step loop     │
   │ (Part 2)    │    │ (Part 3)     │    │ (Part 4)        │
   └─────────────┘    └──────────────┘    └────────┬────────┘
                                                    │
                          ┌─────────────────────────┼─────────────────────────┐
                          │                         │                         │
                    ┌─────▼──────┐          ┌──────▼──────┐          ┌───────▼──────┐
                    │  Memory    │          │ Plan-Exec   │          │  Extensions  │
                    │  (Part 5)  │          │  (Part 6)   │          │  (Part 6)    │
                    └────────────┘          └─────────────┘          └──────────────┘

Each article addresses an orthogonal dimension of the Agent, independently upgradable:

Prompt (Part 2): swap in a better prompt, compliance improves immediately
Tools (Part 3): add a tool, capability boundary expands by one
Loop (Part 4): swap the model, only LLMClient needs changing
Memory (Part 5): plug in Mem0 or a vector database, the Agent starts "remembering" users
Plan-Exec (this article): add a planning phase, the Agent handles complex multi-step coordination

Extension Directions: The Agent's Next Step

Six articles in, you can build a fully functional Agent. But Agent technology goes far beyond this. Here's what to explore next.

Multi-Agent Collaboration

When you decompose tasks to their limit, some subtasks are best handled by specialists, not generalists. The core idea of multi-agent systems: each Agent owns its specialty; they collaborate through messages or shared state.

txt

User: Write a technical blog about WebSocket with runnable example code

Coordinator Agent:
  ├── Research Agent: search for WebSocket best practices and latest RFCs
  ├── Writer Agent: compose the blog post based on research
  ├── Coder Agent: write and validate example code
  └── Reviewer Agent: review article accuracy and code correctness

Mainstream frameworks: LangGraph (graph-based Agent orchestration), OpenAI Agents SDK (handoff pattern), CrewAI (role-based Agent collaboration).

Self-Reflection and Self-Improvement

A current Agent either gets it right or gets it wrong within a single conversation — it doesn't learn from mistakes. Self-reflection (Reflexion) adds a "post-mortem" step:

txt

Execute task → Fail →
  Reflect: Why did it fail? What should I have done differently?
    → Store reflection in long-term memory
    → On similar future tasks, retrieve past reflections first

The Generative Agents paper's Reflection mechanism is an early implementation of this idea. Letta's "sleep-time compute" goes further — the Agent autonomously organizes memory, reflects on errors, and optimizes behavior during idle time.

Code Execution Sandboxes

Shell tools are powerful, but giving an LLM direct access to your host machine's terminal is risky. Code execution sandboxes (like OpenAI's Code Interpreter, E2B, Fly.io Machines) provide an isolated, disposable execution environment:

The Agent writes code, executes it, and observes the results inside the sandbox
The sandbox has its own filesystem and network environment
Once the task completes, the sandbox is destroyed, leaving no trace

This is far safer than granting raw shell access.

Human-in-the-Loop

Fully autonomous Agents are inappropriate in many scenarios — deployments need human confirmation, payment-related operations need authorization, critical decisions need human judgment.

Human-in-the-loop is typically implemented as: the Agent pauses at key decision points, notifies a human via callback or message queue, and waits for confirmation before proceeding. LangGraph's interrupt mechanism and the OpenAI Agents SDK's approval flow both have this pattern built in.

Hands-On Practice

Exercise 1: Add Plan-Exec to Your Agent (30 min)

Based on the Agent built in previous articles:

Write a buildPlanPrompt function — similar to the ReAct System Prompt, but asks the LLM to output PLAN: + numbered steps
Implement a parsePlan function
Add a /plan command to your CLI to toggle modes
Test Plan-Exec with complex tasks like "deploy a project" or "initialize a new project"

Exercise 2: Introduce Simple Self-Reflection (20 min)

After each Agent conversation ends, automatically trigger a "reflection call":

Send the full conversation history to the LLM, asking it to summarize "what went well, what could be improved"
Write the reflection to a local file
At the start of the next conversation, inject a recent reflections summary into the System Prompt

Observe whether the Agent can "learn" from past mistakes.

Advanced Challenge: A Minimal Multi-Agent Prototype

Launch two Agent instances (they can share the same LLM), one as "Researcher" (readFile, listFiles, runShell only) and one as "Writer" (writeFile only). Use a temp file or shared variable to pass intermediate results. Have the Research Agent collect information and write it to the file; the Writer Agent reads it and produces the final output.

Summary & Key Terms

ReAct vs. Plan-Exec: ReAct suits exploratory tasks (think-and-act); Plan-Exec suits structured tasks with dependencies (plan first, then execute)
Plan-Exec's two phases: Plan (LLM generates a step list) → Execute (inject the Plan as context into the ReAct loop)
Three advantages of Plan-Exec: global perspective (Agent always sees the full plan), error recovery (can replan on failure), transparent progress (user can audit the plan)
Six dimensions, independently upgradable: Prompt / Tools / Loop / Memory / Plan-Exec / Extensions — each dimension is an orthogonal upgrade path
Extension directions: multi-agent collaboration, self-reflection (Reflexion), code execution sandboxes, human-in-the-loop

Keywords: Plan-Exec, Two-Phase Planning, ReAct vs Plan-Exec, Multi-Agent, Self-Reflection, Reflexion, Code Sandbox, Human-in-the-Loop, LangGraph, CrewAI