Building a ReAct Agent from Scratch (Part 4): The Core Loop — Putting It All Together

Agent

ReAct

LLM

TypeScript

DeepSeek

Streaming

60 minutes ago2847 words·15 min read

What You'll Learn

Understand the complete ReAct execution loop — from LLM call to tool execution to Observation feedback
Master the principles of streaming output: how to display the LLM's thinking process in real time
Grasp the core rule of message management: why every step must preserve the complete reasoning chain
Learn to decouple Agent logic from the user interface using the observer pattern
Build an interactive Agent CLI tool from scratch

Prerequisites

You've read Parts 1–3 of this series and understand the ReAct format, tool interface, and response parsing
You have a basic grasp of the OpenAI SDK's streaming API (stream: true)

Recap: The Parts Are Ready. Now Assemble Them.

The first three parts built:

Part 1: the core Agent concept — LLM (brain) + Tools (hands) + Loop
Part 2: the communication protocol — designing Prompts so LLMs reliably output Action/Action Input/Final Answer, and parsing those responses
Part 3: the Tool system — interfaces and implementations for 6 tools (calculator, clock, file I/O, directory listing, shell execution)

Now we have the System Prompt, the parser, and the tools — but they're still separate pieces. The task of this article is to connect them with a main loop, turning them into a program that understands natural language, calls tools, and interacts with users.

Specifically, we need to do three things:

Encapsulate LLM calls: Agent logic shouldn't depend on the OpenAI SDK directly — switching models should require changing only one place
Implement the main loop: think → parse → execute tool → observe → think again, until a final answer emerges
Build an interactive interface: let users converse with the Agent in the terminal

Step 1: Encapsulating LLM Calls

The Agent's core logic shouldn't care which model or SDK is being used underneath. Let's write a thin wrapper with a simple chat(messages) interface.

Why Streaming?

First, a fundamental question: why must we use streaming?

A non-streaming call looks like this:

// Non-streaming: wait 10 seconds, get everything at once
const response = await client.chat.completions.create({
  model: "gpt-4",
  messages: [...],
});

The user stares at a blank screen, not knowing what the Agent is thinking or whether it's stuck.

Streaming is completely different — the API returns text token-by-token via SSE (Server-Sent Events):

const stream = await client.chat.completions.create({
  model: "gpt-4",
  messages: [...],
  stream: true,  // The key parameter
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (delta?.content) {
    process.stdout.write(delta.content);  // Output character by character
  }
}

For an Agent, streaming's value goes beyond user experience — you can see the LLM's chain-of-thought in real time. If a reasoning step goes off track, you'll notice immediately rather than discovering a bizarre result 30 seconds later.

Separating Reasoning from Body Content

If you use a model with chain-of-thought capabilities (like DeepSeek Reasoner or OpenAI o1), the API returns two types of content: content (the body text) and a special reasoning_content field — the model's "inner monologue." They need separate handling:

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;
  if (!delta) continue;

  // Chain-of-thought: the model's internal reasoning process
  const reasoningDelta = (
    delta as { reasoning_content?: string }
  ).reasoning_content;
  if (reasoningDelta) onReasoning?.(reasoningDelta);

  // Body content: Action / Final Answer extracted from here
  if (delta.content) {
    fullContent += delta.content;
    onContent?.(delta.content);
  }
}

The point of separating them: reasoning content is for the user (so they can see what the Agent is "thinking"), while body content drives the loop logic (parsing Action / Final Answer).

Note: reasoning_content is a non-standard field specific to DeepSeek Reasoner. The standard OpenAI API doesn't have this field — both reasoning and body text live in content. If your Agent uses a regular chat model, you can ignore this distinction.

Model Compatibility: Handling the System Role

Most models support the system role (for setting system-level instructions), but some reasoning models don't. If your model rejects the system role, you need a simple preprocessing step — merge system messages into the first user message:

function foldSystemIntoUser(messages: Message[]): Message[] {
  // Collect all leading system messages
  const systemTexts: string[] = [];
  while (messages.length > 0 && messages[0].role === "system") {
    systemTexts.push(messages.shift()!.content);
  }

  if (systemTexts.length === 0) return messages;

  const combined = systemTexts.join("\n\n");
  if (messages[0]?.role === "user") {
    messages[0].content = `${combined}\n\n${messages[0].content}`;
  } else {
    messages.unshift({ role: "user", content: combined });
  }

  return messages;
}

This way, regardless of whether the underlying model supports the system role, the Agent's System Prompt gets delivered correctly.

The Complete LLM Wrapper

Putting it all together, here's a clean LLM client:

class LLMClient {
  private client: OpenAI;
  private model: string;

  constructor(config: { apiKey: string; baseURL: string; model: string }) {
    this.client = new OpenAI({
      baseURL: config.baseURL,
      apiKey: config.apiKey,
    });
    this.model = config.model;
  }

  async chat(
    messages: Message[],
    callbacks?: {
      onReasoning?: (delta: string) => void;
      onContent?: (delta: string) => void;
    }
  ): Promise<string> {
    const isReasoner = this.model.includes("reasoner");

    const stream = await this.client.chat.completions.create({
      model: this.model,
      messages: isReasoner ? foldSystemIntoUser([...messages]) : messages,
      temperature: isReasoner ? undefined : 0,
      stream: true,
    });

    let content = "";
    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta;
      if (!delta) continue;

      const reasoningDelta = (
        delta as { reasoning_content?: string }
      ).reasoning_content;
      if (reasoningDelta) callbacks?.onReasoning?.(reasoningDelta);

      if (delta.content) {
        content += delta.content;
        callbacks?.onContent?.(delta.content);
      }
    }
    return content;
  }
}

Note the design choices:

temperature: 0 (non-reasoning models): ensures deterministic output — critical for ReAct parsing; we don't want the same prompt producing random formats
[...messages] shallow copy: foldSystemIntoUser mutates the array; the copy prevents affecting the original reference
Returns the full accumulated text: the return value is used for parsing Action / Final Answer

Step 2: Implementing the Main Loop

With the LLM wrapper done, the next step is the heart of the Agent — the ReAct main loop.

What Does the Loop Actually Do?

Recalling Part 1's minimal prototype, the loop's essence is:

txt

Send messages to LLM
  ↓
Get LLM response → Parse it
  ↓                    ↓
Has Final Answer?    Has Action?
  End                 Execute tool → Get Observation → Append Observation to history → Back to top

But a real implementation needs to handle many edge cases. Let's build it up from the simplest framework.

The Minimal Framework

const MAX_STEPS = 8; // Prevent infinite loops

async function executeReAct(
  messages: Message[],
  tools: Tool[],
  llm: LLMClient,
  callbacks?: AgentCallbacks
): Promise<string> {
  for (let step = 0; step < MAX_STEPS; step++) {
    callbacks?.onThinking?.("Thinking...");

    // 1. Call the LLM
    let reasoning = "";
    const response = await llm.chat(messages, {
      onReasoning: (delta) => { reasoning += delta; },
    });

    // 2. Append the full response (reasoning included) to message history
    const assistantContent = reasoning
      ? `<thinking>\n${reasoning}\n</thinking>\n${response}`
      : response;
    messages.push({ role: "assistant", content: assistantContent });

    // 3. Parse the response
    const parsed = parseLLMResponse(response);

    // 4. If Final Answer → done
    if (parsed.isFinal) {
      return parsed.finalAnswer!;
    }

    // 5. If no Action parseable → return the raw text as the answer
    if (!parsed.action) {
      return response;
    }

    // 6. Execute the tool
    const observation = await executeTool(parsed, tools);
    callbacks?.onObservation?.(observation);

    // 7. Append Observation to history; continue to next round
    messages.push({
      role: "user",
      content: `Observation: ${observation}`,
    });
  }

  return "Sorry, I thought for too long without reaching an answer. Please try rephrasing.";
}

That's the skeleton of the ReAct loop. About 30 lines containing the entire core logic of an Agent. But several key design decisions deserve a closer look.

Message Management: Why Reasoning Must Be Preserved

In step 2, we set the assistant message content to reasoning + response, not just response. This is a critical design decision.

Imagine this multi-step reasoning scenario:

txt

Round 1 LLM reasoning: I need to list files first to understand the project structure...
Action: listFiles → Observation: src/ components/ utils/ README.md

If the LLM in the next round can't see the round 1 reasoning ("I need to list files first to understand the project structure"), it loses context — it doesn't remember why it listed files or what it hoped to discover. It's like a person who gets amnesia after every single step, stumbling in the dark.

Every mainstream Agent framework (LangChain, OpenAI Assistants, Anthropic Claude) preserves the complete reasoning chain. This is a principle, not an option.

If you're worried about verbose reasoning exhausting the context window, the correct approach is truncation or summarization, not outright deletion. We'll explore this in detail in Part 5.

Handling an "Uncooperative" LLM

LLMs don't always output in the format you expect. The loop needs fallbacks for two cases:

Empty body content: some reasoning models (like DeepSeek Reasoner) occasionally put Action in the reasoning channel while the body is empty. When parsing fails, try parsing the reasoning content as a fallback:

let parsed = parseLLMResponse(response);

// Fallback: if body is empty, try reasoning
if (!parsed.action && !parsed.isFinal && reasoning) {
  parsed = parseLLMResponse(reasoning);
}

Malformed output: if there's neither a Final Answer nor an Action, return the LLM's raw response as the final answer. This is far better than crashing or entering an infinite loop.

Error Handling in Tool Execution

Tool execution can fail in many ways: unknown tool name, JSON parse failure in parameters, internal exceptions. None of these should crash the Agent:

async function executeTool(
  parsed: ParsedResponse,
  tools: Tool[]
): Promise<string> {
  const tool = tools.find((t) => t.name === parsed.action);
  if (!tool) {
    return `Error: tool "${parsed.action}" does not exist. Available: ${tools.map(t => t.name).join(", ")}`;
  }
  try {
    return await tool.execute(parsed.actionInputJson ?? {});
  } catch (e) {
    return `Tool execution error: ${e instanceof Error ? e.message : String(e)}`;
  }
}

The key design choice: failure info is returned to the LLM as an Observation, letting the LLM decide the next move. Seeing an error, it can:

Fix the parameters and retry
Switch to a different tool
Tell the user it can't be done right now

This fail gracefully strategy gives the Agent basic self-correction capability.

Step 3: Decoupling UI with the Observer Pattern

The Agent's core loop shouldn't depend on any specific UI. Today we want terminal output; tomorrow we might want a web interface — the Agent code shouldn't change for that.

The solution is the observer pattern — the Agent notifies the outside world of its progress through a set of callbacks:

interface AgentCallbacks {
  onThinking?: (label: string) => void;
  onReasoning?: (delta: string) => void;
  onAction?: (action: string, input: string) => void;
  onObservation?: (observation: string) => void;
  onFinalAnswer?: (answer: string) => void;
}

Each key point in the loop fires the corresponding callback:

// Inside the loop
callbacks?.onThinking?.("Thinking...");

// During LLM reasoning
callbacks?.onReasoning?.(delta);

// When a tool is called
callbacks?.onAction?.(parsed.action, parsed.actionInput);

// When a tool result arrives
callbacks?.onObservation?.(observation);

// Final answer
callbacks?.onFinalAnswer?.(answer);

This design completely decouples the Agent from the presentation layer. The colored output you see in the terminal (cyan tool names, yellow Observations, green answers) is just one concrete implementation of the callbacks — the Agent knows nothing about it.

Step 4: Building Terminal Interaction

With the Agent's core logic in place, the final step is making it conversational. We need two interaction modes.

Single-Shot Mode

The user types a question directly on the command line, the Agent answers, then exits:

bash

$ node agent.js "What is 128731 × 46291?"

The implementation is straightforward — construct messages, run the loop, print the answer:

async function runOnce(agent: ReActAgent, query: string): Promise<void> {
  console.log(`\n🧠 ${query}\n`);
  const answer = await agent.run(query, terminalCallbacks());
  console.log(`✨ ${answer}\n`);
}

REPL Mode

When launched without arguments, the Agent enters a continuous conversational mode (REPL: Read-Eval-Print Loop):

bash

$ node agent.js
🤖 Agent ready. Type /help for commands.

❯ What does README.md say?
✨ It's a Node.js project...

❯ What dependencies are in package.json?
✨ Dependencies include openai, dotenv...

Implementation-wise, use Node.js's readline module to create an input loop:

import readline from "node:readline";

function startREPL(agent: ReActAgent): void {
  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });
  rl.setPrompt("❯ ");
  rl.prompt();

  rl.on("line", async (line: string) => {
    const input = line.trim();
    if (!input) { rl.prompt(); return; }
    if (input === "/exit") { rl.close(); return; }
    if (input === "/clear") {
      agent.clearHistory();
      console.log("Conversation history cleared");
      rl.prompt();
      return;
    }

    const answer = await agent.run(input, terminalCallbacks());
    console.log(`✨ ${answer}\n`);
    rl.prompt();
  });
}

A few slash commands make the interaction more natural:

/clear — clear conversation memory, start a new topic
/exit — quit
/help — show available commands

Terminal Rendering

To give the output a sense of "intelligence," we do a few things in terminalCallbacks:

Spinner animation: while the LLM thinks, show a braille spinner (⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏) rotating every 80ms
Stream reasoning: when onReasoning receives a delta, output it character-by-character in gray — the user sees what the Agent is thinking in real time
Color coding: Action (cyan), Observation (yellow), Final Answer (bold green)

function terminalCallbacks(): AgentCallbacks {
  const spinner = createSpinner();

  return {
    onThinking: (label) => spinner.start(label),
    onReasoning: (delta) => process.stdout.write(dim(delta)),
    onAction: (action, input) => {
      spinner.stop();
      console.log(cyan(`  🔧 ${action}`));
      console.log(dim(`     ${input}`));
    },
    onObservation: (obs) => console.log(yellow(`  👀 ${obs}`)),
    onFinalAnswer: (answer) => {
      spinner.stop();
      return greenBold(`  ${answer}`);
    },
  };
}

The complete user experience:

txt

❯ What is 128731 × 46291?
  ⠋ Thinking...
  I need to do large-number multiplication. LLMs often make
  errors on multiplication, so I should use the calculator
  tool to ensure accuracy.
  🔧 calculate
     {"expression": "128731 * 46291"}
  👀 128731 * 46291 = 5958833521
✨ 128731 × 46291 = 5,958,833,521

Assembly: Wiring Everything Together

We now have all the components. The final step is connecting them — and it takes just a few lines:

// 1. Load configuration (API key, model name, etc.)
const config = loadConfig();

// 2. Create the LLM client
const llm = new LLMClient(config);

// 3. Create the Agent (inject LLM and tool list)
const agent = new ReActAgent(llm, tools);

// 4. Determine launch mode
const query = process.argv.slice(2).join(" ");
if (query) {
  runOnce(agent, query);
} else {
  startREPL(agent);
}

The dependency graph is clear at a glance:

txt

Config ──→ LLMClient ──┐
                        ├──→ ReActAgent ──→ CLI
       tools[] ─────────┘

Each module has a clear responsibility:

Config: reads environment variables
LLMClient: calls the API, handles streaming
Tool[]: defines and executes tools
ReActAgent: manages the loop logic and message state
CLI: handles user interaction

Any layer can be independently replaced — swap models in LLMClient, swap interaction style in CLI — the Agent's core logic stays untouched.

Hands-On Practice

Exercise 1: Build a Minimal Working Agent (30 min)

Based on the first four articles, write a complete, runnable single-file Agent. Requirements:

At least 3 tools (calculator, clock, file reader)
Support single-shot mode
Use streaming to display the LLM's thinking process

You don't need a full REPL — just get one complete conversation working end-to-end.

Exercise 2: Add Logging (15 min)

Modify the AgentCallbacks implementation to record each interaction (Action, Observation, Final Answer) to a log file. Observe how many loop rounds a complex multi-step task actually consumes.

Advanced Challenge: Support Multiple Models

Make your Agent support switching the underlying model via command-line argument or environment variable (e.g., GPT-4, Claude, DeepSeek). Observe the behavioral differences between models on the same task — some models are more "obedient" (strictly following output format), while others are more "opinionated" (preferring to reason on their own rather than calling tools).

Summary & Key Terms

LLM encapsulation: expose a simple chat(messages) interface; internally handle streaming, reasoning/body separation, and model compatibility (system role folding)
Seven steps of the main loop: call LLM → update message history (include reasoning) → parse → check Final Answer/malformed output → execute tool → append Observation → next round
The iron rule of message management: assistant messages must contain the complete reasoning chain (reasoning + content), so the model can see its own thinking context in subsequent rounds
MAX_STEPS hard limit: prevent infinite loops; exit gracefully rather than crashing
Observer pattern: the Agent notifies the outside world through callbacks, completely decoupled from the UI
Fail gracefully: tool execution failures don't crash — errors are returned as Observations so the LLM can self-correct

Keywords: ReAct Loop, Streaming, Message Management, Observer Pattern, Fail Gracefully, MAX_STEPS, Context Window