Building a ReAct Agent from Scratch (Part 5): Memory — The Agent's Context Window

• Understand the three-layer model of Agent memory: short-term, working, long-term
• Master mainstream short-term memory strategies: sliding window, summarization, OpenAI Compaction
• Dive deep into two cutting-edge research approaches: MemGPT's virtual context management and the Generative Agents memory stream architecture
• Understand the role and limitations of RAG and vector databases in long-term memory
• Build the judgment to choose the right memory strategy for your use case

• You've read Parts 1–4 of this series and understand the ReAct loop and message management
• Basic familiarity with embeddings and vector search (not required; explained inline)

Previous article:

Imagine this conversation with an Agent:

The third question should be answerable from the first two Observations. But the Agent has already "forgotten" — the context window filled up with accumulated messages, and the earliest Observations were truncated.

This is the central tension of memory systems: an Agent's capability depends on how much it can "remember," yet the context window is a hard physical limit. GPT-4's 128K tokens sounds generous, but a few file operations and ReAct cycles consume it entirely.

Agent memory systems draw on the layered memory architecture of human cognition — not as metaphor, but as engineering practice:

Let's go through each layer in depth.

Short-term memory is the first hurdle every Agent framework must clear. Here's the evolution from "naive" to "smart."

Advantage: no information loss. Disadvantage: API throws errors when the conversation gets slightly long.

Keep only the most recent N messages or N tokens; delete the oldest:

Advantage: simple implementation. Fatal flaw: deleted content is gone forever — if the user says "go back to what we first discussed," the Agent can only apologize.

Instead of deleting history, use the LLM to compress old conversations into a summary:

This summary replaces the original conversation as part of the system message, drastically saving tokens:

This is the core logic behind LangChain's ConversationSummaryMemory and one of the strategies used internally by OpenAI's Assistants API.

Keep the most recent K turns in full; compress everything older into a summary. This is LangChain's ConversationSummaryBufferMemory and the most commonly used approach in production:

Why keep recent turns intact? Summarization inevitably loses detail — tone of voice, exact numbers, code snippets. Recent content is most likely to be referenced; keeping it uncompressed is a worthwhile token investment.

OpenAI's Responses API provides an automated compaction feature. Developers don't need to write their own summarization logic — when context approaches the limit, the API automatically compresses older messages into shorter equivalent representations. It's essentially weighted summarization, fully managed server-side.

Comparing the five approaches:

Short-term memory handles "remembering the conversation." But when executing complex tasks, an Agent also needs to pass intermediate results between steps within the same task — this memory doesn't span sessions, but lasts longer than a single Observation.

Consider this task:

The Agent's execution:

The problem: Rounds 2–11 produced massive Observations (file counts per directory) that filled the context window. By Round 12, when aggregation is needed, the earliest results are gone.

Give the Agent a "notepad" — let it write intermediate results down and read them back when needed:

This "notepad" file is working memory — it's not part of the conversation history, but it can be retrieved via tool calls when needed.

• Claude Code / Codex CLI: create .claude/ or .codex/ directories in the repo root during multi-step tasks, storing intermediate state and todo lists
• OpenAI Deep Research: internally maintains a research notes document, appending findings after each search and reading round, ultimately generating a report from the accumulated notes
• LangChain Deep Agents: provides a built-in virtual filesystem where agents can freely read and write temporary files

Key insight: working memory isn't something the Agent framework manages for you — it's something you enable by giving the Agent "write file + read file" capabilities and letting it manage its own scratch paper.

Short-term memory lives only within a single conversation. Reopen the terminal tomorrow and the Agent is brand new — it doesn't know you, doesn't remember what you did before. Long-term memory solves this "cross-session amnesia."

This isn't a "can we store it?" problem — disk is cheap. The real challenge is precisely retrieving the most relevant fraction from a vast memory store.

This is the most mainstream engineering approach, used by LangChain, OpenAI Assistants, and countless startups.

Write:

Retrieve:

The advantage of vector search is semantic understanding — "React project" and "frontend framework" are close in vector space even without literal overlap. The limitation is lack of structured reasoning — it can tell you "the user previously used Zustand" but cannot infer "the user dislikes Redux, so don't recommend Redux this time either."

In 2023, UC Berkeley's MemGPT paper proposed a radical idea: let the LLM manage its own memory like an operating system.

The core insight comes from OS virtual memory:

MemGPT splits the context window into two parts:

• Main Context: information currently being processed, like pages loaded into RAM
• External Context: all historical information archived in a database, like disk storage

The LLM manages memory through a set of special "memory management functions":

• conversation_search(query) — retrieve relevant memories from external storage
• archival_memory_insert(content) — store important information to external storage
• archival_memory_search(query) — search the archive

This essentially delegates the "paging" decision to the LLM itself, rather than using preset truncation rules. When context is nearly full, the LLM can proactively decide: what to "swap out" to external storage, and what to "swap in" from external storage.

MemGPT later evolved into Letta, which further introduced:

• Context Constitution: a "constitution" for memory management, defining what the agent should remember, forget, and how to organize memory
• Context Repositories: Git-based management of agent memory, supporting branching, rollback, and diff — because memories can "rot" and need version control
• Sleep-time Compute: the agent autonomously organizes its memory during idle time — reflecting, deduplicating, distilling insights

In 2023, Stanford's Generative Agents paper built a simulated town with 25 AI agents that wake up, cook breakfast, go to work, and make friends. What powers all this is a carefully designed memory architecture:

The core innovation isn't retrieval (RAG is everywhere) — it's reflection: letting the agent not just store raw memories, but periodically distill higher-level understanding from many low-level memories.

Example:

This high-level understanding is far more useful in subsequent conversations than raw conversation fragments — it helps the Agent make decisions aligned with user preferences.

In practice, these approaches are often combined — vector retrieval as the first recall layer, reflection as the second distillation layer, and the LLM making the final decision about what to place in context.

With all these approaches laid out, you might wonder: which one should my Agent use?

Answer these three questions first:

1. How long are your conversations? If each session is under 10 turns, a sliding window may suffice. If sessions can run into hundreds of turns, you absolutely need summarization or compaction
2. Do you need cross-session memory? If each conversation is an independent task (e.g., one-off code review), short-term memory is enough. If this is a long-term assistant, long-term memory is mandatory
3. How structured is your information? If it can be abstracted to facts like "user prefers X," vector retrieval is sufficient. If complex reasoning is needed, consider a reflection mechanism

Recommended starting point:

Most applications stop at Step 3 and are already excellent. Step 4 is for when your Agent needs to "know you" like a genuine human assistant.

On top of the Agent built in Parts 1–4:

1. Implement a countTokens function (use tiktoken or simple estimate: 1 token ≈ 4 characters)
2. When history exceeds 80% of the window, call the LLM to compress the earliest 1/3 into a summary
3. Observe how response quality changes before and after summarization

Add saveToScratchpad and readScratchpad tools to your Agent (under the hood, just read/write a temp file). Then give the Agent a task requiring intermediate results across multiple rounds. Observe whether it actively uses these tools.

1. Set up a local vector database with npx chroma or sqlite-vec
2. After each conversation, call the OpenAI Embeddings API to convert the conversation summary into a vector and store it
3. At the start of the next conversation, use the user's query to retrieve the most relevant historical memories
4. Inject the retrieved results into the System Prompt

Observe how an Agent with long-term memory responds as if it "knows the user."

• Memory determines the upper bound of Agent capability: the context window is the physical bottleneck; memory management is how you break through it
• Three-layer model: short-term memory (conversation window management) → working memory (scratchpad) → long-term memory (cross-session persistence)
• Short-term strategy evolution: full retention → sliding window → summarization → buffer + summary hybrid → API-managed compaction
• Cutting-edge long-term approaches: RAG + vector retrieval (production-ready), MemGPT virtual context (OS-inspired), Generative Agents memory stream (reflection-based), Letta Git-based memory (versioned)
• Where to start: begin with sliding window, progressively add summarization → add vector retrieval → add reflection as needed

Keywords: Short-term Memory, Working Memory, Long-term Memory, Context Window, Sliding Window, Summarization, Compaction, RAG, Vector Database, MemGPT, Generative Agents, Memory Stream, Reflection, Context Constitution

The next article is the final chapter of this series: Plan-Exec Mode and Beyond. When tasks become too complex for ReAct's "think-and-act" alone, how do we introduce the "plan first, then execute" two-phase pattern — and what lies ahead for Agent technology: multi-agent collaboration, self-reflection, and long-running task orchestration.

Next article: