The Augmented LLM: The Building Block of AI Agents
An augmented LLM is a plain LLM plus retrieval, tools, and memory, the primitive Anthropic says every AI agent is built from.
A coding assistant once answered a question about a file I hadn't opened in three months, and for a second it felt like it remembered my whole project. It didn't. It had just searched the codebase and pulled the right lines into the prompt, the same trick a plain model can do if you hand it a search tool.
An augmented LLM is a large language model plus three additions. Retrieval (looking things up), tools (taking actions), and memory (carrying state across turns). Anthropic calls this the basic building block that every agentic system is made from, and once you can name the three parts, you can take apart almost any "smart" AI product you've ever used.
Key Takeaways
- An augmented LLM is a plain model plus retrieval, tools, and memory, nothing more mysterious than that.
- Retrieval fetches outside information at the moment you ask, tools let the model act on the world, memory persists information after the conversation moves on.
- Retrieval and memory get confused constantly: retrieval is stateless, memory is not.
- Every coding assistant you've used is an augmented LLM wearing a different outfit, once you know where to look.
What Is an Augmented LLM?
An augmented LLM is a language model wired up with retrieval, tools, and memory, the exact vocabulary Anthropic uses in its guide to building effective agents. On its own, a raw LLM only does one thing: it takes text in and predicts text out. It cannot check today's date, run a calculation it can't do in its head, or remember what you told it five minutes ago in a different session.
Augmentation fixes that, and Anthropic is specific about how: "Our current models can actively use these capabilities, generating their own search queries, selecting appropriate tools, and determining what information to retain." That word "actively" matters. The model isn't just plugged into a database, it decides when to search, which tool fits the job, and what's worth keeping. That decision-making is what separates an augmented LLM from a model bolted onto a script.
This is also the primitive the rest of this path builds from. Anthropic's own framing treats retrieval, tools, and memory as the three knobs you tune before you ever reach for a full agent, and every later concept in this course, RAG, tool calling, agent memory, is one of these three knobs turned up. It's also worth noting that having these three parts doesn't automatically make something an agent, the line between a chatbot and an agent is about who controls the sequence of steps, a question the next lesson takes on directly.
Retrieval: How an Augmented LLM Looks Things Up
Retrieval means fetching outside information at the moment the model needs it, instead of relying only on what it learned during training. A model trained two years ago has no idea what happened last week, retrieval is how it finds out without being retrained.
In practice, retrieval usually means searching something: a folder of documents, a database, a codebase, the web. The model generates a query, the search runs, and the model pulls the results into the prompt so it can reason over them. That's genuinely how Cursor's codebase indexing works when it "understands" your project. It searched. It didn't memorize.
Retrieval only works as well as the query the model writes. A vague search returns vague results, which is why "generating its own search queries" is listed as an active model skill, not a fixed pipeline step.
The moment that reset my own assumptions was watching an assistant answer a question about a file I hadn't touched in months. It felt like it remembered my project. It didn't. It searched the repo, found the right three lines, and stuffed them into that one prompt. No memory was involved, just a retrieval step that ran and disappeared.
Tools: How an Augmented LLM Takes Action
Tools let a model do things beyond generating text: run a calculation, call an API, edit a file, execute a command. Without tools, a model can only describe what should happen. It can tell you the shell command to run, but it can't run it.
Anthropic's Model Context Protocol is one concrete way this gets standardized: "one approach is through our recently released Model Context Protocol, which allows developers to integrate with a growing ecosystem of third-party tools with a simple client implementation." You'll build against MCP directly later in this path, for now, the point is simpler: a tool is anything the model can call, get a result back from, and use to decide its next move.
This is the difference you already felt in the loop from the last lesson: a one-shot prompt can only describe a fix, a model with tools can open the file, make the edit, and run the test to check its own work. Tools turn description into action.
Memory: How an Augmented LLM Carries State
Memory is information the system deliberately saves and reloads later, after the original conversation has scrolled out of view. It's what lets an agent pick up a multi-day project without you re-explaining everything from scratch.
This is where most beginners get tangled, because a context window looks a lot like memory from the outside. It isn't the same thing. A context window is everything the model can see in this exact turn, your prompt, recent messages, whatever retrieval just pulled in. Memory is what gets written down on purpose so it survives after that window moves on and the original text is gone.
If an agent can only "remember" what happened three messages ago, that's the context window doing its job, not memory. Real memory means the information is still there tomorrow, in a fresh session, with none of today's messages in view.
Memory isn't automatically worth adding, either. It earns its cost only when continuity actually improves the result. Summarizing a single document once doesn't need it. A coding agent working across a multi-week feature branch usually does.
Retrieval vs. Memory: The Line People Blur
Retrieval and memory solve different problems, even though both involve fetching information into a prompt. Retrieval is stateless: it brings in outside knowledge at the moment you ask, and forgets that it did so the instant the answer is done. Memory is stateful: it's built to persist, updated on purpose, and pulled back out later.
Put another way, retrieval answers "what's out there that's relevant right now," memory answers "what have we already established, and does it still apply." A search engine has retrieval. A search engine has no memory of you unless it's built one separately.
This distinction matters because a system can have excellent retrieval and zero memory, and still feel impressively capable in a single conversation, right up until you start a new session and it has forgotten everything, including things it definitely should have kept.
Pulling Apart a Coding Assistant Into Its Three Parts
Here's the exercise that makes this concrete: take a coding assistant you already use and label which feature is which.
- Retrieval: Cursor's codebase search, it looks up relevant files the moment you ask a question.
- Tools: Claude Code's ability to read a file, edit it, and run your test suite, taking action instead of just describing what to do.
- Memory: a saved conversation history or a project's
learning-log.mdfile, information that outlives the current session.
None of those three parts is "the AI being smart" on its own. The intelligence is in the base model deciding when to search, which tool to reach for, and what's worth writing down, exactly the "active" decision-making Anthropic describes. Strip away retrieval, tools, and memory, and you're left with a model that can only talk, which is a real limitation and also exactly why this vocabulary is worth having: it tells you precisely what to add when a plain model isn't enough.
This sounds simple once it's named. It didn't feel simple the first time a tool felt "smart" for knowing my whole codebase, when what it had actually done was run one good search.
Why This Primitive Matters for the Rest of This Path
Everything you'll build from here decomposes into one of these three knobs. Retrieval-augmented generation, which you'll meet properly later, is retrieval done at scale over a document store. The Model Context Protocol you'll build against down the line is a standardized way to wire up tools. Persistent memory systems, the kind that survive across sessions, are a deeper version of the memory piece you just met here.
Knowing the three-part vocabulary now means none of those later lessons will feel like a brand-new concept, they'll feel like an old friend with more detail added. That's the whole point of learning the primitive first.
Your Lab
Open a new markdown file in Cursor
Create a file called augmented-llm-diagram.md in your project (or a scratch folder). This is where you'll do the labeling.
Pick a real product you use
Choose one AI tool you've actually used, a coding assistant, a research chatbot, a customer-support bot, anything with more than a plain chat box.
Label its three parts
In your markdown file, write three headings: Retrieval, Tools, Memory. Under each, write one to three sentences naming the specific feature of your chosen product that fits, and why. If a part is missing entirely, say so, that's a valid finding, not a mistake.
Commit it to your learning log
Copy your three labeled sections into learning-log.md under a heading for Lesson 15.02. This becomes your reference the next time a course concept (RAG, MCP, memory systems) asks "which part of the augmented LLM is this?"
Done? You've completed Lesson 15.02.
FAQ