Lesson 2: The Augmented LLM: The Building Block

One unit, three attachments

Before we draw any multi-step diagram, look at what you already ship. A plain model call can only work from its prompt and its training. An augmented LLM is that same call with three attachments:

• Tools: functions the model can ask you to run.
• Retrieval: information fetched at request time and put in front of the model.
• Memory: anything carried forward from earlier turns or earlier conversations.

Anthropic's December 2024 engineering essay Building Effective Agents names this unit as the basic building block of agentic systems. Everything later in this course is this block, wired up in different ways. So it pays to know it well: when a bigger system misbehaves, the fault is usually inside one of these blocks.

The model drives the attachments itself. It writes the search query, picks the tool and decides what is worth keeping. Your job is to give it a clean, well-documented interface to each one.

Map it to code you have written

Here is a support assistant in one call. Each attachment is labelled.

import anthropic

client = anthropic.Anthropic()

# RETRIEVAL: your code fetched these before the call (vector search, SQL, grep...)
policy_chunks = search_policies("refund window damaged item")  # your function

# TOOLS: a name, a description and a JSON schema. This is all the model sees.
tools = [{
    "name": "get_order",
    "description": "Look up one order by its ID. Returns status, items, "
                   "delivery date and amount paid. Use before discussing a refund.",
    "input_schema": {
        "type": "object",
        "properties": {"order_id": {"type": "string", "description": "e.g. ORD-10482"}},
        "required": ["order_id"],
    },
}]

# MEMORY: the simplest form is the message history you resend each time
history = load_conversation(user_id)  # your function

response = client.messages.create(
    model="claude-opus-5-5",   # check the current models page; IDs change
    max_tokens=1024,
    system="You are a support assistant for Acme.\n\n<policies>\n"
           + "\n".join(policy_chunks) + "\n</policies>",
    tools=tools,
    messages=history + [{"role": "user", "content": "Can I return ORD-10482?"}],
)

The response comes back with stop_reason set to "tool_use" and a block like this:

{"type": "tool_use", "id": "toolu_01A...", "name": "get_order",
 "input": {"order_id": "ORD-10482"}}

You run get_order, send the output back as a tool_result block in a user message, and the model writes its answer. That round trip is still one augmented LLM. The block is the model plus its attachments, not a single HTTP request.

Two notes on the mapping:

• Retrieval has two shapes. Here your code fetched before the call. You could instead expose search_policies as a tool and let the model write the query. It is the same attachment, and the difference is who decides when to search.
• Memory has two shapes too. Resent history covers one conversation. For memory across conversations, Claude's API offers a memory tool ({"type": "memory_20250818", "name": "memory"}). It is client-side: Claude requests file operations under /memories and your application executes and stores them. The tool needs no beta header, though the SDK helper classes for it sit in the beta namespace.

Gotcha: the model never sees your function body. The tool's name, description and schema are the whole interface. Write them as you would a docstring for a new colleague.

Try one well-equipped call first

The same essay advises finding the simplest solution that works and adding complexity only when it measurably improves results. It notes that for many applications, a single optimised call with retrieval and in-context examples is enough.

Before you sketch anything multi-step, ask three questions:

1. Could the model answer if the right information were in the prompt? If so, fix retrieval.
2. Does it need a small, known set of actions or lookups? If so, add or sharpen tools.
3. Does it need something from before? If so, add memory.

If the answer to all three is yes and the task still fails after you have tuned the prompt, tool descriptions and retrieved context, you have evidence for something bigger. A hunch does not count as evidence.

The trade-off is real. One call is cheaper, faster and far easier to debug, but it also has a ceiling. Find that ceiling with test cases before you build past it.

Debugging one call: three suspects

When a single augmented call goes wrong, do not start by rewording the prompt. Dump the exact request and response and check three places in order.

A worked example: the assistant tells a customer the return window is 30 days, but your policy says 14.

1. Logged request -> <policies> contains only the shipping policy chunk.
   The returns passage was never retrieved.
2. Verdict: retrieval. The prompt and the tool were fine.
3. Fix: the search query or the chunking, not the system prompt.

Now a second failure with the same assistant: it calls get_order with {"order_id": "10482"} and your API returns nothing. The context was fine and so was the prompt. The schema's description did not state the full format clearly enough. That is a tool-definition bug. Fix it there, and return a useful error (set is_error to true on the tool_result) so the model can recover.

Also check stop_reason. A value of max_tokens means the output was cut off, possibly in the middle of a tool call. That is a budget problem, not a reasoning problem. Raise the limit and retry.

Best practice: log the full request for every call, including the system prompt, retrieved chunks, tool definitions and message history. Without that log you cannot tell the three suspects apart.

Key takeaways

• An augmented LLM is one model call extended with tools, retrieval and memory. You have already built all three.
• A tool round trip is still the same block. The block is the model plus its attachments.
• Start with one well-equipped call. Move past it only when test cases show it falls short.
• A failure in a single call comes from the prompt, the tool definition or the retrieved context. Read the logged request to find out which.
• The model sees only what is in the request. If the information or the interface is not there, the model cannot use it.