AI agents for web apps

What Claude Code is doing that you cannot see

This guide builds an agent three times over. First by hand, against the Claude Messages API, until the loop underneath every agent framework is something you could write from memory. Then under load, where that loop meets the five ways it goes wrong and the one place the SDK stops protecting you.

Then in Pydantic AI, where most of the hand-written parts disappear and the ones that remain turn out to be the ones that matter. Along the way it builds a chat assistant over PlanurLife's own data — vision, phrases, goals, tasks, journal — because an agent that reads one user's private rows and writes new ones is a harder problem than a weather demo, and the hard parts are exactly where the interesting mistakes live.

You have driven Claude Code hard for months. That is genuinely useful: you know what a good agent feels like, when one is thrashing, when a tool description is too vague, when the context has gone stale. What it has not given you is a single true intuition about the API underneath, and that gap is not the kind you can read your way across.

Claude Code is a harness: a program that owns the loop, manages the message array, caps the iteration count, compacts the context, retries the failures, and renders the result. Every behaviour you have learned to expect is a behaviour of that harness. None of it is in the API.

The chapters that follow dismantle three of those intuitions, each building on the one before.

The API is stateful. It is not. There is no session, no conversation ID, no server-side thread, and the follow-up call is the expensive one, not the cheap one.

The model runs the tool. It does not. Declaring a tool grants the model no ability whatsoever; it sends back a name and a JSON object, and nothing happens until your Python runs. If that sounds obvious, notice that it makes "how do I make my tool faster on Anthropic's servers?" a category error, and people ask it.

The loop terminates because the model behaves. It does not. while stop_reason == "tool_use" has no upper bound of any kind, and you have never seen a runaway agent in Claude Code because the harness caps it for you, silently, and tells you nothing.

By the end of chapter 12 you will have built the PlanurLife chat assistant twice. Once as about thirty lines of hand-rolled loop that you can read line by line, and once as a Pydantic AI Agent constructed at import time and shared across every request — with its tools scoped so that the signed-in user's id is not something the model can supply, guess, or be talked into changing. You will know why a tool_result goes in a user message, why max_iterations is a trap and not a safety net, why one AsyncSession on your dependency object is a correctness bug and not a style preference, and why a system_prompt freezes every conversation you have ever stored at the prompt it was born with.

Everything after that — putting the agent behind FastAPI, streaming it to a browser, weighing the framework against the rest of the field, and deciding when a protocol like MCP earns its place — is the rest of the guide. It rests entirely on the loop, so the loop comes first.

One request carries the whole world

Start here, because nothing downstream makes sense without it: the Claude API stores nothing between calls.

There is no session object. There is no conversation ID you can pass to continue where you left off. POST /v1/messages takes a request body containing the model, a max_tokens ceiling, an array of messages, and optionally a system prompt and a tools array — and returns exactly one assistant message.

Then it forgets you existed. The next request is unrelated to the last one in every way except that you chose to make its body look like a continuation.

So "the conversation" is not a thing the API has. It is a Python list that lives in your process, that you own, and that you grow.

Two roles go in it: user and assistant. Sending a follow-up means re-sending the entire prior array plus one more entry.

messages = [{"role": "user", "content": "Add a changelog task to my beta goal"}]

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=2048,
    system="You help the signed-in user plan. Never mention other users.",
    messages=messages,
)

messages.append({"role": "assistant", "content": response.content})
messages.append({"role": "user", "content": "Actually make it due Friday"})
# the next create() re-sends all three entries

Notice where system sits. It is a sibling of messages, not an entry inside it, and the same is true of tools. The request body the SDK serialises has them as three separate top-level keys, and the model sees them rendered in the order tools, then system, then messages.

That ordering is not incidental: it is exactly the prefix that prompt caching matches against, which the next section is about.

The follow-up call is the expensive one

Here is the consequence that catches everyone. Because the array is re-sent whole, turn N pays input tokens for turns 1 through N. The conversation does not accumulate cheaply on a server somewhere; it accumulates in your request body, and you re-buy it every time.

Chapter 5 builds a three-turn agent run — find a goal, create a task under it, report back. Here is what its request bodies actually measured on the wire, from the SDK's own serialiser:

call 1:  1006 bytes  messages=1
call 2:  1394 bytes  messages=3
call 3:  1741 bytes  messages=5

That is a tiny run with two tool calls and short results. Now picture the shape it has at turn forty of a real chat, where every tool result is a JSON dump of the user's goals and every assistant turn is a paragraph.

There is one more consequence, and it is the one that makes the rest of the guide possible. Because you own the array, you can do anything you like to it between calls.

Drop old turns. Summarise them. Persist them to Postgres and reload them tomorrow. Rewrite a tool result before the model sees it again.

None of that requires permission from the API, because the API has no opinion: it renders whatever array you hand it. Chapter 12 uses this to survive a process restart.

Caching pays for the prefix once

Pruning the array is one answer to the cost, and it has a floor: the turns you cannot drop still have to be re-sent, because the API has nothing to remember them with. Prompt caching is the other answer, and it works on exactly the part pruning cannot touch — the part that does not change from one turn to the next.

Look again at the render order: tools, then system, then messages. Turn after turn the front of that concatenation is identical — the same tool definitions, the same system prompt, the same frozen early messages. Only the tail grows. That identical front is a prefix, and caching lets you pay for a prefix once, then read it back at a tenth of the price on every later request that repeats it byte-for-byte.

It is not server-side memory, and it does not dent the statelessness this chapter is built on. The docs are precise about what gets cached:

"Prompt caching references the entire prompt — tools, system, and messages (in that order) up to and including the block designated with cache_control."

The transcript still goes on the wire whole. Caching changes what the already-seen part costs, not whether you send it.

You mark the end of the prefix with a cache_control field on the last block you want inside it:

system=[
    {"type": "text", "text": SYSTEM_PROMPT,
     "cache_control": {"type": "ephemeral"}},      # 5-minute lifetime by default
]

ephemeral is the only cache type; its default lifetime is five minutes, and {"type": "ephemeral", "ttl": "1h"} buys an hour. You get at most four breakpoints in one request.

The price is three multipliers on the base input rate, and they are the whole economic argument:

usage field	what it counts	price vs base input
cache_creation_input_tokens	the prefix, written — 5-minute TTL	1.25×
cache_creation_input_tokens	the prefix, written — 1-hour TTL	2×
cache_read_input_tokens	the prefix, read back on a later turn	0.1×
input_tokens	the uncached tail	1×

The write costs a 25% premium, once; every read after it is a 90% discount. On a 5,000-token prefix against Opus 5 at $5 per million input tokens, an uncached turn is $0.025; the turn that writes the cache is $0.031; each turn after it is $0.0025. Over a ten-turn chat that prefix costs about $0.054 cached against $0.25 uncached — roughly 78% off the part you had no choice but to re-send.

You find out whether it actually happened by reading the usage object the next chapter takes apart. A hit shows up as a non-zero cache_read_input_tokens:

u = response.usage
if u.cache_read_input_tokens:      # a hit: the prefix came back from cache
    ...

When both cache_read_input_tokens and cache_creation_input_tokens come back zero, nothing was cached — and the API returns no error to tell you so. The usual cause is the prefix being under the model's minimum: 512 tokens for Opus 5, 1,024 for Opus 4.8 and Sonnet 5, 4,096 for Haiku 4.5, below which a cache_control is silently ignored. The tiny three-call run above is far under every one of those; caching is a turn-forty concern, not a turn-one one.

The other cause is a breakpoint that moves. Anything at or before it that changes — a timestamp baked into the system prompt, or the newest user message if you marked that — produces a different prefix and a fresh, full-price write every time. So does changing the thinking or effort configuration the next chapter introduces: those are rendered into the prompt, and altering them starts the cache over. One thing that does not have to: on Opus 5 and Opus 4.8 you can append a {"role": "system"} message mid-conversation without invalidating the prefix — on Sonnet 5 you cannot — a reminder that a mid-array system message and the top-level system you cached are different objects.

Model ids are pinned, and there are no sampling knobs

Two small facts that will otherwise cost you a confusing hour. Current Claude model ids are bare strings — claude-opus-5, claude-sonnet-5, claude-haiku-4-5 — and despite the lack of a date suffix, each one is a pinned snapshot, not a moving pointer.

And temperature, top_p and top_k are not parameters of messages.create() at all in the 1.x Python SDK; passing one is a TypeError before any socket opens. If you learned this API on an older SDK, that is the first thing that will surprise you.

Content is a list of blocks

Another false intuition is smaller than statelessness but breaks more code, because it survives your first three test scripts and then fails on the fourth.

response.content is not a string. It is a list of typed blocks, discriminated on a .type field.

So is the content you send, whenever it is anything more interesting than plain text. The union runs both directions: TextBlock, ThinkingBlock, ToolUseBlock and friends come back; TextBlockParam, ToolUseBlockParam, ToolResultBlockParam and friends go out.

r = client.messages.create(model="claude-opus-5", max_tokens=1024,
                           messages=[{"role": "user", "content": "what am I working on?"}])

type(response.content) = list
len(response.content)  = 2
  content[0]: TextBlock      .type='text'
  content[1]: ToolUseBlock   .type='tool_use'
response.stop_reason   = 'tool_use'
response.usage         = 12 in / 34 out

content[0].text  -> 'Checking your goals.'
content[1].name  -> 'search_goals'
content[1].id    -> 'toolu_01'
content[1].input -> {'query': 'beta'}

content[1].text  -> AttributeError: 'ToolUseBlock' object has no attribute 'text'

That last line is the whole lesson in one error. response.content[0].text works right up until the model does something interesting, and then it either raises or — worse — quietly reads the wrong block. Dispatch on .type, always.

Why a list, and not a string

A single turn can hold several kinds of content at once — a paragraph of text, then two tool_use blocks, then more text — and each block carries structure the model produced, not prose you have to re-parse: a ToolUseBlock has an .id, a .name and an .input dict that arrived as JSON.

The reason that matters beyond tidiness is that the union is open. Beyond TextBlock and ToolUseBlock, the SDK's outbound union includes ThinkingBlock, RedactedThinkingBlock, ServerToolUseBlock, the several server-tool result blocks, and ContainerUploadBlock.

New types get added. Code that indexes content[0] breaks when a thinking block lands in front of the text; code that filters on .type does not.

The inbound direction mirrors it, with Param on the end of every name: you send TextBlockParam, ToolUseBlockParam, ToolResultBlockParam, ImageBlockParam. A plain string is accepted as shorthand for a single text block. {"role": "user", "content": "hi"} works because it is sugar, not a second representation.

A thinking block is just another block

Turn extended thinking on and the open union stops being hypothetical. A ThinkingBlock arrives at the front of response.content, ahead of the text and any tool_use — the exact case the previous section warned about, made real. Index content[0] and you read the model's scratch work where you meant to read its answer.

On the models this guide uses you turn thinking on with the adaptive configuration, and you set depth with an effort level rather than a token budget:

r = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,                                # thinking and text share this ceiling
    thinking={"type": "adaptive", "display": "summarized"},
    output_config={"effort": "high"},                # low | medium | high | xhigh | max; high is the default
    messages=[...],
)

The old spelling — thinking={"type": "enabled", "budget_tokens": N} — is not deprecated so much as rejected. The docs are flat about it: "Claude 4.7 and later models do not support it and reject requests that use it, returning a 400 error." The migration is one line: "remove budget_tokens, set thinking: {type: "adaptive"}, and control reasoning depth with output_config: {effort: ...}." This is the server-side half of chapter 7's finding — the SDK still serialises the budget_tokens shape without complaint, because older models still take it, so you learn it is wrong from the API rather than your editor.

Now look at what a returned thinking block actually contains — which, by default, is nothing you can read:

{ "type": "thinking",
  "thinking": "Let me break this down. The question has two parts, so I'll start with the simpler one…",
  "signature": "WaUjzkypQ2mUEVM36O2Txu...." }

{ "type": "thinking", "thinking": "", "signature": "EosnCkYICxIMMb3LzNrMu..." }

The first is display: "summarized". The second is the default, "omitted": "thinking blocks are returned with an empty thinking field. The signature field still carries the encrypted full thinking for multi-turn continuity." You never receive the raw chain of thought either way — summarised or empty, and a signature on both.

That signature is why a thinking block is not decoration you can drop. In a tool loop you must hand every thinking block back to the API unchanged, next to the tool_use block it came with — the same "append response.content verbatim" rule the loop chapter is about to make load-bearing, except the server now verifies it against the signature. Which turns the block-filtering habit from this chapter into a specific trap: "filtering on block.type == "thinking" alone silently drops redacted_thinking blocks and breaks the multi-turn protocol." Dispatch on .type, and remember that redacted_thinking is a .type too.

usage, and the three input numbers

Every response carries a usage object, and it has more than two fields. input_tokens is the uncached remainder only; cache_creation_input_tokens and cache_read_input_tokens sit alongside it, and the total input for a request is the sum of all three. If you are building a cost line for a conversation, summing input_tokens alone will quietly under-report as soon as caching is on.

stop_reason is the control signal

Every response carries a stop_reason. It is the single field that tells your program what to do next, and learning the enum before you write a loop is the difference between a loop you understand and a loop you copied.

There are seven values on the stable API. Read them off the SDK's own type alias:

['end_turn', 'max_tokens', 'stop_sequence', 'tool_use',
 'pause_turn', 'refusal', 'model_context_window_exceeded']

Value	What it means	What you do
end_turn	The model reached a natural stopping point.	Use the response. This is "done".
tool_use	The model emitted one or more tool_use blocks.	Run them, send the results back. This is the normal mid-loop state.
max_tokens	Generation hit your ceiling or the model's.	Raise max_tokens and retry. Treat the content as truncated.
stop_sequence	One of your stop_sequences was generated.	Read response.stop_sequence to see which.
pause_turn	A long-running server-side tool turn was paused.	Send the assistant content back to let it continue.
refusal	A safety classifier intervened. HTTP 200, not an error.	Read stop_details. Do not execute that turn's tool blocks.
model_context_window_exceeded	The array outgrew the window.	Treat the response as truncated; shrink the history.

In streaming mode stop_reason is null in the message_start event and arrives in message_delta. In non-streaming mode it is always populated.

A tool is a description, not a capability

A tool definition is three fields: a name, a description, and an input_schema that is ordinary JSON Schema. That is all. It is sent as part of the request body, rendered into the prompt ahead of the system prompt, and it grants the model exactly nothing.

Here is the PlanurLife tool surface this guide uses. Read the descriptions as prompt text, because that is what they are.

TOOLS = [
    {
        "name": "search_goals",
        "description": "Search the signed-in user's own goals by title. "
                       "Use before creating a task, to find the goal it belongs to.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query":  {"type": "string",
                           "description": "Words to match against the goal title."},
                "status": {"type": "string", "enum": ["active", "paused", "done"],
                           "description": "Only return goals in this status. "
                                          "Defaults to active."},
            },
            "required": ["query"],
        },
    },
    {
        "name": "create_task",
        "description": "Create one task under one of the signed-in user's goals.",
        "input_schema": {
            "type": "object",
            "properties": {
                "goal_id": {"type": "string",
                            "description": "Id of the goal, from search_goals."},
                "title":   {"type": "string", "description": "One line, imperative."},
            },
            "required": ["goal_id", "title"],
        },
    },
]

Look at what is not in either schema: user_id. That is deliberate, and it is the most important design decision in this chapter.

The signed-in user's id is not a parameter the model fills in. It is a value your handler already holds, closed over at the call site where the tool actually runs. A parameter the model can supply is a parameter the model can get wrong, and one that a hostile journal entry could try to talk it into changing.

def search_goals(user_id, query, status="active"):
    rows = [g for g in GOALS.get(user_id, []) if g["status"] == status]
    return [g for g in rows if query.lower() in g["title"].lower()]

def create_task(user_id, goal_id, title):
    owned = {g["id"] for g in GOALS.get(user_id, [])}
    if goal_id not in owned:
        raise PermissionError(f"goal {goal_id} does not belong to you")
    ...

create_task re-checks ownership even though goal_id came from a search_goals result that was already scoped. Do that. The model chose that string; it is untrusted input that happens to look plausible.

What comes back, and what you send back

When the model wants a tool, you get a ToolUseBlock with an id, a name, and an input dict — and stop_reason: "tool_use". You run whatever that names, and then you reply.

The reply is where habits from other APIs go to die. A tool result is a content block inside a user message.

There is no tool role in the Claude API. There is no function role. The docs are explicit about why it feels wrong: "Unlike APIs that separate tool use or use special roles like tool or function, the Claude API integrates tools directly into the user and assistant message structure."

{
  "role": "user",
  "content": [
    {
      "type": "tool_result",
      "tool_use_id": "toolu_A",
      "content": "[{\"id\":\"g-2\",\"title\":\"Run a half marathon\"}]"
    }
  ]
}

Only tool_use_id and type are required. content is optional — an empty tool result is legal — and it takes either a plain string or a list of blocks, including images. If the tool failed, add is_error: true and put something useful in content:

[
  {
    "type": "tool_result",
    "tool_use_id": "toolu_C",
    "is_error": true,
    "content": "No goal g-77 belongs to you. Call search_goals to list your goals."
  }
]

Write those messages as instructions, not as stack traces. "failed" teaches the model nothing; the sentence above tells it exactly what to do next, and it will.

Parallel tool use, and the failure that isn't an error

The model can ask for several tools in one turn. That produces one assistant message containing n tool_use blocks, and the correct answer is one user message containing n tool_result blocks, matched by tool_use_id.

How you execute them is entirely your business. The API doesn't prescribe an order, so asyncio.gather is fine for independent reads and a sequential pass is fine for writes.

{
  "role": "user",
  "content": [
    {"type": "tool_result", "tool_use_id": "toolu_A",
     "content": "[{\"id\":\"g-2\",\"title\":\"Run a half marathon\"}]"},
    {"type": "tool_result", "tool_use_id": "toolu_B",
     "content": "[{\"date\":\"2026-08-14\",\"text\":\"10k, felt strong\"}]"},
    {"type": "text", "text": "(that's everything since August)"}
  ]
}

Two formatting rules will 400 you if you break them. Tool result blocks must come first in the content array. Any text goes after all of them.

And the tool result message must immediately follow its tool use message, with nothing in between. If you see "tool_use ids were found without tool_result blocks immediately after", one of those two is what you did.

The loop

Everything is now in place, so here it is in one paragraph. Call the API. If stop_reason is not "tool_use", you are done. Otherwise: append the assistant message with response.content verbatim, run every tool_use block, append one user message holding every tool_result, and call again.

        ┌──────────────────────────────────────────────┐
        │  messages: list  (chapter 2 — you own this)  │
        └───────────────────┬──────────────────────────┘
                            │  whole array, every time
                            v
                  ┌──────────────────────┐
                  │  POST /v1/messages   │   + system, + tools
                  └──────────┬───────────┘
                             │  one assistant message
                             v
            ┌────────────────────────────────────┐
            │ response.content : list of blocks  │ ch. 3
            │ response.stop_reason               │
            └───────┬────────────────────┬───────┘
                    │                    │
       stop_reason  │                    │  stop_reason
       == tool_use  │                    │  != tool_use
                    v                    v
      ┌───────────────────────────┐   ┌───────────────┐
      │ append assistant message  │   │  return       │
      │   content VERBATIM        │   │  (ch. 6: not  │
      │ run each tool_use block   │   │   always done)│
      │ append ONE user message   │   └───────────────┘
      │   with every tool_result  │  ch. 4
      └───────────┬───────────────┘
                  │
                  └──────────► back to the top

And here it is in Python, complete, for PlanurLife:

def run_tool(user_id, block):
    fn = {"search_goals": search_goals, "create_task": create_task}[block.name]
    return fn(user_id, **block.input)

def chat(user_id: str, question: str, max_turns: int = 8):
    messages = [{"role": "user", "content": question}]
    for _ in range(max_turns):
        response = client.messages.create(
            model="claude-opus-5",
            max_tokens=2048,
            system="You help the signed-in user plan. Never mention other users.",
            tools=TOOLS,
            messages=messages,
        )
        if response.stop_reason != "tool_use":
            return response, messages

        results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            try:
                out = run_tool(user_id, block)
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": json.dumps(out)})
            except Exception as exc:
                results.append({"type": "tool_result", "tool_use_id": block.id,
                                "content": repr(exc), "is_error": True})

        messages.append({"role": "assistant", "content": response.content})
        messages.append({"role": "user", "content": results})
    raise RuntimeError(f"gave up after {max_turns} model turns")

Three things in that code matter, and only one of them is obvious.

messages is initialised outside the loop. If you build it inside, every iteration starts a fresh conversation and the model "forgets" — which looks like a model problem and is a Python problem.

response.content is appended verbatim. Not .content[0].text, not a rebuilt list of the blocks you cared about. The whole block list, text and tool_use and thinking alike.

Drop the tool_use blocks and the tool_result blocks you send next reference ids that appear nowhere in the array.

The except clause returns is_error: true instead of raising. In a chat assistant that is usually right: a PermissionError on create_task is information the model can act on, and the error text is what it has to act on.

Write it as an instruction. Chapter 6 covers the case where it is very wrong.

Run it, and the loop makes three HTTP calls, writes one row, and stops:

  turn: stop_reason='tool_use' blocks=['text', 'tool_use']
  turn: stop_reason='tool_use' blocks=['tool_use']
  turn: stop_reason='end_turn' blocks=['text']

final text: Added “Write the launch changelog” under Ship PlanurLife open beta.
HTTP requests made: 3
rows written: [('u-42', 'g-1', 'Write the launch changelog')]

What the array looks like when the loop is done

The loop appends exactly two entries per iteration, in that order. Here is the array it handed to the third and final call, printed from the SDK's serialiser, not from the code that built it:

[
  {"role": "user", "content": "Add a changelog task to my beta goal"},
  {"role": "assistant", "content": [
    {"text": "Let me find the goal first.", "type": "text"},
    {"id": "toolu_01", "type": "tool_use", "name": "search_goals",
     "input": {"query": "beta", "status": "active"}}
  ]},
  {"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_01",
     "content": "[{\"id\": \"g-1\", \"title\": \"Ship PlanurLife open beta\", \"status\": \"active\"}]"}
  ]},
  {"role": "assistant", "content": [
    {"id": "toolu_02", "type": "tool_use", "name": "create_task",
     "input": {"goal_id": "g-1", "title": "Write the launch changelog"}}
  ]},
  {"role": "user", "content": [
    {"type": "tool_result", "tool_use_id": "toolu_02",
     "content": "{\"id\": \"t-1\", \"goal_id\": \"g-1\", \"title\": \"Write the launch changelog\"}"}
  ]}
]

Read it top to bottom and the strict alternation is visible: user, assistant, user, assistant, user. That is the shape the loop guarantees for free. The question of whether the API tolerates two consecutive same-role messages never comes up in an agent: you are never in a position to send them.

system and tools are not in that array. They are re-sent as siblings on every one of the three calls, unchanged, because there is nowhere for them to persist.

You also have a free choice the API declines to make for you: execution order. Independent reads — search_goals and get_journal_entries in the same turn — are safe to run under asyncio.gather. Writes usually are not.

Run them in the order the blocks appear, and if an early one fails and you skip the rest, still return a tool_result for every skipped block with is_error: true and a one-line explanation. A missing result is a 400; a result saying "not executed, the preceding call failed" is something the model can reason about.

Termination is a property of the model's output

max_turns=8 above is not part of the pattern. I put it there because the pattern without it is unbounded, which is the next chapter.

When the loop does not stop

The loop's exit condition is a field in someone else's output. Your program's termination is delegated to a system you do not control, over a network, with no contract about how many turns it will take.

There are five ways that goes wrong and five distinct fixes. Confusing them produces the classic bad patch — a bigger iteration cap — which fixes exactly one of them.

1. No cap at all

I pointed the chapter 5 loop at a transport that returns tool_use forever and let it run:

CASE 1 — no cap. Model asks for the same tool forever.
  200 model turns, still stop_reason='tool_use'. Aborted by hand.
  last request body: 45,088 bytes, 399 messages

Two hundred turns, no slowdown, no warning, a request body that grew from one kilobyte to forty-five. In production: a handler that never returns and a bill that does not stop. Fix: a cap — which leads straight into the second failure, because most caps are written wrong.

2. A cap that ends silently

The sharpest failure here. A cap that breaks or returns cannot distinguish "finished" from "gave up":

CASE 2 — a cap that returns instead of raising.
  loop returned after 8 turns, no exception.
  final stop_reason = 'tool_use'   <- unfinished work that looks finished
  final text blocks = []

No exception. A normal return. A response object your caller will happily render — except it holds no text, because the model was mid-thought. The only thing separating this from success is the stop_reason you did not check.

The SDK's own Tool Runner has the same shape: hitting max_iterations ends the iterator cleanly with no exception, last message still reading stop_reason == 'tool_use'. At max_iterations=3 it reports "loop ended cleanly after 3 yields. NO exception raised."

3. A tool that always fails

Chapter 5 catches every tool exception and feeds it back as is_error: true. Good default; also a way to spend a lot on something that will never work. The model reads the error, adjusts, tries again, reads the same error, adjusts differently:

CASE 3 — a tool that always fails, answered with is_error.
  25 model turns, 25 failed tool calls, all billed.
  request 1 body 111 bytes -> request 25 body 7,503 bytes

Nothing in your logs but twenty-five identical exceptions, every turn priced against a bigger array than the last. Fix: count failures per tool per run. After the second or third in a row, replace the is_error with a terminal result — "this tool is unavailable for the rest of this conversation" — and let the model finish without it.

4. max_tokens truncating a tool_use block

max_tokens does not only truncate prose. It can cut a tool_use block off mid-JSON, leaving a turn that carries a half-written call:

CASE 4 — max_tokens truncated the tool_use block.
  loop exited after 1 turn(s), stop_reason='max_tokens'
  but the last block is 'tool_use' with input {'goal_id': 'g-1'}
  -> `while stop_reason == 'tool_use'` walks straight past a half-written call

title never finished generating, so the input dict is missing a required key. Which bug you get depends on the exit condition you wrote. Test stop_reason == "tool_use" and you fall out of the loop with unfinished work and no error.

Test "does this turn contain a tool_use block?" and you execute a truncated call — for create_task, a row with a field the model never chose. The SDK's own Tool Runner takes neither path in 1.2.0: its stop-reason table maps max_tokens to stop, so it ends the loop without running the half-written call. That spares you the phantom row, but it is only half the fix — the turn is still truncated, and noticing that is on you.

Fix, and it is the documented one: detect that the last content block is an incomplete tool_use and retry the whole turn with a higher max_tokens. Do not patch the arguments and do not execute the partial call.

5. pause_turn

Only if you add a server-side tool — web search, code execution — but worth knowing in advance: a long turn can come back as pause_turn, meaning not finished, not asking you for anything, just paused. Chapter 5's loop exits on it after one turn with no error, believing it succeeded.

Fix: send the assistant content straight back. The documented shape replaces the message array instead of appending — [original user message, assistant turn] — and adds no "please continue" text, because the API detects the trailing server-tool block.

Cap the continuations at five, the documented recommendation; the server's own tool loop stops at ten.

And refusal, which is not a loop bug at all

refusal deserves the last word, because it is the one case where the default handling is unsafe, not merely wasteful.

CASE 5 — refusal.
  loop exited after 1 turn(s), stop_reason='refusal'
  blocks still present: ['tool_use']
  -> HTTP 200, no exception; running those blocks writes a row the model never confirmed

A refusal is a normal HTTP 200: a safety classifier intervened mid-generation and the turn ended there. But the blocks generated before the intervention are still in response.content.

If your dispatch iterates content looking for tool_use without checking stop_reason first, it will execute them: a task written in a turn the model abandoned. The SDK's own runner treats refusal as terminal for exactly this reason — in its stop-reason table refusal maps to stop, so a turn that ends in a refusal never has its tool_use blocks run.

Fix: check stop_reason before you iterate content. Read stop_details for the category, and retry on a different model if you want a second opinion.

Nothing checks your work

There is a comfortable assumption in using a first-party SDK: that a wrong request shape fails locally, with a clear message, before it costs you anything. For the Anthropic Python SDK, that assumption is mostly false, and the exceptions are not where you would guess.

Almost every structured input to messages.create() is a TypedDict. messages, tools, tool_result blocks, thinking, tool_choice — all of them.

A TypedDict is a static-analysis construct. At runtime it is a plain dict and it enforces nothing at all.

So I sent six malformed requests through the real SDK and watched what happened to them:

things the SDK sends without a murmur:
  tool_result with no tool_use_id              SENT
  input_schema that is a string                SENT
  tool with no input_schema at all             SENT
  tool_result for an id nobody emitted         SENT
  assistant prefill (a trailing assistant turn) SENT
  budget_tokens thinking (the pre-1.0 spelling) SENT

All six serialised cleanly and went on the wire. A tool definition whose input_schema is the literal string "not-a-schema"; a tool_result with no tool_use_id to match it to anything; a trailing assistant message, which returns a 400 on every current model. The SDK's job is serialisation, and it did it faithfully.

What is caught is a much shorter list, and it is caught by Python's own function-signature machinery, not by anything schema-aware:

>>> client.messages.create(model="claude-opus-5", max_token=64, messages=[...])
TypeError: Missing required arguments; Expected either ('max_tokens', 'messages' and 'model')
or ('max_tokens', 'messages', 'model' and 'stream') arguments to be given

>>> client.messages.create(..., temperature=0.2)
TypeError: Messages.create() got an unexpected keyword argument 'temperature'

>>> client.messages.create(model="claude-opus-5", max_tokens=21334, messages=[...])
ValueError: Streaming is required for operations that may take longer than 10 minutes.

Read the first one carefully: misspelling max_tokens as max_token is diagnosed as a missing required argument, and the error never names the parameter you actually typed. The third is a real cliff: the SDK computes an expected duration from max_tokens and refuses non-streaming requests above it.

I binary-searched the boundary: 21,333 is accepted, 21,334 raises. Every example in Anthropic's own materials uses max_tokens=16000, the largest round number under it.

The instrument you are missing

The five things the SDK waved through are not undetectable. They are just not detectable at runtime. Point pyright at the same code and every one of them is an error before you run anything:

error: Argument of type "list[dict[str, str]]" cannot be assigned to parameter "tools"
       of type "Iterable[ToolUnionParam] | Omit" in function "create"
    Type "dict[str, str]" is not assignable to type "ToolUnionParam"
      "dict[str, str]" is not assignable to "ToolParam"

error: Argument of type "list[MessageParam | dict[str, str | list[dict[str, str]]]]" cannot
       be assigned to parameter "messages" of type "Iterable[MessageParam]" in function "create"
    Type "dict[str, str]" is not assignable to type "ContentBlock | TextBlockParam | …"
      "dict[str, str]" is not assignable to "ToolResultBlockParam"

2 errors, 0 warnings, 0 informations

The other half: what happens when the network is bad

The second thing nothing checks for you is how long a single call is allowed to take.

DEFAULT_MAX_RETRIES      = 2
DEFAULT_TIMEOUT          = Timeout(connect=5.0, read=600, write=600, pool=600)
INITIAL_RETRY_DELAY      = 0.5
MAX_RETRY_DELAY          = 8.0

Units are seconds. read=600 is ten minutes, and max_retries=2 means three attempts. Multiply:

  3 attempts x 600s read timeout + 1.5s of backoff = 1801.5s = 30.0 minutes

Thirty minutes inside a single create() before APITimeoutError reaches your code. Behind a FastAPI request handler with no override, that is not a resilient retry. It is a pinned worker, a browser that gave up twenty-nine minutes ago, and a connection pool slot held the whole time.

Two knobs fix it and you almost always need both. timeout= bounds a single attempt; max_retries= bounds the count. Set them on a derived client, not the shared one:

fast = client.with_options(timeout=20.0, max_retries=1)

  base    Timeout(connect=5.0, read=600, write=600, pool=600) 2
  derived 20.0 1 | same object: False

The exception hierarchy, and the trap inside it

AnthropicError
   APIError
      APIResponseValidationError
      APIWebhookValidationError
      APIStatusError
         BadRequestError   400
         AuthenticationError   401
         PermissionDeniedError   403
         NotFoundError   404
         ConflictError   409
         RequestTooLargeError   413
         UnprocessableEntityError   422
         RateLimitError   429
         ServiceUnavailableError   503
         OverloadedError   529
         DeadlineExceededError   504
         InternalServerError
      APIConnectionError
         APITimeoutError
   RetryableError

The shape trap is one line down from the bottom. APITimeoutError descends from APIConnectionError, which is a sibling of APIStatusError. So a handler chain built out of except APIStatusError will never catch a timeout, which is the failure you are most likely to actually see.

One more subtlety worth knowing: prefer exc.type to exc.status_code when you branch. APIStatusError parses body["error"]["type"] into a .type field, and billing_error and permission_error are both HTTP 403. The status code cannot tell "your card was declined" from "this key can't do that".

The retry policy itself is read from _base_client.py and is more deferential than you might expect: an x-should-retry header from the server overrides everything in either direction, then only 408, 409, 429 and 5xx retry. Backoff is 0.5s, 1s, 2s, 4s, 8s capped at 8, each multiplied by jitter in [0.75, 1.0), so jitter only ever shortens the wait. And if a retry-after header arrives with a sane value, the server's number wins.

Typed output

Two different things in this API constrain what the model produces. They aim at two different places, and the docs are unusually clear that they compose. Learn them as a pair.

output_config.format constrains the text the model writes: give it a JSON Schema and the reply's text block is guaranteed to validate against it. strict: true on a tool constrains the arguments the model passes when it calls one of your functions. Anthropic's phrasing is the one to remember: JSON outputs control "what Claude says"; strict tool use validates "how Claude calls your functions."

messages.parse(), and what it actually sends

The Python SDK gives you a pydantic-shaped front door. Define the shape you want as a model and hand the class — not a dict — to messages.parse():

class ExtractedTask(BaseModel):
    title: str = Field(description="One line, imperative.")
    goal_hint: str = Field(description="Words from the entry naming the goal it belongs to.")
    urgency: Literal["today", "this-week", "someday"]

class JournalExtract(BaseModel):
    mood: Literal["low", "flat", "good", "high"]
    tasks: list[ExtractedTask]

msg = client.messages.parse(
    model="claude-opus-5",
    max_tokens=1024,
    output_format=JournalExtract,
    messages=[{"role": "user", "content": entry_text}],
)
extract = msg.parsed_output        # a JournalExtract instance

output_format=SomeModel is pure client-side sugar. What goes on the wire is output_config.format, with your model's JSON Schema inlined — pydantic's $defs and title keys and all, plus additionalProperties: false injected for you because the API requires it on every object:

{
  "max_tokens": 1024,
  "model": "claude-opus-5",
  "messages": [{"role": "user", "content": "Shipped the beta signup flow today. ..."}],
  "output_config": {
    "format": {
      "type": "json_schema",
      "schema": {
        "type": "object", "title": "JournalExtract",
        "additionalProperties": false,
        "required": ["mood", "tasks"],
        "properties": {
          "mood": {"type": "string", "enum": ["low", "flat", "good", "high"], "title": "Mood"},
          "tasks": {"type": "array", "title": "Tasks",
                    "items": {"$ref": "#/$defs/ExtractedTask"}}
        },
        "$defs": {"ExtractedTask": { "...": "..." }}
      }
    }
  }
}

You get two guarantees at two layers. The server constrains decoding so the text validates; then parse() runs the text back through a pydantic TypeAdapter on the way out. When something slips through the first layer the second one raises where you can see it:

pydantic_core._pydantic_core.ValidationError:
  2 validation errors for JournalExtract
  mood
    Input should be 'low', 'flat', 'good' or 'high' [type=literal_error, input_value='elated']
  tasks
    Field required [type=missing, input_value={'mood': 'elated'}]

Not every JSON Schema feature is supported. Objects, arrays, strings, numbers, booleans, null, enum, const, anyOf, allOf, $ref/$defs and the common string formats are in. Recursive schemas are out, and so are numeric and length bounds — minimum, maxLength, multipleOf and friends.

The SDKs strip unsupported constraints from the schema they send and validate them client-side instead, which means a maxLength you wrote is enforced after generation, not during it. A new schema also pays a one-time compilation cost on its first request, then caches for 24 hours.

They compose, in one request

The two mechanisms are orthogonal, and nothing stops you sending both:

client.messages.create(
    model="claude-opus-5", max_tokens=1024,
    messages=[{"role": "user", "content": entry_text}],
    output_config={"format": {"type": "json_schema",
                              "schema": JournalExtract.model_json_schema()}},
    tools=[{
        "name": "create_task",
        "description": "Create one task under one of the signed-in user's goals.",
        "strict": True,
        "input_schema": {
            "type": "object", "additionalProperties": False,
            "properties": {"goal_id": {"type": "string"}, "title": {"type": "string"}},
            "required": ["goal_id", "title"],
        },
    }],
)

top-level keys sent: ['max_tokens', 'messages', 'model', 'output_config', 'tools']
tools[0].strict     : True
output_config.format.type: json_schema

strict: true is worth turning on for any tool that writes. Chapter 6's fourth failure mode — a create_task call missing its title — is precisely the class of thing it eliminates.

What a framework actually buys

Chapter 5's loop works, and it is thirty-one lines you would rather not retype for every agent you build. There are two steps up from it, and they buy different things. The first keeps you inside the SDK you already have; the second is a framework proper. Take them in order, because a framework's value stays invisible until you can see exactly what it replaced.

The thinnest step up: the Tool Runner

The Anthropic SDK ships the loop itself. client.beta.messages.tool_runner() is chapter 5's while with the bookkeeping removed, and the decorator @beta_tool turns a plain function into a tool definition — deriving the schema from the signature and the docstring, the same trick chapter 11 shows Pydantic AI using:

from anthropic import Anthropic, beta_tool

client = Anthropic()

@beta_tool
def search_goals(query: str, status: str = "active") -> str:
    """Search the signed-in user's own goals by title.

    Args:
        query: Words to match against the goal title.
        status: Only return goals in this status: active, paused or done.
    """
    return db_search(query, status)              # your Python, your process

runner = client.beta.messages.tool_runner(
    model="claude-opus-5",
    max_tokens=1024,
    tools=[search_goals],
    messages=[{"role": "user", "content": "find my beta goal"}],
    max_iterations=8,
)
final = runner.until_done()                      # drains the loop, returns the last message

The decorated function is still callable as itself; the tool definition is what the decorator adds beside it. Print it and the derivation is exact — description from the docstring summary, per-parameter text from its Args:, required from which parameters lack a default:

{
  "name": "search_goals",
  "description": "Search the signed-in user's own goals by title.",
  "input_schema": {
    "additionalProperties": false,
    "properties": {
      "query":  {"type": "string", "title": "Query",  "description": "Words to match against the goal title."},
      "status": {"type": "string", "title": "Status", "default": "active",
                 "description": "Only return goals in this status: active, paused or done."}
    },
    "required": ["query"]
  }
}

That is the same schema chapter 11 will show Pydantic AI deriving from an identical signature — the shared move under both frameworks. What the runner removes is real: the while, the stop_reason test, the block-type filter, the name-to-function dispatch, the two appends, the tool_result construction. What it does not add is the point. There is no typed dependency channel, no structured-output helper, no history management — and it does not call your tools' close hooks, so a stateful tool handed to it "leaks its resource," in the words of its own docstring. It is beta-namespaced, and in 1.2.0 it has quietly grown past "just the loop" — it now also brokers mcp_servers, context_management, output_config and fallbacks — but the framing holds: it is a harness, and you still host everything under it.

Its termination is chapter 6's cases, written out as a table in its source. max_iterations ends the iterator cleanly with no exception, the last message still reading stop_reason == "tool_use" — chapter 6's second failure, now inside someone else's code:

tool_use                       -> run_tools
pause_turn / compaction        -> resume   (hand the turn back unchanged)
end_turn / stop_sequence       -> stop
max_tokens                     -> stop      # the truncated tool_use block is NOT run
model_context_window_exceeded  -> stop
refusal                        -> stop      # the pre-refusal tool_use blocks are NOT run

That is the whole reason to read the source before reaching for it. You can now see that max_tokens and refusal both stop without executing — the safe choice chapter 6 argued for — and that nothing here raises when the model simply never finishes. The one thing the Tool Runner cannot give you is the reason this guide keeps going: a typed, per-request channel for the signed-in user's id that the model can never see. That is deps_type, it is chapter 10, and it is what a framework buys.

What Pydantic AI adds

Here is chapter 5's agent again, rewritten in Pydantic AI 2.36.0. Same tools, same conversation, same result.

from dataclasses import dataclass
from pydantic_ai import Agent, RunContext, ModelRetry

@dataclass
class Deps:
    user_id: str

chat = Agent(
    "anthropic:claude-opus-5",
    deps_type=Deps,
    instructions="You help the signed-in user plan. Never mention other users.",
)

@chat.tool
async def search_goals(ctx: RunContext[Deps], query: str,
                       status: str = "active") -> list[dict]:
    """Search the signed-in user's own goals by title.

    Args:
        query: Words to match against the goal title.
        status: Only return goals in this status: active, paused or done.
    """
    rows = await load_goals(ctx.deps.user_id)
    return [g for g in rows
            if g["status"] == status and query.lower() in g["title"].lower()]

@chat.tool
async def create_task(ctx: RunContext[Deps], goal_id: str, title: str) -> dict:
    """Create one task under one of the signed-in user's goals.

    Args:
        goal_id: Id of the goal, as returned by search_goals.
        title: One line, imperative.
    """
    if goal_id not in await owned_goal_ids(ctx.deps.user_id):
        raise ModelRetry(f"No goal {goal_id} belongs to you. Call search_goals first.")
    return await insert_task(ctx.deps.user_id, goal_id, title)

result = await chat.run("Add a changelog task to my beta goal", deps=Deps(user_id="u-42"))

output: Added “Write the launch changelog” under Ship PlanurLife open beta.
rows written: [('u-42', 'g-1', 'Write the launch changelog')]
usage: RunUsage(input_tokens=207, output_tokens=73, requests=3, tool_calls=2)

messages the framework built:
  ModelRequest   ['UserPromptPart']
  ModelResponse  ['TextPart', 'ToolCallPart']
  ModelRequest   ['ToolReturnPart']
  ModelResponse  ['ToolCallPart']
  ModelRequest   ['ToolReturnPart']
  ModelResponse  ['TextPart']

Three requests, two tool calls, one row written — the same run, message for message. What disappeared is the 31 non-blank lines of loop and dispatch from chapter 5: the while, the stop_reason test, the block-type filter, the run_tool name-to-function dict, the two appends, the tool_result construction, the try/except around the call. All of it, replaced by await chat.run(...).

Two things you wrote by hand also disappeared and are worth naming, because they were the error-prone ones. The tool schemas are gone — derived from your signatures and docstrings, which chapter 11 takes apart. And the mapping from a tool_use block's name back to a Python function is gone, along with the KeyError you get when they drift apart.

The Agent is a template, not a session

This is the shape you have to get right before anything else. An Agent is stateless. It holds a model, a set of tools, and instructions — a description of how to run, not a record of a run.

Nothing about a particular conversation or a particular user lives on it. That is what makes the module-level construction above correct, not sloppy: you build one at import time and every request in your FastAPI process shares it.

Per-request state travels on the run, not the agent: deps= for your typed payload, message_history= for the conversation. Chapter 10 is about why that is a security property and not an ergonomic one.

One import-time gotcha: Agent("anthropic:claude-opus-5") resolves the provider immediately, so a missing ANTHROPIC_API_KEY raises a UserError at import, not at first use. That is usually what you want in production and inconvenient in tests; defer_model_check=True postpones it.

Five ways to run it, ordered by how much of the loop you see

The execution surfaces all take the same twenty-odd keyword arguments and differ only in what they hand back.

run() is the coroutine you will use most. It runs the whole loop and returns an AgentRunResult with .output, .usage, .all_messages() and .new_messages(). You see the ending.

run_sync() is the same thing wrapped in an event loop. Use it in scripts and tests. Never in a FastAPI async def route — it blocks the worker for the entire turn.

run_stream() gives you the output as it arrives. It is an async context manager yielding a StreamedRunResult:

async with chat.run_stream(question, deps=Deps(user_id)) as s:
    async for chunk in s.stream_text(delta=True):
        yield chunk

   delta 'You '
   delta 'have '
   delta 'one '
   delta 'active '
   delta 'beta '
   delta 'goal.'
   complete: True

Note what is not in that output: the search_goals call that happened first. run_stream() shows you the answer being written, and nothing about how it was reached. It also coalesces by default — debounce_by=0.1 — so pass debounce_by=None if you want every delta.

run_stream_events() is a different API with a confusingly similar name, and it is the one that shows tool calls:

   PartStartEvent           ToolCallPart
   PartEndEvent             ToolCallPart
   FunctionToolCallEvent    ToolCallPart
   FunctionToolResultEvent  ToolReturnPart
   PartStartEvent           TextPart
   FinalResultEvent
   PartDeltaEvent           'have '
   PartDeltaEvent           'one '
   PartDeltaEvent           'active '
   PartDeltaEvent           'beta '
   PartDeltaEvent           'goal.'
   PartEndEvent             TextPart
   AgentRunResultEvent

That is the feed behind "Searching your goals…" in a chat UI. FunctionToolCallEvent fires when the model asks; FunctionToolResultEvent when your function returns; FinalResultEvent marks the point the output is committed; AgentRunResultEvent terminates. If you want a status line, this is the API — not run_stream().

iter() exposes the loop itself as a sequence of graph nodes:

    UserPromptNode
    ModelRequestNode
    CallToolsNode
    ModelRequestNode
    CallToolsNode
    End

Compare that to chapter 5's diagram: ModelRequestNode is the create() call, CallToolsNode is the dispatch block, and the pair repeating is the while. You will rarely need iter(), but it is worth running once, because seeing the same four node types you hand-wrote is what turns the framework from magic into a library.

Who calls your function

RunContext is usually introduced as "how you get a database handle into a tool", which makes it look like a dependency-injection container with extra ceremony. That framing loses the point. The point is this:

You never call your tool function. The framework does.

Look at what that means concretely. search_goals is not invoked from any line you wrote.

It is invoked from inside CallToolsNode, in a loop you did not start, in response to a JSON blob the model produced, with arguments the model chose. There is no call site of yours anywhere near it.

So the ordinary way of getting a value into a function — passing it as an argument — is not available. There is no argument list you control.

That is the whole reason RunContext exists. The framework needs a channel to hand your callback things that came from you and not from the model, and since it owns the invocation, that channel has to be a parameter it supplies. ctx is that parameter.

@chat.tool
async def search_goals(ctx: RunContext[Deps], query: str,
                       status: str = "active") -> list[dict]:
    ...

query and status come from the model. ctx comes from the framework. They arrive in the same call and they have completely different trust levels, and the signature is where that distinction lives.

ctx carries a lot besides deps: ctx.messages is the live history, ctx.usage the running totals, ctx.retry which attempt this is, ctx.run_step which model turn, ctx.tool_call_id and ctx.tool_name for the call in flight, and ctx.run_id / ctx.conversation_id for correlating a log line to a trace. But ctx.deps is the one you will touch every time.

deps_type is for the type checker; deps= is the runtime

These two look like a pair and are not. From Pydantic AI's own constructor docstring, on deps_type:

"The type used for dependency injection, this parameter exists solely to allow you to fully parameterize the agent, and therefore get the best out of static type checking."

Solely. Passing deps_type=Deps changes no runtime behaviour whatsoever.

What it does is make Agent[Deps, ...] a parameterised type, so that RunContext[Deps] in your tool signature type-checks and ctx.deps.user_id resolves to str in your editor. The runtime channel is deps= on the run:

result = await chat.run(question, deps=Deps(user_id=current_user.id))

Make Deps a @dataclass and put on it exactly what your tools need to serve this request: the user id, a session factory, whatever request-scoped clients you have. Chapter 11 says why a session factory and not a session.

The bug this chapter exists to prevent

Everything above becomes urgent when you combine it with chapter 9's other fact: the Agent is constructed once, at import time, and shared by every request in the process.

So consider the shortcut. You have a user_id. You have a tool that needs it.

deps looks like ceremony, and the tempting shortcut is to close over the value instead.

CURRENT_USER = None                      # set at the start of each request
bad = Agent("anthropic:claude-opus-5")

@bad.tool_plain
async def list_goals() -> list[dict]:
    """List the signed-in user's goals."""
    return await load_goals(CURRENT_USER)

async def handle(user_id: str):
    global CURRENT_USER
    CURRENT_USER = user_id
    return await bad.run("what am I working on?")

In a single-file script this works perfectly, and that is what lets it survive review. Now run two overlapping requests — two users, one process, an await inside the tool:

closure over a module-level variable, two overlapping requests:
  request for u-42 got: [{'id': 'g-7', 'title': 'Move to Lisbon'}]
  request for u-99 got: [{'id': 'g-7', 'title': 'Move to Lisbon'}]

User u-42 was shown user u-99's goals. Not an error, not an exception, not a 500 — a normal, successful, well-formatted answer containing another person's life plan. The second request overwrote CURRENT_USER while the first was suspended at an await, and the first one's tool read the new value when it resumed.

The same two requests through deps:

the same two requests, scoped through deps:
  request for u-42 got: [{'id': 'g-1', 'title': 'Ship PlanurLife open beta'}]
  request for u-99 got: [{'id': 'g-7', 'title': 'Move to Lisbon'}]

Notice, finally, where the correct version puts the check. The scoping happens in the one place it can be checked: inside your function, on a value that came from your session, one line before the query.

Tools, retries, and the concurrency trap

Two decorators register a tool. @agent.tool for a function whose first parameter is RunContext; @agent.tool_plain for one that needs nothing from the run. The difference is only the signature: a tool that reads the user's rows needs ctx, a today() helper does not.

Everything the model knows about your tool is derived from that function. The name comes from __name__, the description from the docstring's summary line, per-parameter descriptions from its Args: entries, types from the annotations, and optionality from the defaults.

Your docstring is shipped to the model. It is prompt text with a """ around it.

Which means an annotation is a prompt decision. Here is the same tool written twice:

async def loose(ctx: RunContext[None], status: str, since: str) -> list[dict]:
    """List goals.

    Args:
        status: the status
        since: the date
    """

{
  "name": "loose",
  "description": "List goals.",
  "parameters_json_schema": {
    "type": "object", "additionalProperties": false,
    "required": ["status", "since"],
    "properties": {
      "since":  {"type": "string", "description": "the date"},
      "status": {"type": "string", "description": "the status"}
    }
  }
}

The model must guess what a status is, what format a date takes, and cannot omit either. Now annotate it properly:

async def tight(ctx: RunContext[None],
                status: Literal["active", "paused", "done"] = "active",
                since: date | None = None) -> list[dict]:
    """List the signed-in user's goals, newest first.

    Args:
        status: Only goals in this status. Defaults to active.
        since: Only goals created on or after this day. Omit for all of them.
    """

{
  "name": "tight",
  "description": "List the signed-in user's goals, newest first.",
  "parameters_json_schema": {
    "type": "object", "additionalProperties": false,
    "properties": {
      "since": {"anyOf": [{"type": "string", "format": "date"}, {"type": "null"}],
                "default": null,
                "description": "Only goals created on or after this day. Omit for all of them."},
      "status": {"type": "string", "enum": ["active", "paused", "done"],
                 "default": "active",
                 "description": "Only goals in this status. Defaults to active."}
    }
  }
}

No required list at all now, because both have defaults. Literal became an enum; date became "format": "date". You changed no prompt text and materially changed what the model is able to get wrong.

Three ways a tool can fail, three different loops

=== raise ModelRetry ===
  output: I could not add that task.
  tool ran 2 time(s); history:
    UserPromptPart   add a task to g-77
    ToolCallPart     {'goal_id': 'g-77', 'title': 'x'}
    RetryPromptPart  No goal g-77 belongs to you. Call search_goals first.
    ToolCallPart     {'goal_id': 'g-77', 'title': 'x'}
    RetryPromptPart  No goal g-77 belongs to you. Call search_goals first.
    TextPart         I could not add that task.

=== raise ToolFailed ===
  output: I could not add that task.
  tool ran 2 time(s); history:
    ToolCallPart     {'goal_id': 'g-77', 'title': 'x'}
    ToolReturnPart   No goal g-77 belongs to you.
    ...

=== raise PermissionError ===
  the run RAISED PermissionError: goal g-77 does not belong to you
  tool ran 1 time(s); history:
    UserPromptPart   add a task to g-77
    ToolCallPart     {'goal_id': 'g-77', 'title': 'x'}

ModelRetry produces a RetryPromptPart and burns a retry from the budget. Use it when the model can plausibly fix the call: a bad id, an unparseable date, a search with no hits.

Exhaust the budget and you get UnexpectedModelBehavior: Tool 'create_task' exceeded max retries count of 1. The default budget is 1 for tools and 1 for output; set it with retries=3 or retries={'tools': 3, 'output': 1}.

ToolFailed produces an ordinary ToolReturnPart containing your message. The model sees it and moves on.

No retry is consumed, and it does not stop the model calling the tool again with different arguments. Use it for "this is broken and no argument will fix it".

Anything else propagates out of agent.run() and ends the run. That is the right default for a genuine bug, and chapter 6's advice about catching everything needs a second look here: swallowing a PermissionError into a retry loop turns a security check into twenty-five billed turns. Wrap the run in capture_run_messages() if you want the partial history when this happens.

The concurrency trap

Pydantic AI runs the tool calls in a single model turn concurrently by default. This is not a footnote; it decides your database design. Two tools, each doing 0.3 s of work:

  search_goals         enter  +0.007s
  get_journal_entries  enter  +0.007s
  search_goals         exit   +0.309s
  get_journal_entries  exit   +0.309s
  TOTAL 0.314s
  peak sessions checked out at once: 2

Now put a single SQLAlchemy AsyncSession on deps and let both tools use it — the natural shape if you are used to one session per request:

--- one AsyncSession on deps, two tools in one turn ---
  sqlalchemy.exc.InvalidRequestError:
    This session is provisioning a new connection; concurrent operations are not permitted

--- async_sessionmaker on deps, a session per tool call ---
  finished: done

That is a correctness bug, not a performance one. AsyncSession is not safe for concurrent use, and two tools awaiting on it in the same turn is exactly the illegal pattern.

@dataclass
class Deps:
    user_id: str
    sessionmaker: async_sessionmaker[AsyncSession]

@chat.tool
async def search_goals(ctx: RunContext[Deps], query: str) -> list[dict]:
    """Search the signed-in user's goals.

    Args:
        query: Words to match against the goal title.
    """
    async with ctx.deps.sessionmaker() as s:
        return await goals_for(s, ctx.deps.user_id, query)

There is a second reason to prefer this, and on a small box it is the bigger one. A chat turn takes tens of seconds, almost all of it waiting on the model.

Hold one session for the whole turn and you pin a pooled connection for those tens of seconds doing nothing; with SQLAlchemy's default pool_size=5 and max_overflow=10 you run out at roughly fifteen simultaneous chats. Session per tool call holds a connection only while a query runs.

Two related knobs, so you know they exist: @agent.tool(sequential=True) forces one tool to run serially — right for a tool that must not overlap with itself, wrong as a substitute for the sessionmaker. And a synchronous def tool is offloaded to a worker thread instead of run inline, so it will not stall the event loop but it also cannot use an AsyncSession. Async all the way down for anything touching Postgres.

Conversations that survive a restart

A chat assistant that forgets everything when the process restarts is a demo. Making it a product means the message array from chapter 2 has to live in Postgres, and Pydantic AI gives you the two halves of that: a stable serialisation, and a way to feed it back in.

The history is a list of two message types. ModelRequest is everything going to the model on one turn; ModelResponse is what came back. Each holds a list of parts, and the part kinds are the vocabulary of the whole framework: user-prompt, system-prompt, text, tool-call, tool-return, retry-prompt, plus thinking, instructions, file and a few more.

r1 = await chat.run(question, deps=Deps(user_id))
blob: bytes = r1.all_messages_json()          # -> a bytea / jsonb column

restored = ModelMessagesTypeAdapter.validate_json(blob)
r2 = await chat.run(next_question, message_history=restored, deps=Deps(user_id))

turn 1 output: You have one active beta goal.
all_messages_json() -> bytes 2263 bytes, 4 messages
new_messages()      -> 4 messages

resumed. all_messages(): 6 | new_messages(): 2
usage on the resumed run: RunUsage(input_tokens=68, output_tokens=19, requests=1)
run_id       : 01a02dea-0ff6-7167-8929-1f9ac1582e27
conversation_id: 01a02dea-0fd8-75fa-8c20-0bfb1ffc0132 (inherited from message_history)

Four facts in that output are worth pulling out.

all_messages_json() returns bytes, not a string: it goes straight into a bytea or through json.loads into jsonb.

all_messages() is everything including the restored history; new_messages() is the delta for this run. Store the delta per turn instead of rewriting the whole blob on every message, or your write cost grows quadratically with conversation length.

usage is per run, not cumulative. The resumed run above reports requests=1 even though it carried four messages of history. To accumulate across a conversation, thread usage= into the next run.

And every stored message carries a run_id and a conversation_id, with conversation_id inherited when you resume from history. That is the key you would otherwise have invented yourself for a conversation_turns table.

{
  "parts": [
    {"content": "what am I working on?",
     "timestamp": "2026-08-23T09:18:29.341438Z", "part_kind": "user-prompt"}
  ],
  "timestamp": "2026-08-23T09:18:29.341838Z",
  "instructions": "You help the signed-in user plan. Be terse.",
  "kind": "request",
  "run_id": "01a02dea-0fd8-75fa-8c20-0bfa4890420c",
  "conversation_id": "01a02dea-0fd8-75fa-8c20-0bfb1ffc0132",
  "metadata": null,
  "state": "complete"
}

Look at that stored record and note what it contains: the user's exact words. A PlanurLife conversation transcript is private life-planning data, and this is the moment it becomes a row in your database, not a value in memory. A companion guide, on observing and evaluating the agent, takes up what the traces of that data must not contain; the same instinct applies to the table.

instructions and system_prompt are not synonyms

Now, and only now, the distinction that would have been an arbitrary rule at construction time. On a single run these two behave identically. On a resumed run they do completely different things.

Two agents with different system_prompts, the second resuming the first's history:

system_prompt — parts stored in history:
    ModelRequest ['SystemPromptPart', 'UserPromptPart']
    ModelResponse ['ToolCallPart']
    ModelRequest ['ToolReturnPart']
    ModelResponse ['TextPart']
    ModelRequest ['UserPromptPart']
    ModelResponse ['TextPart']
   instructions the model was handed, per request: [None, None, None]

The first agent's prompt is a SystemPromptPart at the head of the transcript. When the second agent resumes that history, its own system prompt is never sent: the model keeps seeing the first one, embedded in the array, forever.

The same experiment with instructions:

instructions — parts stored in history:
    ModelRequest ['UserPromptPart']
    ModelResponse ['ToolCallPart']
    ModelRequest ['ToolReturnPart']
    ModelResponse ['TextPart']
    ModelRequest ['UserPromptPart']
    ModelResponse ['TextPart']
   instructions the model was handed, per request:
     ['C: you are the FIRST agent.', 'C: you are the FIRST agent.', 'D: you are the SECOND agent.']

No SystemPromptPart anywhere. Instructions live on ModelRequest.instructions, and only the current agent's value is applied to the new request. The historical values are inert record-keeping.

The agent is now complete: a stateless template, tools scoped through deps, failures that steer the loop instead of crashing it, and a conversation that survives a restart. What it does not have yet is a caller. The next chapter puts it, renamed planner (which is what the rest of the guide calls it from here), behind a FastAPI endpoint, where the module-level construction from chapter 9, the session factory from chapter 11, and the conversation_id from this one all stop being design choices and start being the reason it works under concurrency.

The agent inside FastAPI

Everything so far has run in a script. Put the same agent behind an HTTP endpoint and one property that was harmless becomes structural: a turn takes tens of seconds, and the request is open for all of them.

@router.post('/chat')
async def chat(body: ChatIn, user: User = Depends(current_user)) -> ChatOut:
    result = await planner.run(
        body.message,
        deps=Deps(user_id=user.id, sessionmaker=session_factory),
        message_history=await load_history(body.conversation_id),
    )
    await save_history(body.conversation_id, result.new_messages())
    return ChatOut(reply=result.output)

run() and run_stream() are coroutines and behave like any other awaited call. run_sync() does not: inside an async def route it blocks the event loop for the whole turn. A plain def tool is safe but limited: the framework offloads it to a worker thread via anyio.to_thread.run_sync, which keeps the loop free and puts it somewhere that cannot touch an AsyncSession.

The deps above carries the session factory from chapter 11, so the connection is held for the length of a query, not the length of a conversation. On those defaults, fifteen held-open sessions would be the whole pool. Capping how many turns run at once is a separate knob, Agent(max_concurrency=...), and it belongs to the model budget, not the pool.

Neon's pooled endpoint

The pooled endpoint is PgBouncer: "Neon uses PgBouncer to provide connection pooling, enabling up to 10,000 concurrent connections", in transaction mode, so "connections are returned to the pool after each transaction completes". Session-level features are unavailable there — SET/RESET, LISTEN/NOTIFY, SQL-level PREPARE/DEALLOCATE, session advisory locks. Scale-to-zero adds a cold start of up to about a second on the first connection after idle, so anything that runs per new connection runs more often than you would like.

SQLAlchemy's asyncpg dialect "makes use of asyncpg.connection.prepare() for all statements", caching 100 per connection, and warns that "asyncpg enumerates prepared statements in numeric order, which can lead to errors if a name has already been taken … if your application uses database proxies such as PgBouncer" — recommending NullPool plus a prepared_statement_name_func. Neon's own documentation says protocol-level prepared statements are supported on its pooler. Both are first-party and they are not obviously reconcilable.

When the browser goes away

A chat turn is the one request a user routinely abandons. Starlette and uvicorn handle this by throwing asyncio.CancelledError into the streaming generator. Here is a generator that yields every 100 ms, with the client killed at 0.6 s:

data: 0
data: 1
data: 2
data: 3
data: 4
curl: (23) Failure writing output to destination
--- curl killed, waiting for server log ---
>>> GENERATOR CANCELLED at i = 5

It fired one iteration past the client's last received chunk — chunks in flight, not an instant signal. Wrap the generator body in try/except asyncio.CancelledError with a finally: that closes sessions and records the partial turn; the alternative is a run that keeps calling the model, and keeps billing you, for a tab that no longer exists.

One distinction to keep straight. Pydantic AI has a first-party cancellation path — cancellation_token= on run(), AgentRun.cancel(), and the RunCancelled exception that carries the run's messages so far. That is for cancellations your own code requests.

A browser disconnect is not one of those: the exception's own documentation says external cancellation "keeps propagating as asyncio.CancelledError instead — it is never translated into this exception". Catch both, and expect the disconnect case to be the plain asyncio one.

Getting it to the browser

A turn produces text over tens of seconds. The transport for that is Server-Sent Events, which is HTTP with only a content type and a framing convention. Here is the real wire, captured from a FastAPI StreamingResponse with curl -N ($ marks a newline):

HTTP/1.1 200 OK\r$
cache-control: no-cache\r$
x-accel-buffering: no\r$
content-type: text/event-stream; charset=utf-8\r$
Transfer-Encoding: chunked\r$
\r$
event: delta$
data: {"text": "You "}$
$
event: delta$
data: {"text": "have "}$
$
event: done$
data: {}$
$

Four facts live in that dump. The media type is text/event-stream. A frame is an event: line, a data: line, and a blank line; the blank line is what ends the frame.

A data: value may not contain a raw newline, so everything is JSON-encoded. And the response is Transfer-Encoding: chunked, flushed once per yield.

You can write those strings yourself or let sse-starlette's EventSourceResponse do it, in which case you yield dicts. The measurable differences: it uses \r\n terminators (equally legal), sets cache-control: no-store, and sets x-accel-buffering: no for you — the header you have to remember with a raw StreamingResponse.

The turn cannot go through a Server Action

PlanurLife's browser never talks to FastAPI. Every backend call is server-side inside Next.js, and almost all of them are Server Actions. The chat turn cannot be one, and the reason is not about streaming:

Next.js dispatches Server Actions one at a time per client. If a user triggers three actions in quick succession, the second waits for the first to finish, then the third waits for the second.

A thirty-second agent turn therefore blocks every other action that client fires: saving a task, renaming a goal, all of it, queued behind the chat. The same page tells you where to go instead: "use a Route Handler for non-mutation requests". The streaming problem is real too — an action's response carries a return value in one Flight roundtrip, not a progressive channel — but the sequential dispatch is the part that would break the rest of the app.

Two buffers to defeat

The Route Handler is a credential-forwarding proxy: fetch FastAPI server-side with the user's session, return upstream.body — a ReadableStream — straight into a new Response(...). Three things make it actually stream instead of merely appear to.

1. Do not touch the body. One await res.text() anywhere and you have bought the whole turn before sending a byte.
2. Set the headers on the way out, instead of forwarding blindly: Content-Type: text/event-stream, Cache-Control: no-cache, no-transform, X-Accel-Buffering: no.
3. Forward the client's AbortSignal into the upstream fetch. Without it a closed tab stops at Next.js and FastAPI keeps generating.

Then the hops in front. Next's own streaming guide enumerates the layers, and two are in this stack: "Content Delivery Networks may buffer entire responses before forwarding them to the client", and "Gzip and Brotli compression can buffer chunks internally before flushing". What saves you at both is the media type, and that makes it more than cosmetic.

Caddy's reverse_proxy "partially buffers the response for wire efficiency" by default, and flushes immediately "if one of the following applies from the response: Content-Type: text/event-stream". Cloudflare keys on the same header: its Tunnel documentation says traffic "is buffered by default unless the origin server includes the Content-Type: text/event-stream response header".

Add no-transform and exclude the SSE path from compression at the Caddy layer, and every hop is accounted for.

The part that makes most of this unnecessary

If the Route Handler is going to exist anyway, there is a shorter road. Pydantic AI speaks the Vercel AI SDK's data-stream protocol natively, so the FastAPI side is one call:

from pydantic_ai.ui.vercel_ai import VercelAIAdapter

@app.post('/chat')
async def chat(request: Request):
    return await VercelAIAdapter.dispatch_request(
        request, agent=planner, deps=Deps(user_id), sdk_version=7,
    )

HTTP/1.1 200 OK\r$
x-vercel-ai-ui-message-stream: v1\r$
content-type: text/event-stream; charset=utf-8\r$
Transfer-Encoding: chunked\r$
\r$
data: {"type":"start"}$
$
data: {"type":"start-step"}$
$
data: {"type":"text-start","id":"b9e28d61-0477-495d-93a6-377224c91e4c"}$
$
data: {"type":"text-delta","delta":"You ","id":"b9e28d61-…"}$
$
data: {"type":"text-delta","delta":"have ","id":"b9e28d61-…"}$
$
data: {"type":"text-delta","delta":"two ","id":"b9e28d61-…"}$
$
data: {"type":"text-delta","delta":"active ","id":"b9e28d61-…"}$
$
data: {"type":"text-delta","delta":"goals.","id":"b9e28d61-…"}$
$
data: {"type":"text-end","id":"b9e28d61-…"}$
$
data: {"type":"finish"}$
$
data: [DONE]$

Note the differences from hand-rolled SSE: there is no event: field, because the protocol carries its type inside the JSON, and the stream ends with the sentinel data: [DONE]. On the React side that is useChat from @ai-sdk/react against a Python backend, with neither end hand-writing a protocol and the Route Handler reduced to forwarding credentials.

One footgun, and it is the sort that produces a silently wrong frame shape instead of an error: sdk_version defaults to 5, while npm ai is at 7.0.77. Pass sdk_version=7 explicitly.

The adapter's own source notes that v7's data-stream protocol equals v6's, so 6 and 7 emit the same wire; 5 does not.

The rest of the field

Pydantic AI is not permanently the answer. What follows is ranked by switch signal — the concrete event that should make you move — instead of by which is best, because none of them is best at the same thing. Versions are as of 2026-08-29: langgraph 1.2.11, anthropic 1.2.0, Managed Agents beta managed-agents-2026-04-01, claude-agent-sdk 0.2.148, langchain 1.3.18 on langchain-core 1.6.1, and openai-agents 0.22.0.

LangGraph is the agent as an explicit state machine whose state is checkpointed after every node. interrupt("Do you approve?") inside a node suspends the graph at that point, persists the state and waits indefinitely; you resume with Command(resume="yes") against the same thread_id, in a different process if you like, with langgraph-checkpoint-postgres keeping the checkpoints in Postgres. The cost is that you stop writing functions and start writing a topology — StateGraph, nodes, edges, conditional edges, a typed state dict, reducers — and the checkpointer is another set of tables to migrate and prune.

The signal: you need to suspend a run and resume it later, for human approval or because it outlives the request. "Show me the twelve tasks you propose before you write any" is that signal, and the most plausible near-term one here.

The Anthropic SDK's Tool Runner, from chapter 9, is that same loop with the bookkeeping removed and nothing else added. It costs you everything the framework was giving you: no typed dependency injection, no structured output, no history helpers, no instrumentation. Harness only; you host.

The signal: one Anthropic-only agent with a handful of tools and no need for a framework. Here that means giving up deps_type, which is the whole reason Pydantic AI is in the stack.

Managed Agents is the only option where Anthropic supplies the harness and the deployment. You POST /v1/agents once to create a persisted, versioned agent, then start Sessions against it; each session gets an Environment as its workspace — an Anthropic-managed cloud sandbox, or "a self-hosted sandbox on your own infrastructure" — where file operations and code execution run instead of on your box. It bills a session-hour on top of tokens; check the current number rather than trusting one here.

Scheduled deployments fire sessions on a cron cadence with no scheduler of yours involved. What it costs is data locality and eligibility, not latency: sessions store history and outputs server-side, and the docs are blunt that Managed Agents is "not currently eligible for Zero Data Retention or HIPAA Business Associate Agreement (BAA) coverage." The self-hosted Environment answers the first half of that; for private life-planning it is still a decision, not a detail.

The signal: a long-running or scheduled agent with a workspace, and no wish to own the box. A nightly "draft next week's tasks" job is that shape; an interactive assistant is not.

The Claude Agent SDK is Claude Code packaged as a library, and you host it — the same harness, with built-in Read, Write, Edit, Bash, Glob, Grep, WebSearch and WebFetch, plus context management, hooks, subagents, permissions and sessions. You call query(prompt, options) and it drives everything, in your own process.

It costs you fit: its native tool surface is a machine, not a database, so serving PlanurLife means supplying MCP tools and switching off most of what you came for. The signal: the agent's job is to operate on files and run commands — a different job from reading a database. "Read my repo and open a PR", not "read my goals".

The OpenAI Agents SDK is the one on this list built around a different provider. Its own description calls it "a lightweight yet powerful framework for building multi-agent workflows" that is "provider-agnostic, supporting the OpenAI Responses and Chat Completions APIs, as well as 100+ other LLMs" — so it can drive Claude, but its centre of gravity is elsewhere. It is the OpenAI-ecosystem peer of Pydantic AI: you host it, you write the tools, and its distinctive primitives are handoffs between agents and guardrails around them.

The signal: the design is genuinely multi-agent, with specialised agents handing work to each other, or you are already OpenAI-centric. For one typed agent over your own SQLAlchemy models in an Anthropic-anchored stack there is no signal — which is why this guide builds on Pydantic AI and mentions this one only to place it on the map.

LangChain, judged fairly, not fashionably: it is neither dead nor the orchestration layer any more. langchain-core is a live dependency of the LangGraph stack — that way round, not the other — and LangChain's role now is provider and tool abstraction plus the integration catalogue. It costs you distance from the provider, which is exactly what makes a failing tool call harder to debug than it should be.

The signal: you need a component that exists only as a LangChain integration and wrapping it yourself would cost more than the dependency. That is a component decision, not an architectural one, and for tools that are your own SQLAlchemy models there is no signal at all.

When MCP earns its place

The last question is whether any of this belongs behind a protocol. The Model Context Protocol is the obvious candidate, and the honest first answer is a one-liner: the agent already calls the application's own tools directly.

That is right, and stated too weakly. Here is the stronger version — the job being to let you make the call, not to sell you a protocol.

What the current revision actually is

If you remember MCP from 2025, most of what you remember is wrong. Revision 2026-07-28 (the previous was 2025-11-25) removed the foundations:

• Sessions are gone, with the Mcp-Session-Id header; cross-call state becomes server-minted handles passed as ordinary tool arguments.
• The handshake is gone — no initialize, no notifications/initialized. Every request carries its protocol version and client capabilities in _meta, and a tools/list without that block is rejected -32602 Invalid params: in a stateless protocol the per-request _meta is the handshake.
• server/discover is new, and servers MUST implement it.
• Server-initiated requests are gone: "servers do not initiate JSON-RPC requests and clients do not send JSON-RPC responses."
• Roots, Sampling and Logging are deprecated on a twelve-month window.

Two transports remain, stdio and Streamable HTTP. HTTP+SSE is formally deprecated.

The reference SDK ships all of this

None of that is paper-only. pip install mcp today gives you version 2.1.1 of the official Python SDK — the package whose repository is modelcontextprotocol/python-sdk — and it has adopted 2026-07-28 wholesale. Import it and the new, stateless protocol is right there in the constants:

LATEST_PROTOCOL_VERSION       = 2026-07-28
PROTOCOL_VERSION_META_KEY     = io.modelcontextprotocol/protocolVersion
CLIENT_CAPABILITIES_META_KEY  = io.modelcontextprotocol/clientCapabilities

-32602  INVALID_PARAMS
-32600  INVALID_REQUEST
-32021  MISSING_REQUIRED_CLIENT_CAPABILITY
-32022  UNSUPPORTED_PROTOCOL_VERSION

Those last codes are worth separating, because they catch different mistakes. -32602 is JSON-RPC's "Invalid params" — the code a server returns when the required _meta block is missing outright — and it is a distinct code from -32600 "Invalid request", which is why "invalid request parameters" is the wrong gloss for it. -32021 MISSING_REQUIRED_CLIENT_CAPABILITY is the newer, narrower one: it fires when clientCapabilities is present but does not declare something the request needs. Block absent is -32602; capability short is -32021.

The latency argument against MCP is wrong

Worth killing first, because it is the reason people usually give. Same function body, 200 calls each:

direct python call     median      0.17 us
MCP in-process         median    220.10 us
MCP over stdio         median    841.62 us

client connect (spawn + discover): 677.9 ms, one-off

Five thousand times slower, and completely irrelevant: 0.8 ms against a turn that takes seconds is noise. The number that does matter is the one-off 678 ms to spawn a subprocess and discover — a real cost if you were tempted to start a server per request, and an argument for Streamable HTTP against a long-lived process, not stdio in a web app.

What you actually lose is RunContext

Here is the in-process tool definition, taken from a real span dump — this is the literal JSON the model receives:

{"type":"function","name":"search_goals",
 "description":"Find the user's goals matching a query.",
 "parameters":{"additionalProperties":false,
   "properties":{"query":{"type":"string"}},
   "required":["query"],"type":"object"}}

The Python signature is async def search_goals(ctx: RunContext[Deps], query: str). ctx is not in the schema. The authenticated user_id and the live database session arrive out of band through the dependency; the model cannot name them, let alone change them.

Tenant isolation is code-controlled. Now the same capability over MCP, as its server advertises it:

{"name":"create_task",
 "inputSchema":{"type":"object",
   "properties":{"goal_id":{"type":"string"},"title":{"type":"string"}},
   "required":["goal_id","title"]}}

There is no ctx across a process boundary. Identity now has to travel in-band, and there are two ways to send it — the fork the rest of this argument turns on.

The first is an OAuth token the server validates. It is the correct answer, and a genuine amount of work: the spec makes the server an OAuth 2.1 resource server that MUST implement RFC 9728 protected-resource metadata, and its clients MUST send RFC 8707 resource indicators and MUST use PKCE (Proof Key for Code Exchange). OAuth 2.1 is itself still an IETF draft, not a finished RFC.

The second is to pass identity as an ordinary tool argument — which hands tenant isolation to the model. MCP's own primitive table says so in as many words: tools are "Model-controlled" by design, against resources (application-controlled) and prompts (user-controlled).

That is what the deferral actually rests on. For an app holding one person's private plans, direct calls are safer, not just sufficient, because RunContext keeps the tenant boundary where the model cannot reach it.

The day it earns its place

MCP is a distribution question, not a capability one. The agent can already call these functions; a protocol changes who else may call them and what they must authenticate as. So the signal is specific: the day you want a tool surface for clients you do not own — Claude Desktop, Claude Code, Cursor, somebody's own agent pointed at their own plans — MCP is the only answer.

On that day the work is not "expose the functions". It is "build an OAuth 2.1 resource server", and that is the project. Until then you have the better end of the trade: an agent whose tools are ordinary typed Python functions, running in the same transaction as the request that called them, with the tenant boundary in a place the model cannot reach. Whether a change to that agent has actually made it better — how you trace it, score it, and turn real failures into regression cases — is the subject of a companion guide on evaluating agents; this one ends where the agent is built.

Further reading

Everything asserted in this guide was established against one of the following; the SDK surfaces and version floor were last re-checked on 2026-08-29. The first group is the one to bookmark: this half of the ecosystem changes faster than any guide, and the version floor named in "The rest of the field" is the honest limit of what was checked.

The protocol

• Tool use overview and Handling tool calls — the tool-definition shape, and the sentence explaining why a tool_result lives in a user message.
• Handling stop reasons — the enum, and the documented recovery for a tool_use block truncated by max_tokens.
• Parallel tool use — read the Troubleshooting section specifically. It is the failure that never raises.
• Structured outputs — output_config.format, strict: true, and the JSON Schema subset that is actually supported.
• Extended thinking and Adaptive thinking — the budget_tokens → adaptive + effort migration, why it 400s on the 5-series models, and why display defaults to omitted.
• Prompt caching — the answer to the cost consequence of statelessness, and the list of things that silently invalidate a prefix.
• Pricing — check it instead of trusting any figure here. Sonnet 5's price changed while this guide was being written.

The framework

• Pydantic AI — and read the changelog before upgrading: the 2.x capabilities argument replaced four constructor parameters that most tutorials still use.
• LangGraph — worth reading the interrupt documentation now, before you need it; it is the one capability that would make you switch.

The application layer

• MCP specification, revision 2026-07-28 — start at the changelog; if you know MCP from 2025, most of what you know has been removed.
• MCP Python SDK — pip install mcp (2.1.1); import it to see the 2026-07-28 protocol version, the _meta keys and the error codes the chapter uses.
• Next.js Server Actions and Streaming — the one-at-a-time dispatch rule, and the list of layers that buffer a stream.
• Neon connection pooling and SQLAlchemy's asyncpg dialect — read both, and notice that they disagree about prepared statements behind PgBouncer.