Try this first

Add one line to Rover’s loop and run a real task:

print(f"turn {turn}: {reply.usage.get('in')} in, {reply.usage.get('out')} out")

Watch the input number.

turn 1: 1,240 in, 180 out
turn 2: 3,100 in, 95 out
turn 3: 7,850 in, 210 out
turn 4: 12,400 in, 160 out

Output stays small. Input climbs, fast. By turn four you are paying for twelve thousand tokens
to get back a hundred and sixty.

Why

Lesson 1.5: the API is stateless, so every call resends the entire messages list.

Turn four sends turns one, two, and three along with it — including every tool result. One
read_file on a 2,000-line file is 25,000 tokens, and you pay for it again on every
subsequent turn, forever.

The pattern is worse than linear. If each turn adds n tokens, the total you pay across t
turns grows with the square of t. Doubling the length of a session roughly quadruples the
cost.

Two ceilings, not one

The hard ceiling: the context window. Anything from a few thousand tokens on a small
local model to a million on a large hosted one. On a big model you will rarely hit it in a
session you are watching; on a small local one you will hit it today.

The soft ceiling: usefulness. This one you hit constantly, and it has no error message.

Long before the window fills, quality drops. The important instruction from turn one is now
surrounded by forty thousand tokens of tool output. The model is not ignoring you. It is
attending to a haystack you built.

That is the “it felt sharp and then it did not” experience, and it is a context problem, not a
model problem.

What is actually in there

Instrument it before you optimise it:

from collections import Counter


def breakdown(messages):
    sizes = Counter()
    for m in messages:
        if m["role"] == "tool_results":
            for r in m["results"]:
                sizes["tool results"] += len(str(r["content"]))
        else:
            sizes[m["role"]] += len(m.get("content") or "")
            for tc in m.get("tool_calls", []):
                sizes["tool calls"] += len(str(tc.arguments))
    return sizes.most_common()

Run it on a real session. Almost every time, the answer is the same: tool results are
seventy to ninety percent of your transcript. Not the system prompt, not the conversation —
the output of read_file and bash, sitting there being resent.

That is where the money is, and it is why the next three lessons are all about tool results.

The transcript is resent in full on every turn, so cost grows with the square of the session
length. Tool results are almost all of it.

Try this before the next lesson

Add the token print and the breakdown. Run a ten-turn task.

Find your single largest block. It will be one tool result — probably a file read or a test
run. Ask yourself whether the model still needed it at turn ten. It almost certainly did not,
and you paid for it ten times.

PreviousNext