From token predictor to agent.

All the model does is guess the next token.

Take away the chat window, the tools, and the personality, and what is left is a function. Each call is one forward pass. You give it a sequence of tokens, and it returns one score for every entry in its vocabulary (the logits). Then a sampler picks one token. You add that token to the end of the sequence and call the function again. There is nothing else. The model has no memory outside the prompt, no way to act on anything, and no goals.

# the whole model, seen from outside
def generate(tokens):
    while True:
        logits = model(tokens)            # one forward pass
        tok = sample(softmax(logits))     # pick one token
        if tok == END_OF_TURN:            # stopping is a token too
            return tokens
        tokens.append(tok)                # append, repeat

You see a chat. The model sees one long string.

There is no "user" object and no "assistant" object inside the weights. Before every call, a chat template joins the whole conversation into one long string, and special tokens mark where each turn begins and ends. The model learned this layout in post-training. So when the string ends at the start of an assistant turn, the most probable continuation is a helpful reply, followed by the end-of-turn token, which tells the server to stop.

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
What's the weather in Tokyo?<|im_end|>
<|im_start|>assistant
It's 22°C and sunny in Tokyo right now.<|im_end|>
▲ generation stops here: the model emitted the end-of-turn special token

# a chat template is string formatting, nothing more
def render(messages):
    out = ""
    for m in messages:
        out += f"<|im_start|>{m.role}\n{m.content}<|im_end|>\n"
    return out + "<|im_start|>assistant\n"  # cue: your turn

A tool call is text the model learned to write.

Two things turn a chat model into one that "uses tools". First, the tool definitions are sent as text. The JSON Schemas you pass to the API are written into the prompt next to the system prompt, so the model can read what each tool does. Second, post-training teaches a strict output format. The model sees thousands of example conversations where the correct assistant turn is a machine-readable block, wrapped in special tags, that names a tool and its arguments. The idea goes back to ReAct prompting and Toolformer. In today's models, the format is trained into the weights.

# one of thousands of supervised examples
prompt = render(system + tool_schemas + "user: weather in Tokyo?")
target = "<tool_call>{\"name\": \"get_weather\", " \
         "\"arguments\": {\"city\": \"Tokyo\"}}</tool_call>"
# training makes this exact shape the most probable reply

The agent is a while loop.

The core of every coding agent is much smaller than most people expect. A harness sends the transcript and the tool schemas, checks stop_reason, runs the requested function, adds the result to the transcript as a new message, and calls the API again. The model reads the tool result as new tokens and predicts what to do next. Step through one task below.

TOOLS = {"read_file": read_file, "run_bash": run_bash}

def agent(task):
    messages = [{"role": "user", "content": task}]
    while True:
        r = api.messages.create(model="...",
                                tools=SCHEMAS, messages=messages)
        messages.append({"role": "assistant", "content": r.content})
        if r.stop_reason != "tool_use":
            return r                      # done: plain text answer
        results = []
        for call in r.content:            # may be several in one turn
            if call.type == "tool_use":
                out = TOOLS[call.name](**call.input)   # the real world
                results.append({"type": "tool_result",
                                "tool_use_id": call.id,
                                "content": out})
        messages.append({"role": "user", "content": results})

Three limited parts add up to an agent.

None of the pieces can do much alone. The predictor has judgment but no way to act. The tools can act but have no judgment. The loop has neither. It only passes tokens back and forth. Put together, they form a feedback cycle. The model perceives the world through tool results, which arrive as tokens. It decides by predicting the next token. It acts through the harness. Run that cycle enough times and you get what we call agency.

This is why the harness matters so much. Claude Code, a CI bot, and a chat app can run the same weights and still behave very differently, because the harness decides which tools exist, what the system prompt says, and how the context window is managed. I took two production harnesses apart in the agent harness explainer. This page covers the mechanics underneath that one.