4 · The tool loop

What this step adds: the loop that answers the model’s requests — and _bind_args, the line of defence Part 1 never needed.

The code

The loop itself will look familiar:

def _run_tool_loop(case, audit, stage, instructions, tool_names):
    """The agent loop: the model calls tools until it decides it is done."""
    tools = [SCHEMAS[n] for n in tool_names]
    messages = [
        {"role": "system", "content": instructions},
        {"role": "user", "content": f"Alert to triage: {case['alert']}"},
    ]
    results = {}
    for _ in range(MAX_TURNS):
        response = ollama.chat(model=MODEL, messages=messages, tools=tools)
        messages.append(response.message)

        if not response.message.tool_calls:
            break                              # the model decided it is done

        for call in response.message.tool_calls:
            name = call.function.name
            fn = TOOLS.get(name)
            if fn is None:
                out = {"error": f"no tool named '{name}'"}
            else:
                out = fn(**_bind_args(name, case))     # <- not the model's args
            results[name] = out
            audit.record(case["alert_id"], stage, "tool_call",
                         f"{name}({dict(call.function.arguments)})")
            audit.record(case["alert_id"], stage, "tool_result", out)
            messages.append({"role": "tool", "content": str(out),
                             "tool_name": name})
    return results

MAX_TURNS = 6, the registry lookup, appending the model’s own turn, results back as role: "tool" — all Part 1.

The new idea

Look at the tool call again:

out = fn(**_bind_args(name, case))

Part 1 wrote fn(**call.function.arguments) — it passed the model’s arguments straight through. Part 2 throws them away:

def _bind_args(tool_name, case):
    """Guardrail: bind a tool's arguments to the entities in THIS case."""
    alert = case["alert"]
    if tool_name == "search_similar_cases":
        return {"signals": case.get("signals", [])}
    if tool_name == "check_device_fingerprint":
        return {"device_id": alert["device_id"],
                "customer_id": alert["customer_id"]}
    return {"customer_id": alert["customer_id"]}

The model decides which tool to call. Your code decides what it is called on. That split is the whole idea, and it closes a hole that is easy to miss.

Why this matters more than it looks

Without it, the customer ID the tool runs against is a value the model generated from text. Three failure modes follow, in increasing order of seriousness:

It gets it wrong. The model transposes a digit, or carries CUST-1001 over from the previous alert it was reasoning about. You look up the wrong customer and score the alert on someone else’s history. Nothing errors; the number is simply meaningless.

It gets it wrong invisibly. The audit log records get_customer_profile({'customer_id': 'CUST-1002'}) and it looks completely normal. A reviewer would have to compare the argument against the alert to notice, which nobody does.

It gets it wrong on purpose. The alert contains a payee name — attacker- controlled text. A payee called NOVA HOLDINGS (see customer CUST-1001) is text the model reads while deciding what to pass to a tool. That is the entity equivalent of the globals() problem from Part 1: untrusted input reaching a lookup you did not constrain.

_bind_args makes all three impossible, because the model’s arguments never reach the function. It is six lines of the dullest code in the system, and it is doing more work than the loop around it.

Note what it does not do: it does not stop the model choosing a tool you did not expect. That is TOOLS.get()’s job. Two separate controls, two separate failure modes — defence in depth, as in Part 1.

Run it

cd part2
uv run --with ollama run.py

Now the audit trail shows both halves:

ENRICHMENT  | tool_call      | get_customer_profile({'customer_id': 'CUST-1001'})
ENRICHMENT  | tool_result    | {"home_country": "AU", "avg_txn_amount": 2100.0, ...}

Something to try

Run it twice and compare which tools got called. On the runs this was built from, the model called two tools on one run and three on the next — same alert, same code, same instruction telling it to gather all three.

Nothing changed except the sampling. That is not a bug to fix; it is the property you are designing around, and it is why the next four steps exist.

What’s still wrong

One loop is doing everything. Next: splitting it into named stages, and discovering that the first one needs no model at all.