Step 5 · Judgment vs code

What this step adds: a deterministic decision rule beside the model’s prose — and the design principle the whole course turns on.

The new idea

Deterministic means the same input always produces the same output. Ordinary code is deterministic: amount > 1000 fires at 1001 and not at 999, every time, forever. Language models are not — the same prompt produces different words on different runs, as you saw in step 4.

So the question for any consequential system is: which parts may vary, and which must not?

The code

Create step5.py:

import ollama

def get_transaction(txn_id):
    records = {"T-1002": {"amount": 4200, "country": "Nigeria", "card_present": False}}
    return records[txn_id]

def decide(txn):
    if txn["amount"] > 1000 and not txn["card_present"]:
        return "FLAG"
    return "ALLOW"

schema = {
    "type": "function",
    "function": {
        "name": "get_transaction",
        "description": "Look up a transaction's amount, country and card_present flag by its id.",
        "parameters": {
            "type": "object",
            "properties": {"txn_id": {"type": "string"}},
            "required": ["txn_id"],
        },
    },
}

messages = [{"role": "user", "content": "Is transaction T-1002 suspicious?"}]

while True:
    response = ollama.chat(model="qwen2.5:3b", messages=messages, tools=[schema])
    messages.append(response.message)
    if response.message.tool_calls:
        for call in response.message.tool_calls:
            txn = get_transaction(call.function.arguments["txn_id"])
            print("TOOL DATA:  ", txn)
            print("CODE SAYS:  ", decide(txn))
            messages.append({"role": "tool", "content": str(txn)})
    else:
        print("MODEL WORDS:", response.message.content)
        break

decide() is four lines and contains no AI. It encodes one policy rule: over $1,000 and card-not-present → FLAG.

Run it

uv run step5.py

Expect:

TOOL DATA:   {'amount': 4200, 'country': 'Nigeria', 'card_present': False}
CODE SAYS:   FLAG
MODEL WORDS: This transaction looks suspicious because...

Run it three times and compare. CODE SAYS: FLAG is identical every run. MODEL WORDS is different every run. Same file, two mechanisms, opposite reliability — printed side by side.

Occasionally the model won’t call the tool at all

Once in a while it asks you for the transaction details instead of looking them up — you’ll see MODEL WORDS with no TOOL DATA line. Just run it again. In testing this happened roughly one run in six, and it’s this page’s lesson arriving early: the model’s behaviour varies, and anything that must not vary cannot live on the model’s side of the line.

Why both exist

It’s tempting to conclude “just use code, then.” But decide() only knows the rule you wrote. It can’t notice that a transaction is odd for this customer, weigh three weak signals none of which crosses a threshold, or explain itself to a human in plain language. That’s what the model is for.

The division of labour, and it’s the most load-bearing sentence in this course:

The model supplies judgment over the facts. The code supplies the rules that must hold. If a rule must never be missed, it goes in code — a prompt is a request, not a guarantee.

Try it yourself: add “always flag transactions over $1000” to the prompt instead, delete decide(), and run it ten times. It will comply most of the time. “Most of the time” is not a control — and no bank’s regulator, and no serious engineer, will accept a policy threshold that lives only in a prompt.

What’s still wrong

The system now produces a reliable recommendation — and nothing stops it from acting on it. There is no boundary between “the system concluded FLAG” and “the card gets blocked.” In anything with real consequences, that boundary is a person. Step 6 builds it.

Step 6 — the human gate →