2 · One model call

What this step adds: a single ollama.chat call over the alert. No tools. It produces something that looks like analysis and is partly fabricated — which is the argument for everything that follows.

The new idea

Where the model runs is a governance decision, not a convenience.

Part 1 used a local model because it was free. In a bank the reasoning is different and much harder to argue with: a local model means no customer data leaves the building. No API key, no cloud account, no third-party processor to add to a supplier register, no cross-border data transfer to explain.

That is not an argument that local models are better. qwen2.5:3b is considerably weaker than a frontier model, and you will watch it fail on this page. It is an argument that the deployment location is part of the design, and worth deciding deliberately rather than defaulting to whatever has the easiest SDK.

The code

import ollama

MODEL = "qwen2.5:3b"

response = ollama.chat(
    model=MODEL,
    messages=[
        {"role": "system", "content": "You are a fraud analyst. Assess this alert."},
        {"role": "user", "content": f"Alert to triage: {alert}"},
    ],
)
print(response.message.content)

That is the whole step. One call, no tools, no loop.

Run it

cd part2
uv run --with ollama run.py

You get three paragraphs of confident-sounding fraud analysis.

Why it worked — and why that’s the problem

Read what it says about ALERT-003. On the run this step was built, the model wrote that COINFLARE was:

a newly registered merchant

Nothing in the alert says that. There is no registration date in the fixture, no merchant age, no field the model could have derived it from. It invented a fact, in the exact register a real analyst would use, and put it in the middle of otherwise reasonable analysis.

This is the concrete case for tool-grounding, and it is worth more than the abstract version. The model was not being reckless — it was doing what a text predictor does. Asked to assess a merchant, it produced the kind of sentence that follows “assess this merchant”, and plausible is not the same as true.

Two things follow:

A fabricated fact is more dangerous than a refusal. If the model had said “I don’t know when this merchant was registered”, you would go and look it up. Instead it produced something that flows into a case summary, gets read by an analyst, and is never questioned because it sounds exactly like the surrounding sentences.

No prompt fixes this. You can tell the model not to invent facts — Part 2’s case-writing instruction literally says “Do not invent any fact that is not in the case” — and it will still do it sometimes. The fix is structural: give it tools, so the facts come from somewhere real.

What’s still wrong

The model has no way to look anything up. Next: the first tool, and the thing Part 1 taught that is worth watching again in a bigger system — the model can only ask.