Tutorial 09/13PatternsAdvanced11 min

Building an agent harness

Claude Code, Codex and Cursor all ship a classifier that decides whether a tool call is safe to run without asking you. Those classifiers have lived inside closed harnesses. A model that costs $0.042 per million tokens and answers in under a second makes that pattern available to anyone's agent.

By the end you'll be able to

Gate dangerous tool calls before they execute
Select a tool from a large catalogue in one request
Detect loops and stalled runs from agent state
Keep the harness honest about what it cannot see

On this page

The control layer
Gating dangerous tool calls
Tool selection at catalogue scale
Loop detection and trace grading
The thing a harness cannot do
Security, restated because it matters here most
Next

The control layer#

Anthony Maio's framing is the most precise one written about where Jev belongs, and it is not "doing the task":

A generative model drafts, plans, or explains. Jev supplies bounded semantic judgments. Code handles state, arithmetic, policy, permissions, and side effects. Humans take the ambiguous or high-risk cases.

— Anthony Maio

He calls Jev "a learned semantic branch instruction", and nominates the agent control layer as its strongest role: selecting tools, grading traces, detecting loops, checking completion, and deciding when to escalate. Every one of those is a small, bounded, high-frequency judgment sitting in the hot path — the exact shape the model was built for.

Gating dangerous tool calls#

LangChain shipped this as middleware on launch week. The whole idea is that the agent proposes a tool call and something cheap checks it before the runtime executes it.

guardrail.py

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (
    AutoModeMiddleware,
)

guardrail = AutoModeMiddleware(tools=["bash"])

agent = create_agent("openai:gpt-5.6-luna", middleware=[guardrail])

Written directly against the SDK, so you can see what the middleware is actually doing, the gate is a fan-out over the proposed call:

gate_tool_call.py

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient(model="jev-1.13.0")

def screen_tool_call(tool_name: str, arguments: dict, task: str):
    r = client.system_one(
        state={
            "task": task,
            "tool": tool_name,
            "arguments": arguments,
        },
        questions={
            "destructive": Noul(
                instructions=(
                    "Would running `tool` with `arguments` delete, overwrite, "
                    "or permanently modify data?"
                ),
            ),
            "outside_scope": Noul(
                instructions=(
                    "Does `tool` with `arguments` act on something outside "
                    "what `task` asked for?"
                ),
            ),
            "reversible": Score(
                instructions="How hard would this action be to undo",
                criteria=[
                    "Trivially reversible; no persistent effect",
                    "Reversible with effort",
                    "Irreversible or externally visible",
                ],
            ),
            "blast_radius": Choice(
                instructions="What is the widest scope this action could affect",
                criteria={
                    "single_file": "One file or record",
                    "project": "The working tree or one project",
                    "system": "The machine, or shared infrastructure",
                    "external": "A third party, a customer, or the public internet",
                },
            ),
        },
    )
    a = r.answers

    # Policy lives here, in code, where it can be reviewed and tested.
    if a["reversible"].score > 1.5 or a["blast_radius"].choice in {"system", "external"}:
        return "ask_user"
    if a["destructive"].noul > 0.6 and a["outside_scope"].noul > 0.5:
        return "ask_user"
    if a["blast_radius"].confidence < 0.5:
        return "ask_user"      # unsure about scope is itself a reason to stop
    return "auto"

Tool selection at catalogue scale#

A Choice takes up to 255 options and TypeSafe's advice is to give the full list rather than a shortlist. Their skill-suggestion cookbook ranks all 182 skills in Nous Research's Hermes catalogue in a single request, then fetches the top three in full and judges them again against that better evidence.

The structure of that cookbook is the part worth stealing, because it uses both primitives for the two genuinely different questions:

Question	Primitive	Why
Which tool fits best?	Choice over the catalogue	Relative — it settles which option wins
Does this turn need a tool at all?	Noul	Absolute — it can legitimately be low for every option

That distinction is not a nicety. A Choice's probabilities sum to 1, so it will always nominate a winner even when nothing fits. The Noul is what gives your harness permission to run no tool at all.

select_tool.py

r = client.system_one(
    state={"turn": latest_message, "context": recent_context},
    questions={
        "tool": Choice(
            instructions="Which tool best serves `turn`?",
            criteria={name: summary for name, summary in CATALOGUE.items()},
        ),
        # Relative pick above; absolute need below. Both are required.
        "needs_a_tool": Noul(
            instructions="Does `turn` require any tool at all, rather than a direct answer?",
        ),
    },
)

if r.answers["needs_a_tool"].noul < 0.5:
    return answer_directly()

pick = r.answers["tool"]
if pick.confidence < 0.6:
    return answer_directly()          # nothing clearly fits

return run_tool(pick.choice)

Loop detection and trace grading#

An agent that has tried the same thing four times is a state your harness can describe cheaply, and a judgment Jev can make in a few hundred milliseconds — fast enough to run on every step.

supervise.py

def supervise(trace):
    r = client.system_one(
        state={
            "goal": trace.goal,
            "last_steps": trace.steps[-6:],     # filter: recent steps only
            "step_count": len(trace.steps),
        },
        questions={
            "looping": Noul(
                instructions=(
                    "Do `last_steps` repeat the same approach without new information?"
                ),
            ),
            "progress": Score(
                instructions="How much closer to `goal` did `last_steps` get",
                criteria=[
                    "No progress; repeating or thrashing",
                    "Incremental progress",
                    "Substantial progress toward the goal",
                ],
            ),
            "complete": Noul(
                instructions="Has `goal` been fully satisfied by `last_steps`?",
            ),
        },
    )
    a = r.answers

    if a["complete"].noul > 0.85:
        return "finish"
    if a["looping"].noul > 0.7 and a["progress"].score < 0.6:
        return "break_loop"
    if len(trace.steps) > HARD_STEP_LIMIT:
        return "break_loop"     # the deterministic backstop, always
    return "continue"

The thing a harness cannot do#

The sharpest observation from launch week came from the author of a computer-use experiment, and it applies to every project in this tutorial:

Every piece of reasoning the frontier model does for free has to be rebuilt here as deterministic state.

— awlevin, typesafe-computer-use

Jev does not read your screenshot, infer what the app is, work out what the user probably meant and then decide. It answers bounded questions about state you assembled. Everything an LLM was doing implicitly between "here is a screenshot" and "click that button" becomes your code — extraction, normalisation, candidate generation, option construction.

That is the real cost of the pattern, and it does not appear on any pricing page. It is often worth paying — deterministic state is testable, loggable and reviewable in a way that an LLM's private reasoning is not — but budget for it.

Security, restated because it matters here most#

An agent harness is precisely the setting TypeSafe's adversarial-content warning is about. The state you feed the gate contains tool arguments, file contents, web pages and model output — all of it attacker-reachable in a real deployment, and none of it treated as hostile by the model.

Keep permissions, path allowlists and spend caps in code. The gate advises; the runtime enforces.
Never let the gate's own state include text that can rewrite the question — pass tool arguments as named fields, not as a concatenated prompt.
Build an injection test set: tool arguments that argue for their own safety, file contents that claim to be instructions.
Log the full probabilities for every gate decision. When something gets through, the distribution is the only forensic trail you have.

Next#

Continue to Shipping it, or read the concept page on what "cannot hallucinate" actually guarantees.

Sources for this page

Last reviewed 2026-09-18. Jev is days old and moving — where a claim is TypeSafe's own rather than independently verified, this page says so in the sentence that carries it.