Real-time loops: the Doom lesson

What the demo actually is#

• Roughly 10 decisions per second, at about $7 per hour. TypeSafe notes the engineer who built it was worried about the rate; the rest of the team thought $7/hour was lower than expected.
• The input is structured game state rendered as text — positions, health, ammo, objectives, allowed actions. Not pixels. TypeSafe: "on structured state as a data structure with text, not on images (yet…)"
• The output is a Choice over the allowed actions, evaluated fast enough that the loop does not stall.

TypeSafe's companion demo, Wikiracing, is arguably the better technical showcase: navigating from one Wikipedia article to a target using only links encountered on the way, where each step can mean choosing between hundreds or thousands of options. That is the 255-option Choice ceiling and the two-stage high-cardinality sampler under genuine load.

Control rate versus advisory rate#

The most useful artefact to come out of launch week was not the game — it was a drone project's tier table, which imposed the discipline the pattern needs. Its author's conclusion is the rule: Jev "cannot be the perception layer, and it cannot run at control rate."

Sub-second is fast for a model and slow for a control loop. The test is simple: if the loop cannot safely proceed when the answer is late or missing, the decision does not belong to a network call. Anything remote — however fast — is advisory. Your code must have a defined behaviour for the frame where no answer arrived.

import time

class AdvisoryDecision:
    """Latest judgment, with an explicit staleness contract.

    The loop never blocks on the model. It reads whatever the last answer
    was, and treats an old one as no answer at all.
    """

    def __init__(self, max_age_ms: int = 400):
        self.max_age_ms = max_age_ms
        self._value = None
        self._at = 0.0

    def update(self, value):
        self._value = value
        self._at = time.monotonic()

    def get(self, default):
        age_ms = (time.monotonic() - self._at) * 1000
        if self._value is None or age_ms > self.max_age_ms:
            return default          # the safe action, chosen by you
        return self._value

Serialising state cheaply#

At ten calls a second, the cost of building the state matters as much as the call. Two things follow.

Keep the state small and stable. You are billed on input tokens, so every field that cannot change the decision is a recurring charge multiplied by your frame rate. The context-rot warning applies with extra force here: a bloated state costs accuracy and money, ten times a second.

def frame_state(world) -> dict:
    """Only what the decision needs. Pre-computed, pre-bucketed."""
    return {
        "health": bucket(world.player.health, [25, 50, 75]),   # "low" / "mid" / ...
        "ammo": bucket(world.player.ammo, [10, 40]),
        "threats": [
            {
                "kind": e.kind,
                "bearing": compass(e.angle),        # "ahead" not 47.3 degrees
                "range": bucket(e.distance, [200, 600]),
            }
            for e in world.nearest_enemies(3)       # three, not thirty
        ],
        "objective": world.current_objective,
    }

Budgeting a loop#

PRICE_PER_MTOK = 0.042

def hourly_cost(decisions_per_second, tokens_per_call):
    calls = decisions_per_second * 3600
    return calls * tokens_per_call / 1e6 * PRICE_PER_MTOK

hourly_cost(10, 2_000)    # ~$3.02/hour
hourly_cost(10, 5_000)    # ~$7.56/hour  — roughly the Doom demo's range
hourly_cost(30, 5_000)    # ~$22.68/hour

Two ceilings bite before your wallet does. The published rate limits are 250,000 tokens per second and 1,200 requests per minute — and 1,200 rpm is 20 requests per second. A single agent at 10 Hz is using half of that, which means "one agent" and "a fleet of agents" are very different conversations. TypeSafe also says the limits move without notice.

What the pattern is genuinely good for#

Strip away the game and the shape is: a decision that must happen faster than a human can supervise, over state your code already has, from a fixed set of actions.

• Trading and market-reaction logic — one launch-week project reports a decision per ~300ms block, with ~81ms model latency.
• Live moderation in a chat or stream, where a two-second LLM round trip is already too late.
• Interactive UI that adapts as the user types rather than after they submit.
• Robotics and simulation, in the advisory tier only.
• Anything currently implemented as a hand-tuned heuristic that everyone agrees is brittle.

Adam Argyle's question after building his own playground is the right one to sit with: "if you can get intelligent judgements multiple times a second…" — what does that make possible that nobody has built because the round trip was always too slow? Three days in, nobody knows. That is the genuinely interesting part of this launch.

Next#

Back to fundamentals: what a System One model is, and when a classifier you already own would be a better idea.