Shipping it

The documented status codes#

Catch the right exceptions#

The Python SDK's hierarchy is worth knowing in full, because it lets you distinguish "my request was wrong" from "the service is busy" from "the network failed" — three situations that need three different responses.

TypeSafeError (base, extends Exception)
├── TypeSafeAPIError                     — unsuccessful HTTP response
│   ├── TypeSafeBadRequestError          — 400
│   ├── TypeSafeAuthenticationError      — 401
│   ├── TypeSafePermissionDeniedError    — 403
│   ├── TypeSafeNotFoundError            — 404
│   ├── TypeSafeUnprocessableEntityError — 422
│   ├── TypeSafeRateLimitError           — 429
│   ├── TypeSafeInternalServerError      — 5xx
│   └── TypeSafeAPIResponseValidationError
└── TypeSafeAPIConnectionError (also extends ConnectionError)
    └── TypeSafeAPITimeoutError (also extends TimeoutError)

Three attributes earn their place in your logs. TypeSafeAPIError.request_id carries the x-typesafe-request-id header — the one thing support will ask for. TypeSafeRateLimitError.retry_after_ms is the server's requested wait. TypeSafeAPIResponseValidationError.field_path gives a dotted path such as answers.tone.confidence.

from typesafe_sdk import (
    TypeSafeAPIConnectionError,
    TypeSafeAPIError,
    TypeSafeRateLimitError,
    TypeSafeUnprocessableEntityError,
)

def decide(state, questions):
    try:
        return client.system_one(state=state, questions=questions)

    except TypeSafeUnprocessableEntityError as e:
        # Our bug. Retrying will fail identically. Fail loudly in CI.
        log.error("malformed jev request", field=str(e.body), request_id=e.request_id)
        raise

    except TypeSafeRateLimitError as e:
        log.warning("jev rate limited", retry_after_ms=e.retry_after_ms)
        raise                              # let the SDK's policy handle the wait

    except (TypeSafeAPIError, TypeSafeAPIConnectionError) as e:
        # Service-side or network. Degrade, don't drop the work.
        log.warning("jev unavailable", error=type(e).__name__,
                    request_id=getattr(e, "request_id", None))
        return None                        # caller falls through to the slow path

The JavaScript SDK mirrors this with the TypeSafe prefix dropped — APIError, RateLimitError, UnprocessableEntityError — plus APIUserAbortError, which has no Python equivalent.

Retries#

The Python SDK ships a RetryPolicy with sensible defaults, and honouring retry-after is on by default:

RetryPolicy(
    max_retries=2,
    backoff_initial=0.5,
    backoff_max=5.0,
    backoff_jitter=0.25,
    http_statuses={408, 429, *range(500, 600)},
    respect_retry_after=True,
    api_connection_error=True,
    api_timeout_error=True,
    timeout=30.0,
)

Two defaults are worth a second look for a request in a user-facing path. The per-operation timeout is 10 seconds — generous for a model that advertises 70–500ms, and long enough to blow a latency budget if you are calling Jev inside a page render. And max_retries=2 on a 429 means a rate-limited burst gets multiplied by three before it gives up.

from typesafe_sdk import RetryPolicy, TypeSafeClient

# In a hot path, fail fast and fall through rather than queue behind retries.
client = TypeSafeClient(
    model="jev-1.13.0",
    timeout=1.5,
    retry_policy=RetryPolicy(max_retries=1, backoff_initial=0.2, backoff_max=1.0),
)

Rate limits move#

Published limits for jev-1.13.0 are 250,000 tokens per second and 1,200 requests per minute, with either one exceeded returning 429. TypeSafe's own warning is unusually direct: the limits "can change without notice while we do, as upcoming large GPU deals land and we let in more users."

• Put a client-side limiter in front of bulk workloads rather than discovering the ceiling with 429s.
• Alert on your 429 rate as a percentage, not a count — the denominator is what tells you whether the limit moved or your traffic did.
• For map-reduce workloads, remember the tokens/second limit is the binding one. A 30k-token state at 1,200 requests per minute is well past it.

Four things that surprise people in week one#

Pin the version, log the version#

jev-latest and jev-preview are aliases; both currently resolve to jev-1.13.0. The instant you tune a confidence threshold against your own data, that threshold is coupled to one model's distributions — and an alias can move underneath it silently.

client = TypeSafeClient(model="jev-1.13.0")

Then assert on the response. The model field reports the versioned ID that actually answered:

EXPECTED_MODEL = "jev-1.13.0"

response = client.system_one(state=state, questions=questions)

if response.model != EXPECTED_MODEL:
    # Not necessarily fatal — but every tuned threshold below this line
    # was fitted against a different model's distributions.
    log.warning("jev model drift", expected=EXPECTED_MODEL, got=response.model)

Log the distribution, not just the answer#

Jev returns no rationale. When a decision turns out to be wrong, the probability distribution is the entire forensic record — and if you logged only choice, you have thrown it away.

def record(question_id, answer, response, latency_ms):
    payload = {
        "question": question_id,
        "model": response.model,
        "input_tokens": response.usage.input_tokens,
        "latency_ms": latency_ms,
    }
    if answer.type == "noul":
        payload["noul"] = answer.noul
    else:
        payload["value"] = getattr(answer, "choice", None) or answer.score
        payload["confidence"] = answer.confidence
        payload["probabilities"] = answer.probabilities   # the part that matters
    log.info("jev.decision", **payload)

With the distributions retained you can do two things later that are otherwise impossible: run a reliability diagram over real traffic to check calibration on your distribution, and re-tune a threshold against history instead of guessing.

The pre-launch checklist#

1. 1

   Every question has an escape option

   other, none of the above, not stated. Probability mass must have somewhere honest to land.
2. 2

   Every action has its own threshold

   In a table, in code, reviewable. Not one global constant.
3. 3

   The fallback path is tested

   Kill the API key in staging and confirm the system degrades to the slow path rather than dropping work.
4. 4

   An adversarial test set exists

   State is not treated as hostile by the model. If users can write it, test it.
5. 5

   The model version is pinned and asserted

   And drift is a log line somebody actually sees.
6. 6

   Probabilities are in your logs

   Not just the summary value. You cannot reconstruct them afterwards.
7. 7

   You know your branch distribution

   What share auto-resolves, escalates to a model, escalates to a person. That ratio is your real unit cost.
8. 8

   Someone owns the thresholds

   They encode risk tolerance. That is a product decision wearing a float.

Next#

For the conceptual grounding underneath all of this, read What a System One model is and Calibration, honestly.