Composite scoring: move the weights into code

The move#

TypeSafe's guidance is direct: if the judgment you want depends on several independent factors, ask about each factor separately and combine the answers with your own logic. Instead of "rate this startup pitch", ask about market size, technical feasibility and differentiation, then weight them in code. "When priorities shift, change the value of weights rather than rewriting a prompt."

Because questions in one request run in parallel, splitting one question into five costs you the tokens of four extra question texts and essentially no latency. The decomposition is close to free.

Worked example: résumé screening#

from typesafe_sdk import Score, TypeSafeClient

client = TypeSafeClient(model="jev-1.13.0")

response = client.system_one(
    state=resume_text,
    questions={
        "python_depth": Score(
            instructions="Depth of Python experience shown",
            criteria=[
                "None mentioned",
                "Mentioned, no detail",
                "Used in projects",
                "Primary language",
                "Deep expertise: architecture, performance",
            ],
        ),
        "team_leadership": Score(
            instructions="Experience leading engineering teams",
            criteria=[
                "None",
                "Informal mentorship",
                "Led a small team",
                "Managed direct reports",
                "Managed multiple teams",
            ],
        ),
        "system_design": Score(
            instructions="Experience designing distributed systems",
            criteria=[
                "None mentioned",
                "Contributed to discussions",
                "Designed components",
                "Owned a system's architecture",
                "Designed at scale across domains",
            ],
        ),
    },
)

Three narrow judgments, each with levels a hiring manager could argue with in a meeting. Now the policy:

a = response.answers

# Each Score has 5 levels, so the raw score runs 0..4. Normalise before
# weighting, or a longer rubric silently counts for more.
WEIGHTS = {
    "python_depth": 0.40,
    "team_leadership": 0.25,
    "system_design": 0.35,
}

composite = (
    WEIGHTS["python_depth"]    * (a["python_depth"].score / 4) +
    WEIGHTS["team_leadership"] * (a["team_leadership"].score / 4) +
    WEIGHTS["system_design"]   * (a["system_design"].score / 4)
)

Written generically, so adding a dimension is a one-line change:

def composite(answers, weights, levels):
    """Weighted mean of normalised Score answers.

    weights: {question_id: weight}, need not sum to 1
    levels:  {question_id: number of criteria levels}
    """
    total = sum(weights.values())
    return sum(
        w * (answers[qid].score / (levels[qid] - 1))
        for qid, w in weights.items()
    ) / total

Why this beats one big question#

That last row is the one that matters in a regulated setting. Jev returns no rationale — it cannot tell you why. Decomposition is how you get an explanation anyway: not from the model, but from the structure you imposed on it.

Composite scoring beyond a weighted mean#

The weighted mean is the starting point, not the only option. Because you own the combination step, you can encode policies a prompt could never express reliably:

• Hard gates. if a["security_review"].score < 1: reject() — no amount of strength elsewhere compensates.
• Confidence-weighted contribution. Down-weight a dimension the model was unsure about, rather than letting a coin flip carry 35% of the decision.
• Band assignment rather than a number. Map the composite onto strong / maybe / no, and send everything in maybe to a person.
• Non-linear scaling. Square the composite if you want top-end dimensions to dominate.

# Let the model's own uncertainty shrink a dimension's vote.
def confidence_weighted(answers, weights, levels, floor=0.3):
    num = den = 0.0
    for qid, w in weights.items():
        a = answers[qid]
        effective = w * max(a.confidence, floor)
        num += effective * (a.score / (levels[qid] - 1))
        den += effective
    return num / den

The deeper lesson, which is not about Jev#

This is the most useful thing anyone wrote about the launch, and it is worth sitting with. In TypeSafe's own evaluation, the LLM baselines also improved when the task was decomposed into a workflow instead of handed over as one prompt. Some of the gain that the launch numbers attribute to the model is really a gain from the architecture.

That cuts both ways, and both ways are useful. It means Jev's headline advantage over a well-structured LLM pipeline is smaller than the homepage suggests. It also means this pattern is worth adopting today, with whatever model you already use, and it will still be worth it if Jev turns out not to be.

Next#

You can now build one composite judgment. The next pattern uses Jev to decide who should be making the judgment at all. Continue to The cascade.