Four Architectural Patterns for Building with Jev
Speculative fan-out, confidence-gated routing, composite scoring, and intent routing, with working code for each.
Patterns, not prompts
Jev is not a chatbot you prompt. It is a component you place inside software that already has control flow. TypeSafe documents four patterns for that placement, and between them they cover most of what people build:
| Pattern | What it does | Benefits |
|---|---|---|
| Speculative Fan-Out | Send many questions in a single call, including speculative ones, and let your code decide what's relevant | Cost, Speed |
| Confidence-Gated Routing | Use confidence as a second decision axis to build safer systems | Reliability, Safety |
| Composite Scoring | Combine several dimensions of analysis into a single score | Cost, Reliability, Speed |
| Intent Routing | Classify a user's intent and route to the appropriate handler | Cost, Speed |
They compose. A production triage system usually runs all four at once: fan out every question you might need, route on the intent, gate the risky branches on confidence, and rank with a composite.
1. Speculative fan-out
The key insight is an economic one, and it is worth stating precisely because it inverts the instinct everyone brings from LLM work.
Jev ingests the state once and evaluates every question against it in parallel. So response time is roughly the time of the slowest question, not the sum. Adding a question adds the tokens of that question's text and almost no latency. And per the Models page, jev-1.13 is charged per input token with output tokens free.
Put those together: asking a question you probably will not need costs a few input tokens and no meaningful time. Asking speculatively is not a hack, it is the rational default. The parallel questions cookbook puts numbers on it, measuring 13 questions batched into one call as 11.5x cheaper and 9.6x faster than 13 separate calls, with no change in the answers.
Consider support ticket triage. You need a category. If it is a bug report you also need severity and whether there are reproduction steps. If it is billing you need to know whether a refund was requested. The naive design fetches the category, then makes a second call for whatever the category implies. The fan-out design asks everything at once.
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
client = TypeSafeClient(model="jev-1.13.0")
ticket = ("Hi, I placed an order (#98423) last Thursday and was charged twice. "
"I also can't log in after the site update, and adding Apple Pay would "
"be really helpful. This is getting frustrating.")
response = client.system_one(
state=ticket,
questions={
"category": Choice(
instructions="Determine the broad category of this support ticket",
criteria={
"bug_report": "The user is reporting something that is broken or producing errors",
"billing": "Charges, invoices, refunds, subscriptions",
"feature_request": "The user is requesting new functionality",
"account": "Login, permissions, profile, security",
},
),
"bug_severity": Score(
instructions="How severe is the reported issue",
criteria=[
"Cosmetic; no impact to functionality",
"Broken or degraded feature; workaround exists",
"Blocking issue; no workaround exists",
],
),
"has_reproducible_steps": Noul(
instructions="The user describes specific steps to reproduce the issue",
),
"refund_requested": Noul(
instructions="The user is explicitly asking for a refund or credit",
),
"frustration": Score(
instructions="How frustrated the user appears",
criteria=["Calm, matter-of-fact", "Frustrated but civil", "Very angry"],
),
},
)bug_severity and has_reproducible_steps only matter if this is a bug report. refund_requested only matters for billing. All five come back anyway, and the code decides:
category = response.answers["category"]
bug_severity = response.answers["bug_severity"]
bug_repro = response.answers["has_reproducible_steps"]
refund = response.answers["refund_requested"]
frustration = response.answers["frustration"]
if category.choice == "bug_report":
if bug_severity.score > 1.5 and bug_repro.noul > 0.6:
escalate_to_engineering(ticket_id, severity="high")
else:
add_to_bug_backlog(ticket_id)
elif category.choice == "billing":
if refund.noul > 0.7:
route_to_billing_with_flag(ticket_id, refund_likely=True)
else:
route_to_billing(ticket_id)
elif category.choice == "feature_request":
log_feature_request(ticket_id)
# Frustration is useful regardless of category
if frustration.score > 1.5:
flag_for_priority_response(ticket_id)One round trip carries the entire decision tree. The irrelevant answers are discarded, and discarding them was cheaper than the round trip you avoided.
The budget you are spending against is the context limit, not a question count: 64k tokens per request for the state plus all questions combined, and 32k for the state plus the single longest question. The practical ceiling is usually the context rot that comes from an oversized state, not the questions themselves.
2. Confidence-gated routing
Choice and Score answers carry a confidence between 0 and 1, derived from how peaked the probability distribution is. The insight is to treat it as a second axis: the answer tells you what, confidence tells you whether to act on it.
Noul answers do not carry a confidence field, and that is not an omission. A Noul's distribution has only two outcomes, so the single noul value describes it completely. Near 0.5 is the uncertain region.
The documented example is voice banking, which is a good illustration because two actions in the same system have wildly different costs of being wrong.
from typesafe_sdk import Choice
questions = {
"intent": Choice(
instructions="What action is the user requesting?",
criteria={
"check_balance": "Check the balance of an account",
"approve_transfer": "Approve the pending transfer request",
"other": "Something else",
},
),
}action = response.answers["intent"]
# Below 0.6 confidence on any action, route to a human
if action.confidence < 0.6:
route_to_support_agent(account_id)
elif action.choice == "check_balance":
# Low stakes. 0.6 confidence is sufficient.
show_balance(account_id)
elif action.choice == "approve_transfer":
if action.confidence > 0.85:
# High stakes, but high confidence. Safe to act automatically.
approve_transfer(account_id)
else:
# High stakes, moderate confidence. Verify intent first.
ask_user_to_confirm(
"Just to confirm: you would like to approve this transfer, is that correct?"
)
else:
route_to_support_agent(account_id)The 0.6 floor catches anything genuinely ambiguous. Above the floor, each action carries its own bar set by consequences. Reading a balance back to the wrong user's phrasing costs a few seconds. Approving a transfer on a misheard command costs money, so it needs 0.85 or a confirmation step.
A confidence threshold is not one number for your system. It is one number per action, and the right values depend on your domain and your data. Start conservative, measure on your own inputs, and move them deliberately.
One caveat that belongs with this pattern: thresholds do not transfer between question types. A cutoff you tuned against a Noul's probability is meaningless applied to a Choice's confidence, for reasons covered in the traps post.
3. Composite scoring
Break a complex judgment into independent dimensions, score each separately, and combine them with weights you own.
Resume screening is the documented case. Instead of "rate this candidate", ask four Score questions, each about one dimension, each with levels you wrote:
from typesafe_sdk import Score
questions = {
"python_depth": Score(
instructions="How much depth of python experience does this candidate have, based on the supplied resume?",
criteria=[
"No Python experience mentioned",
"Mentioned but no detail",
"Used in projects, some specifics",
"Primary language, multiple projects",
"Deep expertise: architecture, performance, libraries",
],
),
"team_leadership": Score(
instructions="How much experience does this candidate have managing or leading engineering teams?",
criteria=[
"No management experience mentioned",
"Informal mentorship or tech lead role",
"Led a small team or project",
"Managed a team with direct reports",
"Managed multiple teams or an engineering org",
],
),
"system_design": Score(
instructions="How much experience does this candidate have designing large-scale or distributed systems?",
criteria=[
"No architecture work mentioned",
"Contributed to design discussions",
"Designed components of a larger system",
"Owned architecture of a significant system",
"Designed systems at scale across multiple domains",
],
),
"generalist": Score(
instructions="How much evidence is there that this candidate picks up unfamiliar tools, roles, or domains outside their core specialty?",
criteria=[
"Only one domain or role mentioned",
"Some variety but within a narrow field",
"Worked across a few different areas or tech stacks",
"Regularly moved between domains, wore many hats",
"Track record of ramping up in unfamiliar areas and delivering",
],
),
}Five levels means scores run 0 to 4, so dividing by 4 normalizes each dimension to 0-1 before weighting:
py = response.answers["python_depth"].score / 4
lead = response.answers["team_leadership"].score / 4
arch = response.answers["system_design"].score / 4
general = response.answers["generalist"].score / 4
# Senior IC
ic_score = (0.40 * py) + (0.10 * lead) + (0.40 * arch) + (0.10 * general)
# Engineering Manager
em_score = (0.15 * py) + (0.40 * lead) + (0.20 * arch) + (0.25 * general)Two roles, one model call, completely different rankings. That is the real payoff: the same four measurements support any number of weighting schemes, and adding a role is a dictionary of weights rather than a new prompt.
The weights are the part you can defend. When the top-ranked candidates do not match what your team would pick, you can see exactly which dimension drove the ranking and adjust a number. A single "rate this candidate" score gives you nothing to adjust and nothing to explain.
Two disciplines make this work. Keep each Score to one dimension: if a level description says "punctual and smart and experienced", it is measuring three things, and a candidate high on one and low on another cannot be placed. And do not read more precision into a score than it has. Using score > 1.5 as a threshold is fine and documented. Reconstructing an exact quantity by interpolating between levels is not.
4. Intent routing
Jev in front of everything else, deciding what handles each request. Some requests need a database lookup. Some need a specialist LLM with domain context. Some need a person. Sending every message through an expensive model just to find out which kind it is wastes most of the spend.
Two questions: what the user wants, and how hard it will be.
from typesafe_sdk import Choice, Score
questions = {
"intent": Choice(
instructions="The primary intent of this customer message",
criteria={
"order_status": "Asking about an existing order",
"product_question": "Asking about a product before buying",
"return_exchange": "Wants to return or exchange something",
"complaint": "Unhappy with experience, wants resolution",
},
),
"complexity": Score(
instructions="How complex is this request to resolve",
criteria=[
"Simple lookup or standard procedure",
"Requires some judgment or multi-step process",
"Unusual situation, edge case, or escalation needed",
],
),
}def route_ticket(ticket_id, response):
intent = response.answers["intent"]
complexity = response.answers["complexity"]
if intent.confidence < 0.5:
# If we don't have enough confidence to classify, route to a human agent
return route_to_human_agent(ticket_id)
if intent.choice == "order_status":
handle_order_status(ticket_id)
elif intent.choice == "product_question":
handle_with_llm(ticket_id, PRODUCT_SPECIALIST)
elif intent.choice == "return_exchange":
handle_with_llm(ticket_id, RETURNS_SPECIALIST)
elif intent.choice == "complaint":
low_confidence = complexity.confidence < 0.5
# A higher complexity.score leans toward the "escalation needed" end of the scale.
if complexity.score > 1 or low_confidence:
# Too complex for safe automation, or we're not sure about the complexity.
route_to_human_agent(ticket_id)
else:
handle_with_llm(ticket_id, COMPLAINT_RESOLUTION)Four destinations of three different kinds: deterministic code with no model involved, two specialist LLMs loaded with different context, and a human. The classification is one fast call; the expensive resources only get invoked for requests that need them.
Note the second confidence check, on complexity. Low confidence there means the model could not place the difficulty, which is itself a reason to escalate rather than to guess. Gating on the confidence of the question you are actually branching on is the habit worth copying.
Composing them
Here is the shape these take together in a real triage path. Fan out every question in one call. Use the intent to pick a branch. Gate destructive or expensive branches on confidence. Rank whatever needs ordering with a composite.
a = response.answers
# Gate first: an unconfident classification is not a classification.
if a["intent"].confidence < 0.5:
return route_to_human_agent(ticket_id)
if a["intent"].choice == "bug_report":
# Composite over speculative answers that this branch happens to need.
urgency = (0.5 * (a["bug_severity"].score / 2)
+ 0.3 * (a["frustration"].score / 2)
+ 0.2 * a["is_paying_customer"].noul)
if urgency > 0.7:
escalate_to_engineering(ticket_id)
else:
add_to_bug_backlog(ticket_id)Every jev answer in that block came from a single request. The control flow, the thresholds, and the weights are all in code, readable and testable, which is the point of the whole architecture. Jev supplies judgment over unstructured text. Your code supplies everything else.