How to Use Jev Correctly: 9 Documented Failure Modes

The list exists, and you should read it

TypeSafe publishes a jaggedness page for jev-1.13 listing nine failure modes in its own model. It is dated: last reviewed 2026-09-17. That is unusual candor for a model vendor, and it is also the single most useful page in the documentation, because every one of these failures is silent. Jev does not error when you ask it to compare two dates. It returns a calibrated-looking probability that happens to be wrong.

The framing that makes the whole list coherent: jev is a System One model. It makes fast, snap judgments over unstructured text and returns typed answers your code can branch on. It is not a reasoning engine, not a calculator, and not a text generator. Nearly every documented failure is a case of asking it to be one of those three things.

Here is each mode, what it looks like when it bites, and the fix.

1. Literal reading

Jev answers the question you wrote, not the question you meant. Scoping words, negations, and implied conditions are read at face value. The docs put it well: the model answers based on the words in the instruction, where a person would have read the intent behind them.

This is the failure mode that generates the most "the model is dumb" bug reports, and it is almost always the prompt.

# Bad: "recent" is doing invisible work. Recent to whom? Since when?
Noul(instructions="Has this customer complained recently?")
 
# Good: the condition is stated, not implied.
Noul(
    instructions="Does `ticket.messages` contain a complaint from the customer?",
    criteria=NoulCriteria(
        true="The customer expresses dissatisfaction with a product, service, or interaction",
        false="The customer asks a question or makes a request without expressing dissatisfaction",
    ),
)

The diagnostic from the docs is sharp enough to use as a rule: when you look at a wrong answer and find yourself explaining what you really meant, that explanation is the missing half of the instruction. Put it in criteria. Put the boundary cases in criteria too. Where interpretation genuinely cannot be avoided, split it into two literal questions and combine them in code.

2. Math and numbers

Jev is not a calculator. The docs recommend keeping mathematical logic in code, without hedging. Two sub-cases matter.

Counting is unreliable. Characters in a word, occurrences of a term in a passage, items in a long list. Per the docs, the model recognizes the shape of an answer rather than tallying, and the error grows with the size of the thing being counted.

The fix is a fan-out: ask one question per item and add up the answers yourself.

from typesafe_sdk import Noul, TypeSafeClient
 
client = TypeSafeClient(model="jev-1.13.0")
YES = 0.5
 
items = ["typesafe", "apple", "california", "banana", "likes", "calibration", "orange", "vertex"]
 
response = client.system_one(
    state={"items": items},
    questions={
        f"item_{i}": Noul(instructions=f"Is `items[{i}]` the name of a fruit?")
        for i in range(len(items))
    },
)
 
count = sum(response.answers[f"item_{i}"].noul > YES for i in range(len(items)))

Each question is now a single atomic judgment jev is good at. The arithmetic is in Python, where it is exact. Note that this costs almost nothing extra in latency because questions in a request are evaluated in parallel.

Before you even reach for this, ask whether the count needs a model at all. If a regex or a parser can find the unit, the count belongs in code and the model adds nothing.

Numeric representations underperform semantic ones. Jev does better on the English name of a color than on its hex value. Given RGB triples or hex values, it cannot reliably judge whether two values are near each other. The same holds up and down the abstraction stack: questions about high-level programming languages beat questions about assembly or binary-encoded instructions.

Convert in code, then pass either the computed number or a named bucket. Keep the model for the part that is genuinely a judgment, such as whether a color reads as a warning.

Do not interpolate a Score. This one has its own section below, because it is the trap people fall into most confidently.

3. Date and time comparison

Jev reads dates as text, not as ordered quantities. Asking which of two dates comes first, how far apart they are, or whether one falls inside a window is unreliable. It degrades further with mixed formats, relative references, and domain boundaries such as quarters, settlement windows, and accrual periods.

The fix is the cleanest split in the whole list, because it falls exactly along the judgment/arithmetic line. Extraction is a judgment, so give it to the model. Arithmetic is not, so keep it in code.

Every part of a date is a small closed set: twelve months, thirty-one possible days, a bounded range of years. That makes extraction a Choice over enumerated options rather than free-form parsing, and it gives you somewhere to put an explicit "not stated" option so a missing part gets reported instead of guessed.

from typesafe_sdk import Choice, TypeSafeClient
 
MONTHS = ["january", "february", "march", "april", "may", "june", "july",
          "august", "september", "october", "november", "december"]
 
client = TypeSafeClient(model="jev-1.13.0")
 
response = client.system_one(
    state={"notice": "Your plan renews on the fifteenth of next March."},
    questions={
        "month": Choice(
            instructions="Which month does the renewal date in `notice` fall in?",
            criteria={**{m: None for m in MONTHS}, "not_stated": "No month is given"},
        ),
        "day": Choice(
            instructions="Which day of the month is the renewal date in `notice`?",
            criteria={**{str(d): None for d in range(1, 32)}, "not_stated": "No day is given"},
        ),
    },
)
 
# Code owns everything from here: ordering, duration, offset, weekday.
month = response.answers["month"].choice
day = response.answers["day"].choice
if month != "not_stated" and day != "not_stated":
    renewal = build_date(year=infer_year(month), month=MONTHS.index(month) + 1, day=int(day))

A Choice accepts up to 255 options, so a 31-way day question is well within budget. Once you have real date objects, every comparison is exact.

4. Indirection

Instructions carrying double negatives or complex indirection are answered less reliably. A question about a property of a property, or anything requiring multiple hops of reasoning, costs accuracy. This is the boundary between System One and System Two work, and jev sits firmly on the fast side of it.

Two fixes, both mechanical. Write instructions as directly as possible, and identify the relevant parts of state by name.

# Indirect: the model must find the manager, then find their department, then judge it.
Noul(instructions="Is the person who approved this request in a department that owns the budget?")
 
# Direct: paths point at the exact values, code did the lookup.
Noul(instructions="Does `approver.department` match `request.owning_department`?")

Backtick paths are the tool here. Jev supports pointing a question at a nested value with a dot-and-index path such as `ticket.messages[0].text`, and the backticks are part of the syntax. Every hop you resolve in code before the call is a hop the model does not have to make.

5. Large state full of irrelevant detail

Accuracy falls as the state grows with content unrelated to the decision. Unrelated detail acts as a distractor. The docs call this context rot, and it has a second cost: a large state makes it much harder to tell which part of the input produced a wrong answer.

Retrieve and filter in code first. Send only the fields the question needs. This is why the docs recommend a structured object state over one giant string: named fields you can select from.

When you cannot filter in code because relevance is itself a judgment, use a Noul as the filter. Score each candidate passage for relevance in one fan-out call, keep the ones above your threshold, and send only those in the second request.

# Pass 1: cheap relevance filter over candidate passages.
response = client.system_one(
    state={"passages": passages, "query": query},
    questions={
        f"rel_{i}": Noul(
            instructions=f"Does `passages[{i}]` contain information that answers `query`?"
        )
        for i in range(len(passages))
    },
)
 
kept = [p for i, p in enumerate(passages) if response.answers[f"rel_{i}"].noul > 0.6]
 
# Pass 2: the real question, against a state that is only relevant material.

Keep the context limits in mind while you do this. Per the Models page, jev-1.13 allows 64k tokens per request covering the state plus all questions combined, and 32k for the state plus the single longest question. The limits are a ceiling, not a target. Accuracy starts falling long before you hit them.

6. Adversarial content

State is data, and jev-1.13 does not treat it as hostile by default. Content written to steer the model adversarially, whether an injected instruction, a deliberately misleading framing, or text that argues for its own classification, can move the answer. TypeSafe says it expects to improve this in future versions, which is a clear statement that it is not fixed now.

This matters more than it first appears, because the natural use of jev is classifying user-generated content: support tickets, forum posts, uploaded documents, email. All of that is attacker-controlled in the general case.

There is no flag to set. The documented mitigations are to be explicit in the criteria and to test your integration thoroughly before deploying it to many users. In practice:

• Write criteria that describe observable properties of the text, not conclusions about it. "The message body asks the recipient to provide a password" is much harder to talk out of than "The message is malicious".
• Put untrusted content in a clearly named state field and point questions at it by path, so the question is structurally about `message.body` rather than about the whole request.
• Build an adversarial test set before launch. Include text that tells the model what to answer.
• Never let a single jev answer be the only thing standing between a user and a destructive action. That is what confidence gating is for.

7. Contradictory instructions and criteria

When the instructions and the criteria ask for different things, jev gets confused. The documented example is a Noul where true maps to no and false maps to yes. It performs worse, and predictably so.

Treat the criteria as an extension of the instruction, not as a separate field. They are read together.

# Contradictory: instruction asks about absence, criteria describe presence.
Noul(
    instructions="Is this message free of personal data?",
    criteria=NoulCriteria(
        true="Contains an email address, phone number, or home address",
        false="Contains none of these",
    ),
)
 
# Aligned: high value means yes to the thing named in the instruction.
Noul(
    instructions="Does this message contain personal data?",
    criteria=NoulCriteria(
        true="Contains an email address, phone number, home address, or government ID number",
        false="Contains none of these",
    ),
)

The related Noul-writing rule: phrase the question so a high value means yes. Inverted phrasings such as "Is the message free of personal data?" do not just risk model confusion, they guarantee that code reading the value later will eventually get it backwards.

Aim for instructions that are easy for an average person to read and understand. If a colleague has to read your criteria twice, jev will too.

8. Common-sense structural invariants

This is the most important entry on the list and the least intuitive, so I have given it a post of its own. The summary:

Jev-1.13 is extremely consistent, meaning semantically similar inputs produce quantitatively similar outputs. But many structural invariants you would assume hold simply are not guaranteed.

TypeSafe publishes two measured examples. On the ticket "I'm not happy with the fit. What are my options here?", the question "Is the customer asking for a refund?" asked as a Noul and as a yes/no Choice:

And a question with its own negation, asked as two separate Nouls on "I was charged twice for the same order. Can someone look into this?":

The probabilities do not sum to 1, and they are not supposed to. The guidance: do not rely on expected structural invariance, do not carry a threshold tuned on a Noul over to a Choice, and do not hold the model to arithmetic identities between separate questions.

The reason is stated directly in the docs, and it is the line worth memorizing: "the Choice is relative, settling which option, while each Noul is absolute and can be low for all of them."

Ask each decision one way. Enforce identities in code.

9. Generation

Jev is not trained to generate text. The docs note you can force it to by chaining choices, and that this will not work well and will be very slow.

The productive reframe is that extraction is usually not generation. When the answer space is bounded, turn it into a Choice over the options rather than asking for the value itself. For data extraction from open text, the documented approach is to pull candidate options out with a regex or a generative model first, then let jev pick the correct extraction. Jev is a very good discriminator over candidates you supply. It is not a producer of candidates.

If you genuinely need prose, use a model built for prose. A common and sensible architecture is jev as a fast front-end classifier deciding which expensive generative model gets invoked, which is the intent routing pattern.

What the nine have in common

Read the list twice and a single rule falls out of it. Six of the nine failures are jev being asked to do work that belongs in code: counting, date arithmetic, multi-hop lookups, filtering, reconstructing numbers from score expectations, enforcing arithmetic identities between answers. Two more are prompt defects that better writing fixes. Only adversarial robustness is a genuine model limitation with no clean workaround today.

TypeSafe's own summary of what to avoid is compact enough to keep on a sticky note: do not ask the model something code can compute exactly, do not hide several judgments inside one question, do not hand it System Two tasks with layers of indirection, and do not give it more context in state than the question needs.

Everything else is engineering.