LevelUpWith

Explore / AI & Agents

System One Models & Jev: Typed AI Decisions

Go from strong LLM skills to expert command of System One models: understand why Jev returns typed values with calibrated confidence instead of prose, place it correctly beside LLMs, agents, rules engines and classical ML, and ship confidence-gated decision layers with thresholds and evals.

Read lesson 1 free →

Levels 1 and 2 are free to read, no account needed. Levels 3 and 4 are free with a free account, and so are 20 answers from the AI tutor.

Three-dimensional cover artwork for “System One Models & Jev: Typed AI Decisions”

Expert · 33 levels · 4 free · Created Sept 2026 · Professionally curated by levelupwith.com

What's inside

  1. Level 1: System One Models: A New Model ClassFree
    Learn what a System One model is — a model that takes unstructured state plus a schema you fix in advance and returns typed values with calibrated confidence instead of generated prose.
    • Define a System One model in terms of its three invariants: pre-declared answer space, typed output, attached probability
    • Distinguish 'a model that decides' from 'a model that writes', and explain why software consumes the former directly
    • Identify, from a list of candidate AI workloads, which ones are decision-shaped rather than generation-shaped
  2. Level 2: Why Kahneman's System 1 Names the CategoryFree
    Trace the name to Daniel Kahneman's fast, intuitive System 1 thinking and see precisely which properties of that metaphor TypeSafe is claiming and which it is not.
    • Map System 1 (fast, automatic, judgement) and System 2 (slow, deliberative, verbal) onto System One models and reasoning LLMs
    • Explain why 'fast typed judgement' is the claim and 'no reasoning at all' or 'always correct' are not
    • Argue why an autoregressive chain-of-thought LLM is structurally a System 2 tool even when it returns JSON
  3. Level 3: Anatomy of a Jev Decision
    Walk the full shape of one Jev interaction — state in, one or more typed questions, and a response carrying a typed answer, a probability distribution and a confidence figure.
    • Label the parts of a Jev request (state, questions) and response (typed answer, probabilities, confidence)
    • Explain why the probability distribution, not just the chosen answer, is the useful part of the payload
    • Rewrite an informal product requirement such as 'flag risky refunds' as an explicit typed question with a fixed answer set
  4. Level 4: What TypeSafe Actually Shipped on 15 Sept 2026
    Establish the verified public facts of Jev's limited early access launch — company, funding, access model, headline latency and price — and separate them from claims nobody outside TypeSafe has reproduced.
    • State the launch facts: 15 September 2026 early-access hosted API, TypeSafe AI of San Francisco, $40M seed led by DCVC, founders including Diogo Almeida
    • Quote the vendor's headline figures (roughly 70–500 ms end-to-end, $0.042 per million input tokens, output not charged) and attribute them as self-run
    • List what is not public: weights, parameter count, architecture details, self-hosting, and independent benchmark or calibration results
  5. Level 5: Fit and Jaggedness: When Not To Use Jev
    Build a sober fit test for the decision layer, including the capability gaps TypeSafe itself documents per model version.
    • Apply a fit test based on high volume, repeated judgements over shared state with answers known up front
    • Rule Jev out for chat, code generation, free-text summaries, written explanations and non-text inputs
    • Design around documented weak spots — counting and arithmetic, date ordering and intervals, and multi-hop indirection — by decomposing or precomputing in code
  6. Level 6: Typed Outputs vs LLM JSON Mode
    Compare Jev's pre-enumerated answer space with LLM structured outputs, function calling and grammar-constrained decoding to see exactly what 'cannot produce a type error' buys you.
    • Explain the difference between constraining generated tokens to a grammar and sampling over an enumerated answer set
    • Justify the 0% structural-error claim as an argument from construction rather than from measurement
    • Show with an example that schema conformance eliminates parse and repair code but leaves wrong-but-valid answers fully possible
  7. Level 7: Latency, Price and the Unit Economics
    Quantify the decision layer's cost and speed profile against frontier LLM calls and work out where the two-orders-of-magnitude claims come from and where they break down.
    • Compute per-decision cost and end-to-end latency for a realistic workload under Jev pricing versus a frontier chat model
    • Explain how free output tokens and a single parallel pass produce the claimed 40x–200x speedups on comparable tasks
    • Critique the comparison methodology and note the risk that early-access pricing is subsidised
  8. Level 8: Hallucination vs Being Wrong
    Separate the failure modes a typed decision layer removes from the ones it keeps, and build a precise vocabulary for each.
    • Classify failures as structural (unparsable, off-schema, refusal) versus semantic (confidently wrong category or score)
    • Explain why 'mathematically cannot hallucinate' is a statement about output shape, not about truth
    • Estimate the residual error budget of a Jev-backed feature and decide what downstream checks it still needs
  9. Level 9: Determinism: What Jev Does Not Promise
    Examine what is actually guaranteed run to run — a valid typed answer — and why that is weaker than the determinism of a rules engine or a regex.
    • Distinguish schema determinism from output reproducibility and note that TypeSafe has not published reproducibility guarantees
    • Explain the operational consequences of a probabilistic decision layer for caching, replay, audit trails and regression tests
    • Choose correctly between a hard rule and a Jev question for a policy that must be explainable and stable
  10. Level 10: Jev vs Classifiers, Embeddings and Fine-Tunes
    Position Jev against classical ML options — labelled-data classifiers, embedding-plus-linear-head pipelines and fine-tuned small models — on accuracy, iteration speed and total cost of ownership.
    • Contrast runtime-defined questions needing no labelled data with a trained classifier needing a labelled corpus and retraining
    • Explain why embeddings yield continuous vectors for downstream work while Jev yields a discrete calibrated decision
    • Recommend, with reasons, when a high-volume stable task should graduate from Jev to a distilled in-house model
  11. Level 11: Locating the Decision Layer in the Stack
    Draw the modern stack — rules engines, classical ML, System One decisions, LLM generation and agents — and place each responsibility where it belongs.
    • Produce an architecture diagram assigning deterministic policy, typed judgement, prose generation and orchestration to distinct layers
    • Apply a selection rubric to route a mixed set of requirements to rules, classical ML, Jev or an LLM
    • Explain why an agent loop benefits from a typed decision layer for control flow while keeping an LLM for authoring
  12. Level 12: Non-Autoregressive Parallel Sampling
    Understand the core architectural move: every question is evaluated against the same state in one parallel pass, so no answer becomes context for another.
    • Explain how removing sequential token dependence collapses latency from per-token to roughly per-request
    • Derive the independence consequence — answers cannot condition on each other, so no chain of thought or self-consistency exists
    • Restructure a genuinely dependent two-step judgement into either separate calls or independent questions combined in code
  13. Level 13: The Constrained Answer Space
    Study how declaring the answer set in advance turns inference into a distribution over a finite, typed space, with the concrete cardinality limits that implies.
    • State the published bounds: a Choice of up to 255 options and a Score with between 2 and 10 ordered levels
    • Explain why probability mass is distributed over allowed answers only, making off-schema output unreachable
    • Diagnose an answer set that is too large, overlapping or non-exhaustive and redesign it within the limits
  14. Level 14: RLCD Against RLHF and RLVR
    Compare Reinforcement Learning for Calibrated Decisions with preference-based and verifiable-reward training to see what objective Jev is actually optimised for.
    • Contrast the three objectives: human-preferred answers, verifiable correctness and epistemically honest probabilities
    • Explain why optimising for calibration rather than preference discourages the confident agreeableness typical of chat models
    • Note honestly that TypeSafe has not published the RLCD method, data or reward details, and say what that costs a reviewer
  15. Level 15: What Calibration Really Guarantees
    Pin down calibration as an aggregate property — among answers reported at p, roughly p should be right — and learn why it says nothing about any single answer.
    • Define calibration formally and distinguish it from accuracy, sharpness and discrimination
    • Read a reliability diagram and compute expected calibration error from bucketed outcomes
    • Explain why calibration can hold on TypeSafe's distribution and fail on your own domain data
  16. Level 16: Reading Probabilities and Confidence
    Learn that confidence is a statistic derived from the shape of the returned distribution, and why the primitives expose it differently.
    • Explain how concentrated versus spread probability mass becomes a 0–1 confidence figure for Choice and Score
    • Explain why a Noul answer returns a probability directly and carries no separate confidence field
    • Extract signal from runner-up probability mass — near-ties, bimodal distributions and flat distributions — instead of using the argmax alone
  17. Level 17: Engineering Around an Opaque Model
    Inventory what TypeSafe has not disclosed about Jev and adopt practices that keep a production system safe despite that opacity.
    • List the open unknowns: architecture, size, training corpus, public benchmark results, throughput ceilings and independent calibration audits
    • Assess the concrete risks of a closed hosted-only API with no weights and no self-hosting path
    • Specify mitigations — your own held-out eval set, an abstraction boundary, and a documented fallback decision path
  18. Level 18: Your First Call: /v1/systemone and the SDKs
    Make a working request against POST https://api.typesafe.ai/v1/systemone using the official Python and Node SDKs, and read the response object correctly.
    • Authenticate with TYPESAFE_API_KEY and issue a call via typesafe-sdk (Python 3.10+) or @typesafe-ai/sdk (Node 20+)
    • Map the raw HTTP request and response body onto the SDK objects and typed accessors
    • Explain the risk of the jev-latest default and pin an explicit model version for reproducible evaluation
  19. Level 19: Constructing Good State
    Learn how to assemble the state a decision is made from — strings, JSON objects or arrays of text — inside the published token budgets, with only what the judgement needs.
    • Choose between string, JSON-object and text-array state and explain why the input is text-only today
    • Budget a request against the roughly 64k-token combined limit and the roughly 32k limit on state plus the longest single question
    • Prune and normalise state — precomputing counts, dates and derived fields in code — so the model judges rather than calculates
  20. Level 20: Designing Choice Questions
    Master the Choice primitive: instructions, optional criteria, and an option set of up to 255 mutually exclusive answers that your code can act on.
    • Write a Choice question with clear instructions and per-option criteria, returning choice, probabilities and confidence
    • Apply option-set design rules: mutually exclusive, collectively exhaustive, comparably scoped, with an explicit none-of-these or unknown option
    • Refactor a vague or overlapping category list into a Choice whose probability distribution is interpretable downstream
  21. Level 21: The Score Primitive: Ordered Spectra
    Learn to model graded judgements with Score, whose two-to-ten ordered criteria return a fractional position on a spectrum rather than a discrete label.
    • Write a Score question with ordered criteria arrays, knowing the level index comes from array order starting at 0
    • Interpret a fractional result such as 1.035 as an interpolated position between adjacent levels, plus its .probabilities and .confidence
    • Decide correctly when a judgement is an ordered spectrum (Score) versus unordered categories (Choice)
  22. Level 22: Noul: Yes/No as a Bare Probability
    Master the Noul primitive, which returns a single 0-1 probability of the affirmative and deliberately carries no separate confidence field.
    • Write Noul questions whose instructions state a crisp, verifiable proposition about the state
    • Explain why a Noul has no .confidence field and why 0.5 means maximal uncertainty rather than 'yes'
    • Pick Noul over a two-option Choice when you want a probability to threshold on directly
  23. Level 23: Batching: Many Questions, One Pass
    Understand how packing multiple typed questions against one shared state into a single request exploits parallel evaluation instead of sequential calls.
    • Batch several unrelated questions over one state object in a single POST /v1/systemone call
    • Explain why batched latency grows roughly logarithmically rather than linearly, and cite TypeSafe's self-reported 12.2x cheaper / 10.0x faster figures as unreproduced vendor benchmarks
    • Avoid the anti-pattern of sequential refinement calls that serialise what could be one parallel pass
  24. Level 24: Limits: Cardinality, Context and Modality
    Work within Jev's hard boundaries — 255 options per Choice, 64k and 32k token context rules, and text-only input — and design around them.
    • Budget a request against the 64k limit for state plus all questions and the 32k limit for state plus the longest single question
    • Decompose a label space above 255 options into a two-stage scoring design
    • Handle images and audio by transcribing or captioning them into state before the call
  25. Level 25: Thresholds and Escalation Logic
    Turn calibrated confidence into per-action decision policy, setting different bars for read-only, reversible and high-stakes actions.
    • Write confidence-gated routing that escalates below roughly 0.5 and demands a higher floor for irreversible actions
    • Justify per-action thresholds instead of one global cutoff by reasoning about the cost of each error type
    • Recognise that retrying an identical state after low confidence is futile and route to a human or another tool instead
  26. Level 26: Speculative Fan-Out in Production
    Adopt the pattern of asking every question you might need up front, since an extra question costs tokens but almost no wall-clock time.
    • Restructure a branching decision tree into one flat fan-out plus ordinary control flow in your own code
    • Quantify the token cost of speculative questions against the latency saved versus conditional round-trips
    • Identify cases where fan-out is wasteful because later questions genuinely depend on earlier answers
  27. Level 27: Composite Scoring From Atomic Dimensions
    Decompose a fuzzy holistic judgement into independent atomic Score questions and recombine them with explicit weights in code.
    • Split a vague rubric into orthogonal dimensions, each a single Score question
    • Combine normalised dimension scores with named weights so re-weighting is a reviewable code change
    • A/B test alternative weightings without retraining or reprompting the model
  28. Level 28: Cascades: Jev as Router, Not Replacement
    Place Jev as the cheap fast triage layer that dispatches to deterministic code, a frontier LLM or a human depending on answer and confidence.
    • Design a cascade routing low confidence to humans, lookups to plain code, and open-ended generation to an LLM
    • Estimate the cost and latency saving of triaging with Jev at $0.042/MTok input before invoking a frontier model
    • State plainly which tasks — chat, code generation, written explanations — Jev must never be asked to do
  29. Level 29: Evaluating Calibration, Not Just Accuracy
    Measure a typed decision layer the way probabilities demand, using reliability curves, expected calibration error and selective risk rather than accuracy alone.
    • Build a labelled evaluation set and compute ECE and a reliability diagram over Jev's confidences
    • Plot a selective-risk (accuracy versus coverage) curve to choose an escalation threshold from data
    • Read published community comparisons of Jev against open alternatives critically, noting that reported ECE varies widely and that vendor benchmarks are self-run
  30. Level 30: Drift Monitoring and Real Failure Modes
    Diagnose how a typed decision layer actually fails — flat probability distributions, missing escape hatches, criteria drift and silent state degradation — and monitor for it in production.
    • Log probabilities, confidence and state fingerprints so calibration drift and distribution shift are detectable over time
    • Diagnose a flat .probabilities spread as under-specified criteria or missing state rather than model confusion
    • Prevent forced misclassification by including an 'other' option and alerting when its share rises
  31. Level 31: The Open System One Ecosystem
    Survey the open-weights and hosted decision models that appeared alongside Jev and judge honestly when a local model beats the API.
    • Describe the open landscape, including small non-autoregressive decision models and library-based typed-decision layers over open-weights LLMs
    • Weigh hosted Jev against self-hosted options on latency, data residency, cost at volume and calibration out of the box versus after temperature refitting
    • Run a reproducible typed-decision benchmark on your own data before committing to any provider
  32. Level 32: Trajectory of Typed Decision Layers
    Reason soberly about where System One models are heading as a software building block, separating verified early-access facts from vendor claims and open speculation.
    • Distinguish what TypeSafe has actually published from what remains undisclosed, such as training data, architecture details and independent benchmarks
    • Argue where a typed decision layer should and should not displace rules engines, classical ML classifiers and LLM agents in the next stack
    • Write a defensible adoption plan for a limited early-access dependency, including exit paths to open alternatives
  33. Level 33: Final Challenge: Architect the Decision Layer Exam
    Prove total mastery by designing, hardening and defending a complete System One decision layer end to end across everything the course has covered.
    • Design a full production decision layer: state schema, mixed Choice/Score/Noul batch, thresholds, cascade and fallbacks within Jev's documented limits
    • Defend every architectural choice against LLM, agent, rules-engine and classical-ML alternatives on latency, cost, determinism and hallucination
    • Specify the evaluation, calibration monitoring and failure-mode playbook that keeps the system trustworthy after launch

Access

Levels 1 and 2 are free to read, no account needed. Levels 3 and 4 are free with a free account, and so are 20 answers from the AI tutor. Every level, the podcast edition and the AI tutor come with All Access at £4.99/month or any Creator plan — see pricing.

Read lesson 1 free →

More like this