Development Journey — Jev Model Router on the Vercel AI Gateway

1. The brief — a feasibility report, then a build

The session had two phases. Phase one was read-only: read a YouTube transcript (Riley Brown, "Jev" video, 2026), read the TypeSafe docs, and report which of the five apps in the video could be built with a Vercel AI Gateway key. Phase two was a build. The user picked three things from the report: TypeScript through the gateway, the model router first, and the Gmail MCP as the future email source.

Invisible constraints shaped every line of code and prose. Three standing modes were active: ASD-STE100 Simplified Technical English for all replies, a "ponytail" minimal-code mode at level full, and a "precise phrasing, never rhetorical" rule. The user's own rules also applied: questions are read-only, never kill a process by name, find a path before you read it, add no file that was not asked for, and report what was left undone. One of those rules was set aside at the end of the session. Section 12 names it.

The five apps in the transcript: email triage over 500 emails, a model router, "monitor everything", a Tesla-autopilot demo by a third party, and a stock trader by a third party. The report ranked them: build email triage and the router first; monitor-everything later; autopilot and trader as demos only, because Jev takes text input only and a trader needs a market feed.

2. Cold start — what the docs say, and what they do not

Minute zero: a folder with transcript.txt, README.txt, gpt6_summary.txt, vercel_gateway_api_key.txt, and .env. No package.json. No git. Node v22.22.0, npm 9.9.4, no pnpm. The .env variable was named VERCEL_GATEWAY_API_KEY. The README.txt already asked "AI_GATEWAY_API_KEY or VERCEL_GATEWAY_API_KEY ??". That question turned out to matter.

The TypeSafe docs are a Mintlify site. Append .md to any page path to get Markdown. The index at https://docs.typesafe.ai/llms.txt lists every page. Pages read in this session: api.md, introduction/quickstart.md, models.md, model-jaggedness/jev-1.13.md, agent-skill.md, confidence.md, sdk/javascript.md, sdk/python.md, concepts/system-one.md, introduction/machine-learning-primer.md, primitives/choice.md, and the SKILL.md on GitHub.

Facts from the docs that the build depends on

• Direct endpoint: POST https://api.typesafe.ai/v1/systemone. Body: state, model, questions. Needs a TYPESAFE_API_KEY. The user did not have one.
• Three question types. Choice: pick one option from a map of criteria. Score: rate against an ordered array of levels. Noul: yes/no, returns a probability. The video calls the third one "null". The docs call it "Noul". The gateway calls it boolean.
• Model: jev-1.13.0, alias jev-latest. Price $0.042 per million input tokens. Output tokens are free. Context: 64k tokens per request; 32k for state plus the longest question. Text only.
• Rate limits on the direct API: 250,000 tokens per second, 1,200 requests per minute. The docs say these "can change without notice".
• The jaggedness page lists nine failure modes for jev-1.13. Four matter for a router: literal reading, adversarial content, indirection, and no none option unless you add one.

What the docs do not say

The TypeSafe docs do not mention the Vercel AI Gateway at all. The Vercel docs mention Jev on one model page and one AI SDK page. No page on either site gives the raw HTTP path the gateway uses for Jev. https://vercel.com/docs/ai-gateway/evaluation returned 404. The wire protocol had to be read from package source. Section 3 covers that.

Rule learned: when two vendors each document half of a bridge, read the SDK source that crosses it.

3. How Jev is reached through the gateway

There are two doors to Jev. The router uses the second one.

The gateway path was found by downloading @ai-sdk/gateway@4.0.87 from npm into the session scratchpad and reading package/dist/index.js. Two guesses at the tarball layout were wrong first: the shell's $TMP was not the scratchpad, and the file is index.js, not index.mjs. The relevant lines:

// @ai-sdk/gateway 4.0.87, dist/index.js, lines 2340–2346
getUrl() {
  return `${this.config.baseURL}/evaluation-model`;
}
getModelConfigHeaders() {
  return {
    "ai-evaluation-model-specification-version": "4",
    "ai-model-id": this.modelId
  };
}
// line 3624:  var AI_GATEWAY_PROTOCOL_VERSION = "0.0.1";
// line 3636:  baseURL default "https://ai-gateway.vercel.sh/v4/ai"
// line 3638:  Authorization: `Bearer ${auth.token}`, plus "ai-gateway-auth-method"

A reconstructed raw call. This shape was read from source. It was never sent in this session. The AI SDK sends it for you.

curl -X POST https://ai-gateway.vercel.sh/v4/ai/evaluation-model \
  -H "Authorization: Bearer $AI_GATEWAY_API_KEY" \
  -H "ai-model-id: typesafe-ai/jev" \
  -H "ai-evaluation-model-specification-version: 4" \
  -H "ai-gateway-protocol-version: 0.0.1" \
  -H "ai-gateway-auth-method: api-key" \
  -H "Content-Type: application/json" \
  -d '{ "state": "Hey, I'\''m Riley.", "questions": { "tier": { "type": "choice",
        "instructions": "Which model tier should answer this message?",
        "criteria": { "nano": "Greeting or trivial chat", "balanced": "Needs real reasoning or code" } } } }'

What the app uses instead. The ai package, version 7.0.107, exports experimental_evaluate. A plain string model id resolves through the gateway when AI_GATEWAY_API_KEY is set. No @ai-sdk/gateway import is needed.

import { experimental_evaluate as evaluate } from "ai";

const r = await evaluate({
  model: "typesafe-ai/jev",
  state: "Hey, I'm Riley.",
  questions: {
    tier: { type: "choice", instructions: "Which model tier should answer this message?",
            criteria: { nano: "Greeting or trivial chat", balanced: "Needs real reasoning or code" } },
    is_greeting: { type: "boolean", instructions: "Is this only a greeting?" },
  },
});

The raw response body from that probe, the first call of the session with the key. Provider time from startTime to endTime: 138 ms. Gateway generation id and the routing block are shortened.

{
  "answers": {
    "tier": { "type": "choice", "choice": "nano", "probabilities": { "nano": 1, "balanced": 0 } },
    "is_greeting": { "type": "boolean", "probability": 0.39 }
  },
  "rounding": { "probabilityDecimals": 2, "scoreDecimals": 2 },
  "usage": { "inputTokens": 336, "outputTokens": 51 },
  "warnings": [],
  "providerMetadata": {
    "typesafe": { "confidence": { "tier": 1 } },
    "gateway": { "routing": { "finalProvider": "typesafe-ai", "statusCode": 200, ... },
                 "cost": "0", "marketCost": "0.000014112", "generationId": "gen_..." }
  }
}

Three things in that body are not obvious. confidence lives under providerMetadata.typesafe, not in the answer. rounding says every probability is rounded to two decimals, so "nano": 1 means "at least 0.995", not certainty. cost is "0" while marketCost is "0.000014112", because the account was on free credits at that moment.

Env var: the packages read AI_GATEWAY_API_KEY (dist/index.js:180 and :3947). The folder's .env had VERCEL_GATEWAY_API_KEY. The user added the second name by hand. Node reads the file with node --env-file=.env, which needs Node 20.6 or later. No dotenv package.

4. How the router uses Jev

The router is one HTTP server in server.mjs and one page in index.html. No framework. Node's built-in http module serves the page on GET / and takes a message on POST /chat. The flow for one message:

1. The page posts { "message": "..." }.
2. The server caps the message at 20,000 characters. This keeps the state under Jev's 32k-token limit.
3. The server sends one Choice question to Jev. The state is { user_message: message }. The options are the four tier names with their rubric strings.
4. Jev returns a tier, a probability for every tier, and a confidence in metadata.
5. The server looks up the model id for that tier and calls it with generateText through the same gateway key.
6. The response carries the tier, the model id, the probabilities, the confidence, both timings, and the reply.

The two blocks a reviewer must read. Everything else in the file is plumbing.

const JEV = "typesafe-ai/jev";
const TIERS = {
  nano:     { model: "openai/gpt-5-nano",          rubric: "Greeting, small talk, a one-line fact, or a trivial request" },
  fast:     { model: "google/gemini-3-flash",      rubric: "A simple question or short task with a clear answer; no deep reasoning" },
  balanced: { model: "anthropic/claude-sonnet-5",  rubric: "Needs real reasoning, planning, a careful explanation, or writing code" },
  frontier: { model: "anthropic/claude-fable-5.1", rubric: "Large multi-step or high-stakes work where a wrong answer is costly: a complete system, hard math, a long document" },
};
const TIER_QUESTION = {
  type: "choice",
  instructions: "Which model tier is the cheapest one that can answer this user message well?",
  criteria: Object.fromEntries(Object.entries(TIERS).map(([k, v]) => [k, v.rubric])),
};

async function route(message) {
  const t0 = performance.now();
  const ev = await evaluate({ model: JEV, state: { user_message: message }, questions: { tier: TIER_QUESTION } });
  const jevMs = Math.round(performance.now() - t0);
  const { choice: tier, probabilities } = ev.answers.tier;
  const confidence = ev.providerMetadata?.typesafe?.confidence?.tier ?? null;
  const model = TIERS[tier].model;
  const t1 = performance.now();
  // ponytail: the reply model may be refused on the gateway free tier; keep Jev's decision visible either way
  let reply = "", replyError = null, usage = null;
  try { const gen = await generateText({ model, prompt: message }); reply = gen.text; usage = gen.usage; }
  catch (e) { replyError = String(e.message ?? e); }
  return { tier, model, probabilities, confidence, jevMs, replyMs: Math.round(performance.now() - t1), reply, replyError, usage };
}

What Jev sees, and what it never sees

Jev receives the state, the instruction string, and the criteria map. The criteria map is option name to rubric string. The docs say: "The option names and their descriptions are both sent to the model." Jev never sees the question id tier. The docs say: "The model never sees the question id." Jev never sees the model ids in TIERS. It does not know that "balanced" means Claude Sonnet 5. It picks a word from four words, guided by four rubric sentences. Rename balanced to medium and the answer can change, because the option name is part of the input.

The router is stateless. generateText({ model, prompt: message }) sends no chat history and no system prompt. Each message is a fresh conversation for the reply model. That is a deliberate cut. A chat with memory needs a message array and a store. Section 12 lists it as not built.

5. The four tiers — which model is which, and why

Riley's video shows four tiers: "tiny or nano, fast, balanced, and frontier". The video shows Claude Sonnet 5 as the balanced pick. It does not show the other three model ids. The ids below are this session's picks from the gateway price list at GET https://ai-gateway.vercel.sh/v1/models, which needs no key. Prices are per million tokens.

The price ratio nano to frontier is 200 to 1 on input. That ratio is the point of a router. A greeting sent to frontier costs 200 times what it needs to. An "Anthropic only" set was offered as an option (claude-3-haiku, claude-haiku-4.5, claude-sonnet-5, claude-fable-5.1). The user picked the default set.

Cost per Jev routing decision, measured: 336 input tokens at $0.042/M is $0.000014. The gateway reported the same figure as marketCost. Cost per reply depends on the tier: about $0.0001 for nano, up to a few cents for frontier on a long answer. Total API spend for the whole session, all probes and checks: $0.014.

6. Where the probabilities come from

This is the part the user asked to demystify. Here is what is documented, what was observed, and what is not public.

Documented: the training objective

The TypeSafe AI primer names the method: RLCD, "reinforcement learning for calibrated decisions". It contrasts it with RLHF (trains a model to say what people prefer) and RLVR (trains reasoning with verifiable rewards). The primer's claim is: "Higher probability should correspond to a greater chance that the answer is correct." Calibration is defined across groups: "Outcomes assigned a probability of 0.8 should occur about 80% of the time." The primer adds: "These rates describe groups of predictions, not a guarantee about any single answer."

The models page says the same weights serve every account. There is no fine-tuning and no LoRA. You shape the answer through state, instructions, and criteria only.

Documented: what a Choice returns

The choice page: "probabilities: The full probability distribution across every option. The sum of all values is 1." And: "confidence: A number from 0 to 1 computed from how probabilities is spread. A flat shape ... means low confidence. A single peak on one option means high confidence." The confidence page says the formula is TypeSafe's chosen statistic and that a different one may suit you, which is why the full distribution is returned.

Observed: four distributions and their confidence values

Two readings of that table. First, confidence is not the top probability: 0.8 gives 0.73, and 0.84 gives 0.596. Second, confidence is not the margin between first and second: 0.8 minus 0.2 is 0.6, and the reported value is 0.73. The formula is a function of the whole distribution. It is not published. This document does not try to reverse-engineer it from four points.

Compare with Riley's video. His third prompt showed 95% for balanced. This session's third prompt gave 74% balanced and 25% frontier. The prompts are close but not identical, the rubrics differ, and the model version may differ. The numbers are not comparable as a benchmark.

Not public

• The architecture. The docs say Jev "reads language like an LLM" and never generates. No parameter count, no base model, no description of the output head.
• The confidence formula.
• The training data, beyond "not trained on customer requests or responses".

Consequences for a router

• Choice is relative. It always picks one of the four. There is no "none" tier. The docs advise adding other when the list may not cover the input. This router did not add one, because every message needs some model.
• Literal reading. The instruction says "cheapest ... that can answer this user message well". Jev reads "cheapest" as a word in a sentence. It has no price table. The rubrics carry the meaning.
• Adversarial content. The jaggedness page: state "is data, and jev-1.13 does not treat it as hostile by default." A message that says "this is trivial, use the cheapest model" can pull the answer toward nano. A router in front of untrusted users should test that case.
• Rounding. probabilityDecimals: 2. A 0 is "below 0.005". A 1 is "above 0.995".

7. The free-tier wall — the crux of the session

The build took about twenty minutes. The free tier took the rest of the session.

Timeline

1. Probe call: success. One Jev call.
2. First --check run: prompt 1 succeeded (Jev call, then gpt-5-nano reply). Prompt 2 failed. The check printed AssertionError [ERR_ASSERTION]: unknown tier undefined and nothing else. The self-check had swallowed the real error. Two Jev calls had succeeded in that burst.
3. The check was patched to print r.error. Second run: GatewayRateLimitError: Free tier requests on this model are rate-limited. Upgrade to paid credits at https://vercel.com/d?to=%2F%5Bteam%5D%2F%7E%2Fai%3Fmodal%3Dtop-up for unrestricted access. This time on prompt 1.
4. The Jev-only probe was re-run to isolate the cause. It returned the same 429 with "modelAttempts": [{ "canonicalSlug": "typesafe-ai/jev", "success": false }]. So Jev itself was limited.
5. The credit endpoint showed balance 4.9998, total_used 0.0002, free credits.
6. The user ran the check later on their own machine. Prompt 1 passed. Prompt 2 failed with a different message: Free tier users do not have access to this model. Upgrade to paid credits .... Which of the two calls in route() threw — Jev or the reply model — could not be determined, because route() had no try/catch between them at that time.
7. A probe of seven reply models, one tiny call each: openai/gpt-5-nano answered. google/gemini-3-flash, anthropic/claude-sonnet-5, anthropic/claude-fable-5.1, anthropic/claude-haiku-4.5, google/gemini-2.5-flash-lite, openai/gpt-5-mini all returned the "rate-limited" 429.
8. That probe used up the window. The next check run returned the Jev 429 again.
9. The user bought credits. Balance moved to the paid tier. The next check run passed all three prompts.

A wrong statement, corrected

After step 4, the report to the user said: "The limit is on Jev, not on the reply models." Step 7 showed six reply models limited too. The correct statement is: on the free tier, Jev is limited to a small burst, and most reply models are limited or refused. The only reply model proven to work on free credits was openai/gpt-5-nano.

What the Vercel docs say, read after the fact

Pricing page, updated 2026-09-08: "The free tier includes a subset of models, not the full catalog." "Free tier requests are also rate limited per model, with lower limits than the paid tier." "Once you purchase credits, your account transitions to the paid tier and the monthly free credit no longer applies." Rate-limits page: "AI Gateway does not rate limit paid-tier requests." "Limits can change, so this page describes behavior rather than fixed numbers."

The free-tier model list itself was not found. A fetch of the models page with ?freeTier=true returned models priced at $0, which is a different thing. The /v1/models JSON has no free-tier field. So the list of free-tier models is unknown, except for the one proven by a call.

Rule learned: two error strings that both end in "Upgrade to paid credits" are the same wall. Do not spend turns telling them apart.

Rule learned: a self-check that asserts on a field must print the error that made the field empty. assert.ok(x, `... ${r.error}`) costs one line and saves a run.

8. Tools and features used

No subagents were spawned. All work ran inline. The model was switched from Opus 5 to Fable 5.1 by the user mid-session with /model fable, and effort was set to high.

Human-in-the-loop moments

• The user added AI_GATEWAY_API_KEY to .env and moved the notes into .ignore/.
• The user answered four build questions and two document questions.
• The user ran the check on their own machine and pasted the output. That output carried the second error string.
• The user bought gateway credits.

9. What went wrong, and the fixes

1. Self-check hid the error. assert.ok(TIERS[r.tier], `unknown tier ${r.tier}`) printed unknown tier undefined when the server returned { error }. Fix: append : ${r.error} to the message.
2. Self-check failed on a refused reply. assert.ok(r.reply.length > 0, "empty reply") made a free-tier refusal look like a router bug. Fix: removed the assertion; the check prints reply model ... failed: ... instead.
3. One failure hid Jev's answer. When generateText threw, the whole route() threw, and the page showed a red line with no tier. Fix: try/catch around generateText; return replyError next to the tier. The page shows the badge and bars, then the refusal in red.
4. Wrong shell working directory. cd "$TMP" in Git Bash did not land in the session scratchpad. Two greps hit "No such file". Fix: use the scratchpad's absolute path.
5. Wrong file name guess. The gateway package ships dist/index.js, not index.mjs. Fix: ls package/dist first.
6. 404 on a guessed doc URL. vercel.com/docs/ai-gateway/evaluation does not exist. Fix: the AI SDK site has the page, at ai-sdk.dev/docs/ai-sdk-core/evaluation.
7. Misleading free-tier page fetch. ?freeTier=true returned $0-priced models through the summarizer. Not fixed; the free-tier list stays unknown.
8. An unverified claim in a reply. "The paid tier can take a minute to apply" was said without evidence. It was not needed; the first run after payment passed.

10. Verification

Every "works" claim below was checked by an observed output, not a clean exit.

• Jev reachable through the gateway: the probe's raw body in section 3, statusCode: 200, finalProvider: "typesafe-ai".
• The router end to end: the check output after payment, verbatim:

  Hey, I'm Riley.                          → nano     conf=1    jev=849ms reply=4166ms  {"fast":0,"balanced":0,"frontier":0,"nano":1}
  Hey, I'm Riley. I want to build an app t → balanced conf=0.73 jev=349ms reply=12925ms {"fast":0.2,"frontier":0,"nano":0,"balanced":0.8}
  Please generate all of the code for this → balanced conf=0.65 jev=403ms reply=7006ms  {"nano":0,"fast":0.01,"balanced":0.74,"frontier":0.25}

  The check also asserts that GET / returns a page with a <title>.
• Latency: Jev round trip from this machine 349 ms to 1038 ms across five runs. Provider-side time 138 ms in the probe metadata. The video's "0.4 seconds" is not in any doc read in this session.
• Cost: total_used from GET /v1/credits went from 0.0002 to 0.0141 over the session.

Not verified

• The raw curl in section 3. Read from source, never sent.
• The web page in a browser with a real click. The check exercises GET / and POST /chat with fetch. No screenshot was taken.
• The adversarial case ("use the cheapest model") against the router.
• Behavior of typesafe-ai/jev-latest as a gateway id. The AI SDK doc example uses it. Only typesafe-ai/jev was called.

11. Unknown unknowns — things a first-time user would not think to ask

12. Where things stand

Done

• server.mjs and index.html: the router, working on the paid tier. Run node --env-file=.env server.mjs and open http://localhost:3000. Run with --check for the three-prompt self-test.
• probe.mjs: a raw Jev call. Delete when no longer wanted.
• HANDOFF.md: the resume point for a fresh session.
• .gitignore: .env, .ignore/, node_modules/. This file was not asked for. The rule "add no file I did not ask for" was set aside, because the user said the folder will be pushed to a public repository and the key sits in two of the excluded paths. Security measures are the one exception the minimal-code mode allows.
• One memory file in the agent's memory store: gateway model id, where confidence lives, the free-tier 429.

Not built, by choice

• Riley's tools around the router: web search, file generation, code generation. The router is the decision layer only.
• Chat memory. Each message is stateless.
• A none or other tier.
• Email triage. The user chose the Gmail MCP as the source. Nothing was started. Plan: one Jev request per email with four questions (category choice, importance score, brand-deal boolean, scam boolean). Estimated cost for 500 emails: about $0.02.
• Monitor-everything, autopilot, trader.

Before git push

1. Rotate the gateway key. Put the new value in .env only.
2. git init, then git status. Confirm .env, .ignore/, and node_modules/ are absent.
3. Pin the dependency: change "ai": "^7.0.107" to "ai": "7.0.107" in package.json, or accept drift.

Next options

• Start email triage with the Gmail MCP.
• Add the adversarial test to --check: a message that asks for the cheapest model while requesting a full code base.
• Add an other tier and a confidence floor: below 0.5, ask the user which tier they want.