Burst probe: 50 Jev calls at once

Date: 2026-09-19  ·  Code: probe-burst.mjs  ·  Model: typesafe-ai/jev through the Vercel AI Gateway

1. What Jev decides

Nothing useful. Each call asks one boolean question, "Is the number even?", about one integer. The point is the gateway, not the answer: how many of 50 parallel requests return, how long the burst takes, and how the per-call latency spreads. Every parallel plan in this repo (the town, the sorter, the log monitor) depends on the answer.

2. The questions

The whole file. This is the block a reviewer must read; the rest of the file is plumbing.

// Fire N parallel Jev calls. Run: node --env-file=.env probe-burst.mjs [N]
// Prints how many returned, the burst wall time, and per-call latency spread.
import { experimental_evaluate as evaluate } from "ai";
const N = Number(process.argv[2]) || 50;
const questions = { even: { type: "boolean", instructions: "Is the number even?" } };
const t0 = performance.now();
const rs = await Promise.allSettled(Array.from({ length: N }, (_, n) => {
  const s = performance.now();
  return evaluate({ model: "typesafe-ai/jev", state: { n }, questions }).then((r) => ({ ms: performance.now() - s, p: r.answers.even.probability }));
}));
const burst = Math.round(performance.now() - t0);
const ok = rs.filter((r) => r.status === "fulfilled").map((r) => r.value);
const ms = ok.map((r) => r.ms).sort((a, b) => a - b);
console.log(`${ok.length}/${N} ok, burst ${burst} ms, per call min ${Math.round(ms[0])} median ${Math.round(ms[ms.length >> 1])} max ${Math.round(ms.at(-1))} ms`);
const wrong = ok.filter((r, i) => (r.p >= 0.5) !== (i % 2 === 0)).length;
console.log(`even/odd wrong: ${wrong}/${ok.length} (p rounded at 0.5)`);
for (const r of rs.filter((r) => r.status === "rejected")) console.log("fail:", String(r.reason?.message ?? r.reason).slice(0, 200));

3. Run it

node --env-file=.env probe-burst.mjs 50

4. The check, verbatim

48/50 ok, burst 7760 ms, per call min 1079 median 1230 max 7739 ms
even/odd wrong: 4/48 (p rounded at 0.5)
fail: Failed after 3 attempts. Last error: GatewayInternalServerError: Service temporarily unavailable. Please try again shortly.
fail: Failed after 3 attempts. Last error: GatewayInternalServerError: Service temporarily unavailable. Please try again shortly.

5. Findings

• 48 of 50 returned. The two failures were 503 responses that survived the SDK's three retries, which span about 6 s.
• The median per call inside the burst was 1,230 ms. Sequential calls in the other projects took 300 to 700 ms. Parallel load roughly doubles the per-call time.
• The 7,739 ms maximum is a call that hit 503, retried three times, and then succeeded.
• Jev got 4 of 48 even/odd answers wrong at a 0.5 cut. The question is a joke, but it shows that a boolean probability near 0.5 is not a coin the caller should trust.
• Consequence: lib.mjs retries four more times at 3, 6, 12, and 24 s when the SDK gives up. The town fires all 50 at once and draws a failed character grey instead of crashing.

6. Cost and latency

50 boolean calls: about $0.0006 in total, $0.0000116 each.