Session look-back: which of your recent prompts a quick Jev call could have handled

Date: 2026-10-03  ·  Code: lookback/lookback.mjs  ·  Model: typesafe-ai/jev through the Vercel AI Gateway

1. What Jev decides

What kind of step each of your recent Claude Code prompts is. One choice question per prompt, with seven kinds: classify_route, pick_from_list, yes_no_check, score_rank, write_or_build, research_or_multistep, chat. The first four are decision kinds. A prompt is a Jev candidate when Jev picks a decision kind at probability 0.6 or more. The script writes lookback/report.md with the counts and the candidate prompts.

2. The questions

// ---- Review these. Everything else is plumbing. ----
const PER_SESSION = 40;    // prompts read per session
const MAX_CHARS = 300;     // characters of each prompt sent to Jev
const WORKERS = 10;        // calls in flight
const MIN_P = 0.6;         // a prompt is a Jev candidate when it is a decision kind with probability at least this
const INSTRUCTIONS = "What kind of step is this request from a user to a coding agent?";

3. Run it

node --env-file=.env lookback/lookback.mjs --sessions 20 --dry   # count only, sends nothing
node --env-file=.env lookback/lookback.mjs --sessions 20
node lookback/lookback.mjs --check                                # free

The prompts, each cut to 300 characters, go to the gateway. Run --dry first. The report is in .gitignore because it contains your prompts.

4. The check, verbatim

ok: extractor keeps 2 of 7 lines and respects the limit

The check is free and calls no model. It tests the prompt extractor on a fixture: it keeps plain user prompts and skips meta lines, command tags, tool results and prompts shorter than 8 characters.

5. Findings

• The dry run on 2026-10-03 found 127 prompts in the 20 most recent sessions on the author's machine.
• A live run on 2026-10-04 labelled 113 prompts from 20 sessions for $0.0025. The counts-only write-up is the prompt look-back report.
• Not done: Jev's labels have not been compared with hand labels. Only the 12 candidates were read by hand.
• Jev labels a request from its first 300 characters only. It does not see the replies or the files, so a short follow-up like "do the second one" is labelled without its context.