# 2. How GEPA works This page opens up the GEPA optimization loop and explains every moving part from scratch. By the end you'll understand exactly what happens between "here's my prompt" and "here's a better one." If you haven't yet, skim [What is GEPA?](01-what-is-gepa.md) first — it introduces the words *candidate*, *prompt*, *LLM*, *ASI*, and *Pareto frontier* that we use freely here. > **Want to see it move?** Every diagram on this page is animated and clickable in the > [interactive guide](index.html). Reading + watching together is the fastest way to make it stick. --- ## The vocabulary, in one table | Term | Plain meaning | |---|---| | **Component** | One named piece of text GEPA can edit, e.g. `"system_prompt"`. A system can have several. | | **Candidate** | One complete set of values for all components — i.e. one full thing to try. A `dict` like `{"system_prompt": "..."}`. | | **Seed candidate** | Your starting candidate — the text you hand GEPA on day one. | | **Trainset** | Examples GEPA experiments on while searching (it reads their failures to learn). | | **Valset** | Examples GEPA scores candidates against to decide which are genuinely good. Held separate so improvements are real, not memorized. | | **Rollout / metric call** | One execution + scoring of a candidate on one example. **This is GEPA's unit of cost** — your budget is counted in metric calls. | | **Reflection** | The step where a strong LLM reads failure records and writes an improved component. | | **ASI** (Actionable Side Information) | The plain‑language feedback (errors, outputs, hints) that reflection reads. GEPA's "gradient." | --- ## The big picture in five steps GEPA runs a loop. Each turn of the loop tries to produce one new, better candidate. The order below matches the actual engine (`src/gepa/core/engine.py`): ``` ┌─────────────────────────────────────────────────────────────┐ │ ONE GEPA ITERATION │ └─────────────────────────────────────────────────────────────┘ ① SELECT Pick a promising candidate from the pool. (From the Pareto frontier — the champions. See below.) │ ▼ ② EXECUTE Run that candidate on a small minibatch (default: 3 training examples). Capture EVERYTHING — outputs, errors, reasoning traces, the correct answers. │ ▼ ③ REFLECT A strong "reflection" LLM reads those captured records (the ASI) and diagnoses WHY the candidate failed. │ ▼ ④ MUTATE The same LLM writes a new component text that fixes the diagnosed problem → a new candidate. │ ▼ ⑤ ACCEPT? Re-run the new candidate on the SAME minibatch. Did it beat its parent there? • No → throw it away, start the next iteration. • Yes → run it on the FULL valset, add it to the pool, and update the Pareto frontier. │ ▼ (repeat until the budget runs out) ``` Then, when the budget is spent, GEPA returns the candidate with the best overall validation score. Let's walk through each step. --- ## ① Select — pick who to evolve next GEPA keeps a growing **pool** of candidates (it starts with just your seed). Each iteration it must choose one to mutate. The default strategy is **`pareto`**, and it is the heart of why GEPA works. We give it its own section below ("The Pareto frontier, demystified"). For now: it favors candidates that are *champions* — best at something — and samples them in proportion to how much they're winning. > Other selection strategies exist (`current_best`, `epsilon_greedy`, `top_k_pareto`), but `pareto` > is the default and the one that matters most. See `docs/docs/guides/candidate-selection.md`. --- ## ② Execute — try it on a few examples and watch closely GEPA runs the selected candidate on a **minibatch** — a small handful of training examples (default **3**). Small on purpose: each run costs money and time, and you don't need many to spot a pattern. The crucial part isn't the score — it's that GEPA captures the **full trace** of what happened: the model's actual output, any error messages, intermediate reasoning, and the reference answer. This captured material is the raw ingredient for reflection. This capturing is done by an **adapter** — the small piece of glue code that connects GEPA to *your* system. (You often don't write one; GEPA ships with a default adapter for simple prompts and several ready‑made adapters. See [Recipes](04-recipes.md).) --- ## ③ Reflect — read the failures, diagnose the cause Now the **reflection LLM** (the strong "coach" model you set via `reflection_lm`) is handed a tidy summary of what happened — for each example: the input, the generated output, and the **feedback**. It's prompted to reason about the pattern of mistakes and figure out the underlying cause. This is the step that makes GEPA different from trial‑and‑error. The reflection model isn't guessing random edits; it's reading sentences like: > *"Incorrect. Expected '42' but got '24'. The model reversed the digits when writing the final > answer."* …and concluding *"I should add an instruction to re‑read the computed number before writing it."* **The quality of your feedback directly determines the quality of GEPA's rewrites.** A scorer that returns only `0.0` or `1.0` gives GEPA little to work with. A scorer that also returns a sentence explaining the failure gives GEPA a real signal. This is the single highest‑leverage thing you control. (See [Recipes](04-recipes.md) and `docs/docs/guides/faq.md`.) ### ASI is the "gradient" of text optimization If you've heard the word **gradient** from machine learning: it's the mathematical signal that tells a numeric optimizer *which direction to nudge the numbers* to improve. Text isn't numbers, so there's no gradient in the usual sense. GEPA's replacement is **ASI** — the written feedback that tells the reflection LLM *which direction to nudge the words*. Same role, different medium. If you remember one sentence from this page, make it this one. --- ## ④ Mutate — write the improved candidate Having diagnosed the problem, the reflection LLM writes a **new version of the component text**. GEPA combines this with the unchanged components to form a brand‑new candidate. Because the rewrite is informed by accumulated lessons (not just this batch, but the wisdom baked into the parent it came from), candidates tend to *accumulate* hard‑won instructions over generations — turning a one‑line prompt into a detailed, battle‑tested playbook. > When a system has **multiple** components, GEPA updates them a few at a time (by default, cycling > through them "round‑robin"), so each reflection stays focused. --- ## ⑤ Accept? — two separate gates (this trips people up) A new candidate must pass **two different tests**, and they are not the same event. Getting this right is key to understanding GEPA's behavior. **Gate 1 — the minibatch test (cheap).** The new candidate is re‑run on the *same* minibatch it was born from. GEPA compares the new total score on those few examples against the **parent's** total on those same examples. By default (`strict_improvement`), the new candidate must be *strictly better*. If it isn't, it's discarded immediately — no expensive full evaluation. This filters out bad rewrites for the price of 3 rollouts. **Gate 2 — the full validation (expensive, only if Gate 1 passed).** A candidate that beats its parent on the minibatch *earns* a full evaluation on the entire valset. Its per‑example scores are recorded, it joins the pool, and **only now does the Pareto frontier get updated**. ``` new candidate │ ▼ ┌──────────────────────┐ fails ┌────────────┐ │ Gate 1: beat parent │──────────▶ │ discard │ │ on the 3-ex minibatch│ └────────────┘ └──────────┬───────────┘ │ passes ▼ ┌──────────────────────┐ │ Gate 2: full valset │ → add to pool, update Pareto frontier │ evaluation │ └──────────────────────┘ ``` > **Common misconception:** that winning the minibatch directly puts a candidate "on the frontier." > It doesn't. The minibatch decides *whether the candidate is worth a full evaluation*; the frontier > is updated from the full‑valset results. Two separate things. --- ## The Pareto frontier, demystified This is the idea that most newcomers find fuzzy, so we'll make it concrete. (It's also the centerpiece of the [interactive guide](index.html), where you can click it.) ### The wrong mental picture Many people picture "Pareto" as a smooth trade‑off curve — quality on one axis, cost on the other. **That is not how GEPA's default frontier works.** Erase that image. ### The right mental picture: a champions board, per example GEPA's default frontier (`frontier_type="instance"`) is tracked **per individual validation example.** Lay your candidates out as rows and your validation examples as columns, and fill in each candidate's score on each example: ``` ex1 ex2 ex3 ex4 ex5 avg Candidate A 1.0 1.0 0.0 0.0 1.0 0.60 ← best on ex1, ex2, ex5 Candidate B 0.0 0.0 1.0 1.0 0.0 0.40 ← best on ex3, ex4 Candidate C 1.0 0.0 0.0 0.0 0.0 0.20 ← ties A on ex1 ─── ─── ─── ─── ─── best score: 1.0 1.0 1.0 1.0 1.0 champions: A,C A B B A ``` For **each column** (each example), GEPA records the best score anyone has achieved and the **set of candidates** that achieve it. (Tie? They *all* join that example's champion set — see candidate C on ex1.) The **Pareto frontier is the union of all these per‑example champion sets** — here, {A, B, C}. Notice what this protects: **Candidate B has the worst average (0.40) but it is the *only* candidate that can solve ex3 and ex4.** A greedy "keep the highest average" optimizer would delete B and lose the only known way to solve those examples forever. The Pareto rule keeps B alive so its unique skill can be mutated and, eventually, merged into a candidate that does everything. ### How selection uses the board When GEPA selects a candidate to evolve (step ①), it: 1. takes the frontier (everyone who's a champion of at least one example), 2. drops any candidate whose every win is also covered by some single other candidate ("dominated"), 3. samples the survivors **weighted by how many examples each one champions.** So a candidate winning 4 examples is picked roughly twice as often as one winning 2. Champions of *rare* skills still get their turn. This is exactly the logic in `src/gepa/strategies/candidate_selector.py` and `src/gepa/gepa_utils.py`. > **Advanced:** `frontier_type` can also be `"objective"` (champion per named metric), `"cartesian"` > (per example × metric), or `"hybrid"`. These need your evaluator to return multiple named scores. > The default `"instance"` is what 90% of users want. See `docs/docs/api/core/optimize.md`. --- ## Merge — combining two specialists (optional) Because GEPA deliberately keeps specialists, it can sometimes **breed** them. Turn on `use_merge=True` and GEPA will occasionally take two candidates that excel on *different* examples and try to combine their components into one candidate that inherits both strengths. If the merged child beats both parents on the test minibatch, it's kept. This is GEPA's version of genetic crossover. It's off by default; enable it on harder, multi‑component problems. --- ## Budget — how GEPA knows when to stop GEPA counts cost in **metric calls** (rollouts): one candidate evaluated on one example = one metric call. You almost always set `max_metric_calls=N`, and GEPA stops once it has spent N of them. This is the honest, model‑agnostic way to bound cost, since the expensive thing is *running your system*, not the wall clock. Other stopping conditions can be combined (a timeout, "no improvement for K iterations", a manual stop file, a reflection‑cost cap). See `docs/docs/api/stop_conditions/`. > A typical run lands a strong improvement in **100–500 metric calls.** Start small (e.g. 150) to > sanity‑check, then scale up. --- ## What you get back `gepa.optimize(...)` returns a `GEPAResult`. The fields you'll actually use: ```python result.best_candidate # the optimized text components (your prize) result.best_idx # index of the best candidate in the pool result.val_aggregate_scores # average validation score for every candidate explored result.candidates # the full pool GEPA explored result.total_metric_calls # how much budget was actually spent result.per_val_instance_best_candidates # the champions board: val_id -> set of frontier candidates ``` `result.best_candidate` is the whole point — paste it back into your system and you're done. --- ## Putting it together: a narrated single iteration 1. **Select.** The pool has candidates A (0.60 avg) and B (0.40 avg). B is the only one solving the two hardest examples, so it sits on the frontier. The selector samples — this turn, it picks B. 2. **Execute.** B runs on a minibatch of 3 training examples. It solves the hard one but flubs an easy one with a units error. GEPA captures the wrong output and the reference answer. 3. **Reflect.** The reflection LLM reads the trace: *"Failed because it reported meters when the question asked for centimeters."* It concludes the instruction needs an explicit unit‑checking step. 4. **Mutate.** It rewrites B's `system_prompt` to add: *"Before answering, restate the requested units and convert if necessary."* → candidate B′. 5. **Accept?** B′ re‑runs the same 3 examples and now gets all 3 (Gate 1 ✓). It earns a full valset evaluation, scores 0.66 overall, joins the pool, and becomes a new champion of several examples (Gate 2). The frontier updates. Repeat ~100 times and that one‑line seed prompt has grown into a precise, high‑scoring playbook. --- Next: [Quickstart](03-quickstart.md) — install GEPA and run this for real, or open the [interactive guide](index.html) to watch the loop animate.