chore: import upstream snapshot with attribution

scripts/agent-eval/offload-eval.md

@@ -0,0 +1,76 @@
+# CodeGraph AI offload — accuracy & adoption eval harness
+
+Measures the managed **offload** (`codegraph_explore` → reasoning model synthesis) and the
+**front-load hook** (approach 1) against plain codegraph and no-codegraph, across repo sizes,
+on **time · main-session tokens/cost · CodeGraph-AI tokens/cost · accuracy**.
+
+All agent arms run `claude -p --model sonnet --effort high` (the deliberate floor model — an
+affordance that lands on Sonnet generalizes up). Everything writes to a scratch dir
+(`AGENT_EVAL_OUT`, default `/tmp/cg-offload-eval`); nothing here is shipped to users.
+
+## Repos (selected via a memory-probe gate — NOT trained on)
+
+Famous repos (express, excalidraw, n8n, …) are useless for *accuracy* evals: Sonnet answers their
+flow questions from memory, so the no-codegraph baseline is dishonest. These four passed a no-tools
+probe (Sonnet could not name their real flow internals) and are cloned fresh by `offload-eval-setup.sh`:
+
+| tier | repo | ~src files | canonical flow |
+|---|---|---|---|
+| small | MTKruto/MTKruto | 322 TS | `sendMessage` → invoke → TL serialize → transport |
+| medium | mvdicarlo/postybirb-plus | 608 TS | submission → queue → per-website `.post()` |
+| complex | shapeshift/web | 3.2k TS (35-pkg monorepo) | swap → swapper registry → concrete swapper |
+| large | trezor/trezor-suite | 8k TS monorepo | send-form → sign thunk → `@trezor/connect` |
+
+Verified ground-truth flows (the judge's reference) live in `offload-eval-ground-truth.json`.
+
+## Arms
+
+- **offload** — codegraph + managed offload ON (requires `codegraph login`); records AI tokens/credits via `CODEGRAPH_OFFLOAD_USAGE_LOG`.
+- **raw** — codegraph, `CODEGRAPH_OFFLOAD_DISABLE=1` (returns raw source).
+- **nocg** — empty MCP config; Read/Grep baseline.
+- **frontload** — codegraph (offload-disabled) + a `UserPromptSubmit` hook (`offload-eval-hook.mjs`) that runs raw explore on the prompt and injects the result into context (approach 1).
+
+## Run it
+
+```bash
+npm run build                       # the harness shells out to dist/
+codegraph login                     # only needed for the offload arm
+export AGENT_EVAL_OUT=/tmp/cg-offload-eval
+
+bash scripts/agent-eval/offload-eval-setup.sh            # clone + index the 4 repos
+bash scripts/agent-eval/offload-eval-matrix.sh           # 3 arms × 4 tiers × REPS (default 3)
+node scripts/agent-eval/offload-eval-judge.mjs \
+     --results $AGENT_EVAL_OUT/results.jsonl \
+     --truth  scripts/agent-eval/offload-eval-ground-truth.json \
+     --out    $AGENT_EVAL_OUT/judged.jsonl
+node scripts/agent-eval/offload-eval-summarize.mjs $AGENT_EVAL_OUT/judged.jsonl
+
+bash scripts/agent-eval/offload-eval-frontload-matrix.sh # frontload arm + judge + merged summary
+```
+
+Single repo: `offload-eval-3arm.sh <indexed-repo> <tier> <reps> "<question>"` (or `-frontload.sh`).
+
+## Files
+
+- `offload-eval-setup.sh` — clone + index the 4 repos.
+- `offload-eval-3arm.sh` / `-frontload.sh` — one repo, the arms.
+- `offload-eval-matrix.sh` / `-frontload-matrix.sh` — drive all 4 tiers.
+- `offload-eval-hook.mjs` — the front-load `UserPromptSubmit` hook (resolves its own engine; `CG_FRONTLOAD_DEBUG=<path>` to log injections; `CG_FRONTLOAD_BUDGET` to cap injected chars).
+- `offload-eval-metrics.mjs` — one run's stream-json + usage log → one JSON metrics line.
+- `offload-eval-judge.mjs` — Sonnet judge: end-to-end (agent final vs ground truth) + per-answer offload fidelity.
+- `offload-eval-summarize.mjs` — per-tier, per-arm table + cross-repo roll-up.
+- `offload-eval-ground-truth.json` — source-verified canonical flows.
+
+## Findings (2026-06, n=3 — direction consistent, magnitudes noisy)
+
+- **Raw codegraph is the efficiency win** — ~nocg accuracy, fewer reads, faster, no AI cost.
+- **The offload is the least-accurate arm in all 4 tiers** — synthesized fidelity 12–27/100 with
+  fabrication in 3/4 (e.g. invented website services; traced `ClientPlain`/`SessionPlain` instead of
+  the real encrypted path). Its speed/cost win is narrow (medium-only) and inversely correlated with
+  accuracy. **Use raw until offload fidelity is fixed.**
+- **The front-load hook SOLVES adoption** — reads → 0–1 in every tier (incl. large, where the agent
+  otherwise read 12–24 files); fired 12/12, 0 errors. Wins on medium/complex (100% pass). But it
+  **regresses small/large to partial** — it suppresses the reads that compensate for explore's gaps at
+  **dynamic boundaries** (async queues, redux thunks, facade/factory indirection).
+- **Master lever for BOTH:** explore's dynamic-dispatch coverage. Fix it → front-load is complete
+  everywhere and the offload has the full flow to synthesize.