chore: import upstream snapshot with attribution

scripts/README.riscv64-smoke.md

@@ -0,0 +1,143 @@
+# RISC-V 64 cross-build smoke harness
+
+Two scripts wire up a one-shot **build → QEMU smoke → JSON report**
+pipeline for every riscv64 native artifact the repo ships:
+
+| script                                 | role                                                              |
+| -------------------------------------- | ----------------------------------------------------------------- |
+| `scripts/build-riscv64-artifacts.sh`   | drive every cross-build that produces a riscv64 ELF               |
+| `scripts/check-riscv64-artifacts.sh`   | exercise each produced artifact under `qemu-riscv64-static`       |
+
+Both wrappers are exposed at the repo root as bun scripts:
+
+```bash
+ELIZA_RISCV64_SMOKE=1 bun run build:riscv64-artifacts
+ELIZA_RISCV64_SMOKE=1 bun run check:riscv64-artifacts
+```
+
+With `ELIZA_RISCV64_SMOKE` unset they are intentional no-ops — that
+keeps the default CI lane cheap, and the smoke is gated to riscv64-
+specific branches / workflow_dispatch runs.
+
+The smoke harness writes a machine-readable report to
+`build/reports/riscv64_artifacts.json`. Top-level shape:
+
+```json
+{
+  "generated_at": "2026-…",
+  "summary": {"pass": 53, "fail": 0, "skip": 0},
+  "final_status": "PASS",
+  "artifacts": [
+    {"path": "…/libqjl.a", "kind": "static-archive", "status": "PASS", "detail": "…", "duration_ms": 0},
+    {"path": "…/qjl_int8_smoke", "kind": "executable", "status": "PASS", "detail": "qemu exit=0", "duration_ms": 412},
+    …
+  ]
+}
+```
+
+## What's exercised
+
+- **Native plugins** (`packages/native/plugins/<pkg>/build/riscv64/`)
+  for `qjl-cpu`, `polarquant-cpu`, `turboquant-cpu`, `silero-vad-cpp`,
+  `voice-classifier-cpp`, `wakeword-cpp`, `yolo-cpp`, `face-cpp`,
+  `doctr-cpp`. Each plugin contributes one `.a`, an optional `.so`,
+  and a handful of GoogleTest-driven smoke executables.
+
+- **`libllama` + `libggml` family + `libeliza-llama-shim.so`** —
+  MTP llama.cpp cross-build via
+  `packages/app-core/scripts/aosp/compile-libllama.mjs --target
+  linux-riscv64-cpu` and `--target android-riscv64-cpu`.
+
+- **`libsigsys-handler.so`** for riscv64 — the Bun seccomp shim,
+  built by `packages/app-core/scripts/aosp/compile-shim.mjs --abi
+  riscv64` into `~/.cache/eliza-android-agent/seccomp-shim/riscv64/`.
+
+## Operator recipe (Debian/Ubuntu)
+
+```bash
+# 1. Toolchain.
+sudo apt-get update
+sudo apt-get install -y \
+    qemu-user-static binfmt-support \
+    cmake build-essential \
+    file binutils
+# Zig 0.14+ — pick the release for your host arch.
+curl -fsSL https://ziglang.org/download/0.14.0/zig-linux-x86_64-0.14.0.tar.xz \
+    | sudo tar -C /opt -xJ
+sudo ln -sf /opt/zig-linux-x86_64-0.14.0/zig /usr/local/bin/zig
+zig version       # → 0.14.0
+
+# 2. Optional: Android NDK r27+ (only needed for android-riscv64-cpu).
+#    Skip this if you're only validating the Linux riscv64 lane.
+curl -fsSL https://dl.google.com/android/repository/android-ndk-r27c-linux.zip -o ndk.zip
+unzip -q ndk.zip -d /opt
+export ANDROID_NDK_HOME=/opt/android-ndk-r27c
+
+# 3. Smoke.
+cd /path/to/eliza
+ELIZA_RISCV64_SMOKE=1 bun run build:riscv64-artifacts
+ELIZA_RISCV64_SMOKE=1 bun run check:riscv64-artifacts
+jq '.summary, .final_status' build/reports/riscv64_artifacts.json
+```
+
+## QEMU-only run (skip the build phase)
+
+If sister-agent worktrees have already cross-compiled the artifacts and
+the on-disk paths are still present, you can run the smoke harness
+alone:
+
+```bash
+ELIZA_RISCV64_SMOKE=1 bun run check:riscv64-artifacts
+```
+
+Missing artifacts are reported as `SKIP` records with a reason, not
+`FAIL` — so the harness is safe to re-run incrementally as each
+upstream build comes online.
+
+## ELF-tag-only mode (no QEMU)
+
+For tier-1 smoke that only validates *every artifact is the right ELF
+arch* (cheap, ~seconds, no QEMU required):
+
+```bash
+ELIZA_RISCV64_SMOKE=1 bash scripts/check-riscv64-artifacts.sh --no-qemu
+```
+
+This mode confirms `ELF 64-bit LSB ... UCB RISC-V ... double-float ABI`
+on every shared library + executable and inspects `.a` archive members,
+but does not run any executable.
+
+## Exit codes
+
+| code | meaning                                          |
+| ---- | ------------------------------------------------ |
+| 0    | every artifact PASSed or SKIPped with reason     |
+| 1    | at least one artifact FAILed                     |
+| 2    | invalid CLI args / missing toolchain (build-only)|
+
+## CI integration
+
+`.github/workflows/riscv64-smoke.yml` runs the full build + smoke on:
+
+- `workflow_dispatch` (manual trigger).
+- Any branch with the `riscv64` label applied to its PR.
+
+The smoke is **not** a required check on `develop` PRs. It is opt-in
+because (a) the build takes ~30-60 min on a typical CI runner and
+(b) the upstream `oven-sh/bun#6266` blocker for the Bun riscv64
+binary is independent of this work.
+
+## Why two scripts (not one)
+
+Splitting build from check has three concrete benefits:
+
+1. **Cache awareness.** If sister-agent worktrees pre-built artifacts
+   into the canonical paths (`packages/native/plugins/*/build/riscv64/`,
+   `build/riscv64-stage/riscv64/`, etc.), the build phase no-ops and
+   the smoke phase exercises what's already on disk. No double work.
+2. **CI cost.** A workflow can run *just* the smoke on a runner that
+   has the artifacts pre-staged, e.g. via an artifact upload from a
+   prior build job.
+3. **Local dev.** When iterating on a single plugin's RVV kernels you
+   only need `bash scripts/check-riscv64-artifacts.sh`; the build
+   driver doesn't get in the way.

scripts/ai-qa/review-lib.mjs

@@ -0,0 +1,138 @@
+/**
+ * Pure logic for the AI-QA vision screenshot review (#9304).
+ *
+ * Today the "screenshot review" layer is a color-bucket + blank-detection
+ * heuristic — it never looks at what the UI actually shows. This module is the
+ * dependency-free core of a real content review: it builds the per-capture
+ * vision prompt, parses the model's structured verdict, aggregates the run, and
+ * decides the gate. The live model call + filesystem I/O live in
+ * `review-screenshots.mjs`; everything here is pure so it is unit-tested without
+ * a network or an API key.
+ */
+
+/** @typedef {"good" | "needs-work" | "broken"} VisionVerdict */
+
+/**
+ * Build the review prompt for one capture. `expectation` is the route's human
+ * label (what the page is supposed to be); `issues` are the console/page errors
+ * the capture already recorded (fed to the model as corroborating signal).
+ * @param {{ label: string, path: string, viewport: string, theme: string, issues?: string[] }} capture
+ */
+export function buildReviewPrompt(capture) {
+  const issues = (capture.issues ?? []).slice(0, 8);
+  return [
+    `You are reviewing a screenshot of the "${capture.label}" page (${capture.path}) of the elizaOS dashboard, captured at the ${capture.viewport} viewport in ${capture.theme} theme.`,
+    "",
+    "Judge what is ACTUALLY VISIBLE — not what should be there. Check for:",
+    "- Render failure: blank/one-color, an error overlay, a stack trace, a spinner that never resolved, or obviously missing content for this page.",
+    "- Layout breaks: overlapping elements, text clipped/cut off, content overflowing its container, a collapsed/zero-height region, controls off-screen.",
+    "- Brand: the accent must be orange; there must be NO blue used as an accent/interactive color (neutral grays are fine). Flag any blue accent.",
+    "- Usability: unreadable contrast, placeholder/lorem text, raw untranslated i18n keys (e.g. `common.save`), leaked markup (e.g. a literal `[CONFIG:` or `[TASK:` marker).",
+    "",
+    issues.length
+      ? `The capture harness also recorded these console/page signals: ${JSON.stringify(issues)}.`
+      : "The capture harness recorded no console/page errors.",
+    "",
+    'Respond with ONLY a JSON object, no prose, in this exact shape: {"verdict":"good"|"needs-work"|"broken","reasons":string[],"layoutIssues":string[],"brandViolations":string[],"detectedText":string}. Use "broken" for a render failure, "needs-work" for a real visual/brand/usability defect, "good" otherwise. Keep arrays empty when nothing applies.',
+  ].join("\n");
+}
+
+const VALID_VERDICTS = new Set(["good", "needs-work", "broken"]);
+
+/**
+ * Parse a vision model's response into a typed verdict. Tolerates the model
+ * wrapping the JSON in prose / code fences by extracting the first balanced
+ * object. Throws on anything that is not a usable verdict (a parse failure is a
+ * real signal — the review did not happen — not something to paper over).
+ * @param {string} text
+ */
+export function parseVisionVerdict(text) {
+  if (typeof text !== "string" || !text.trim()) {
+    throw new Error("vision-review: empty response");
+  }
+  const start = text.indexOf("{");
+  const end = text.lastIndexOf("}");
+  if (start < 0 || end <= start) {
+    throw new Error(
+      `vision-review: no JSON object in response: ${text.slice(0, 120)}`,
+    );
+  }
+  let parsed;
+  try {
+    parsed = JSON.parse(text.slice(start, end + 1));
+  } catch (err) {
+    throw new Error(`vision-review: unparseable JSON: ${err?.message || err}`);
+  }
+  if (
+    !parsed ||
+    typeof parsed !== "object" ||
+    !VALID_VERDICTS.has(parsed.verdict)
+  ) {
+    throw new Error(
+      `vision-review: missing/invalid verdict: ${JSON.stringify(parsed).slice(0, 120)}`,
+    );
+  }
+  const arr = (v) =>
+    Array.isArray(v) ? v.filter((x) => typeof x === "string") : [];
+  return {
+    verdict: parsed.verdict,
+    reasons: arr(parsed.reasons),
+    layoutIssues: arr(parsed.layoutIssues),
+    brandViolations: arr(parsed.brandViolations),
+    detectedText:
+      typeof parsed.detectedText === "string" ? parsed.detectedText : "",
+  };
+}
+
+/** Aggregate per-capture results into run totals. */
+export function aggregateVerdicts(results) {
+  const totals = {
+    total: results.length,
+    good: 0,
+    "needs-work": 0,
+    broken: 0,
+    error: 0,
+  };
+  for (const r of results) {
+    const key = r.error ? "error" : r.verdict;
+    if (key in totals) totals[key] += 1;
+  }
+  return totals;
+}
+
+/**
+ * Gate decision: the captures that should fail the run. A `broken` verdict (or
+ * an `error` — the review could not be obtained) fails unless its
+ * `${id}-${viewport}-${theme}` key is in the shrinking debt allowlist. In strict
+ * mode `needs-work` also fails. Mirrors the aesthetic-audit / story-gate ratchet.
+ * @returns {Array<{key:string, verdict:string, reasons:string[]}>}
+ */
+export function gateFailures(results, { debt = {}, strict = false } = {}) {
+  return results
+    .filter((r) => {
+      const fail =
+        r.error ||
+        r.verdict === "broken" ||
+        (strict && r.verdict === "needs-work");
+      return fail && debt[r.key] === undefined;
+    })
+    .map((r) => ({
+      key: r.key,
+      verdict: r.error ? "error" : r.verdict,
+      reasons: r.error
+        ? [String(r.error)]
+        : [
+            ...(r.reasons ?? []),
+            ...(r.layoutIssues ?? []),
+            ...(r.brandViolations ?? []),
+          ],
+    }));
+}
+
+/** The Anthropic image content block for a base64 PNG. */
+export function imageBlock(base64Png) {
+  return {
+    type: "image",
+    source: { type: "base64", media_type: "image/png", data: base64Png },
+  };
+}

scripts/ai-qa/review-screenshots.mjs

@@ -0,0 +1,243 @@
+#!/usr/bin/env node
+
+/**
+ * AI-QA vision screenshot review (#9304).
+ *
+ * Closes the gap the audit named: the existing "review" (color buckets + blank
+ * detection) never looks at WHAT the UI shows. This sends each captured
+ * screenshot + its per-view expectation to a vision model and records a
+ * structured verdict (good / needs-work / broken + reasons / layout issues /
+ * brand violations / detected text), then gates on `broken` via a shrinking
+ * debt allowlist — the same ratchet as the story gate + aesthetic audit.
+ *
+ * It consumes the output of `ai-qa-capture.spec.ts`
+ * (`reports/ai-qa/<runId>/{manifest.json, captures/<id>/<id>__<vp>__<theme>.{png,json}}`).
+ *
+ * OPT-IN + CI-SAFE: requires `ANTHROPIC_API_KEY`. With no key it logs and exits
+ * 0 (the keyless PR lane is unaffected); wire it into a keyed nightly lane to
+ * actually review. Model: `AI_QA_VISION_MODEL` (default a vision-capable Haiku).
+ *
+ * Usage:
+ *   node scripts/ai-qa/review-screenshots.mjs [--run-dir reports/ai-qa/<id>]
+ *     [--concurrency 4] [--strict] [--update-debt]
+ */
+
+import { existsSync } from "node:fs";
+import { readdir, readFile, writeFile } from "node:fs/promises";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import {
+  aggregateVerdicts,
+  buildReviewPrompt,
+  gateFailures,
+  imageBlock,
+  parseVisionVerdict,
+} from "./review-lib.mjs";
+
+const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
+const MODEL = process.env.AI_QA_VISION_MODEL || "claude-haiku-4-5-20251001";
+const ANTHROPIC_VERSION = "2023-06-01";
+
+function parseArgs(argv) {
+  const a = { runDir: null, concurrency: 4, strict: false, updateDebt: false };
+  for (let i = 0; i < argv.length; i++) {
+    const arg = argv[i];
+    if (arg === "--run-dir") a.runDir = argv[++i];
+    else if (arg === "--concurrency") a.concurrency = Number(argv[++i]);
+    else if (arg === "--strict") a.strict = true;
+    else if (arg === "--update-debt") a.updateDebt = true;
+  }
+  return a;
+}
+
+async function latestRunDir() {
+  const base = join(REPO_ROOT, "reports", "ai-qa");
+  if (!existsSync(base)) return null;
+  const entries = (await readdir(base, { withFileTypes: true }))
+    .filter((e) => e.isDirectory())
+    .map((e) => e.name)
+    .sort();
+  return entries.length ? join(base, entries[entries.length - 1]) : null;
+}
+
+/** Collect every capture record (one per route × viewport × theme) with a PNG. */
+async function loadCaptures(runDir) {
+  const capturesDir = join(runDir, "captures");
+  if (!existsSync(capturesDir)) return [];
+  const out = [];
+  for (const routeEntry of await readdir(capturesDir, {
+    withFileTypes: true,
+  })) {
+    if (!routeEntry.isDirectory()) continue;
+    const routeDir = join(capturesDir, routeEntry.name);
+    for (const f of await readdir(routeDir)) {
+      if (!f.endsWith(".json")) continue;
+      let rec;
+      try {
+        rec = JSON.parse(await readFile(join(routeDir, f), "utf8"));
+      } catch {
+        continue;
+      }
+      if (!rec.screenshotRelPath) continue;
+      const png = join(runDir, rec.screenshotRelPath);
+      if (!existsSync(png)) continue;
+      out.push({ ...rec, pngPath: png });
+    }
+  }
+  return out;
+}
+
+async function reviewOne(capture, labelByRoute) {
+  const key = `${capture.routeId}-${capture.viewport}-${capture.theme}`;
+  try {
+    const base64 = (await readFile(capture.pngPath)).toString("base64");
+    const prompt = buildReviewPrompt({
+      label: labelByRoute[capture.routeId] || capture.routeId,
+      path: capture.routePath,
+      viewport: capture.viewport,
+      theme: capture.theme,
+      issues: (capture.issues ?? []).map((i) =>
+        typeof i === "string" ? i : `${i.kind}: ${i.detail}`,
+      ),
+    });
+    const res = await fetch("https://api.anthropic.com/v1/messages", {
+      method: "POST",
+      headers: {
+        "content-type": "application/json",
+        "x-api-key": process.env.ANTHROPIC_API_KEY,
+        "anthropic-version": ANTHROPIC_VERSION,
+      },
+      body: JSON.stringify({
+        model: MODEL,
+        max_tokens: 1024,
+        messages: [
+          {
+            role: "user",
+            content: [imageBlock(base64), { type: "text", text: prompt }],
+          },
+        ],
+      }),
+    });
+    if (!res.ok)
+      throw new Error(
+        `HTTP ${res.status}: ${(await res.text()).slice(0, 200)}`,
+      );
+    const body = await res.json();
+    const text = (body.content ?? [])
+      .filter((b) => b.type === "text")
+      .map((b) => b.text)
+      .join("");
+    return { key, ...capture, ...parseVisionVerdict(text) };
+  } catch (err) {
+    return { key, ...capture, error: err?.message || String(err) };
+  }
+}
+
+async function mapPool(items, n, fn) {
+  const out = new Array(items.length);
+  let cursor = 0;
+  await Promise.all(
+    Array.from({ length: Math.min(n, items.length) }, async () => {
+      while (cursor < items.length) {
+        const i = cursor++;
+        out[i] = await fn(items[i]);
+      }
+    }),
+  );
+  return out;
+}
+
+async function main() {
+  const args = parseArgs(process.argv.slice(2));
+  if (!process.env.ANTHROPIC_API_KEY) {
+    console.log(
+      "[vision-review] ANTHROPIC_API_KEY not set — skipping (CI-safe no-op). Set it to run the content review.",
+    );
+    return;
+  }
+  const runDir = args.runDir
+    ? resolve(REPO_ROOT, args.runDir)
+    : await latestRunDir();
+  if (!runDir || !existsSync(runDir)) {
+    console.error(
+      "[vision-review] no ai-qa run dir found. Run `ai-qa-capture.spec.ts` first.",
+    );
+    process.exit(2);
+  }
+  const manifest = JSON.parse(
+    await readFile(join(runDir, "manifest.json"), "utf8"),
+  );
+  const labelByRoute = Object.fromEntries(
+    (manifest.routes ?? []).map((r) => [r.id, r.label]),
+  );
+  const captures = await loadCaptures(runDir);
+  if (!captures.length) {
+    console.error(
+      `[vision-review] no captures with screenshots under ${runDir}`,
+    );
+    process.exit(2);
+  }
+  console.log(
+    `[vision-review] reviewing ${captures.length} screenshots with ${MODEL} (concurrency ${args.concurrency})`,
+  );
+
+  const results = await mapPool(captures, args.concurrency, (c) =>
+    reviewOne(c, labelByRoute),
+  );
+  const totals = aggregateVerdicts(results);
+
+  const debtPath = join(
+    dirname(fileURLToPath(import.meta.url)),
+    "vision-review-debt.json",
+  );
+  const debt = existsSync(debtPath)
+    ? JSON.parse(await readFile(debtPath, "utf8"))
+    : {};
+
+  await writeFile(
+    join(runDir, "vision-review.json"),
+    JSON.stringify(
+      {
+        model: MODEL,
+        totals,
+        results: results.map(({ pngPath, buttons, ...r }) => r),
+      },
+      null,
+      2,
+    ),
+  );
+
+  if (args.updateDebt) {
+    const next = {};
+    for (const r of results)
+      if (r.error || r.verdict === "broken")
+        next[r.key] = (r.reasons ?? []).join("; ") || String(r.error);
+    await writeFile(
+      debtPath,
+      JSON.stringify(next, Object.keys(next).sort(), 2),
+    );
+    console.log(
+      `[vision-review] debt updated (${Object.keys(next).length} entries)`,
+    );
+    return;
+  }
+
+  const failures = gateFailures(results, { debt, strict: args.strict });
+  console.log(
+    `[vision-review] ${JSON.stringify(totals)} | strict=${args.strict} | undebted failures=${failures.length}`,
+  );
+  if (failures.length) {
+    for (const f of failures.slice(0, 40))
+      console.error(
+        `  [${f.verdict}] ${f.key}\n      ${f.reasons.join(" | ")}`,
+      );
+    console.error(`\nReport: ${join(runDir, "vision-review.json")}`);
+    process.exit(1);
+  }
+  console.log(`[vision-review] PASSED — ${join(runDir, "vision-review.json")}`);
+}
+
+main().catch((err) => {
+  console.error("[vision-review] fatal", err);
+  process.exit(1);
+});

scripts/ai-qa/review-walkthrough.mjs

@@ -0,0 +1,337 @@
+#!/usr/bin/env node
+
+/**
+ * Vision review for the full-walkthrough run (#10198 / #10204).
+ *
+ * The per-route AI-QA reviewer (`review-screenshots.mjs`) only ever sees an
+ * isolated route at a time. This adapter binds the SAME vision review library
+ * (`review-lib.mjs`) to the ordered walkthrough steps: it reads each
+ * `reports/walkthrough/<runId>/<viewport>/steps.json`, sends every captured
+ * `NN-<step>.png` + its per-step expectation/diagnostics to the vision model,
+ * records a structured good/needs-work/broken verdict, gates `broken` via a
+ * shrinking debt allowlist (the same ratchet as the story gate + aesthetic
+ * audit), and writes the committed verdict markdown next to JOURNEY.md.
+ *
+ * OPT-IN + CI-SAFE: requires ANTHROPIC_API_KEY. With no key it logs and exits 0
+ * (the keyless PR lane is unaffected). Model: AI_QA_VISION_MODEL (default Haiku).
+ *
+ * Usage:
+ *   node scripts/ai-qa/review-walkthrough.mjs [--run-dir reports/walkthrough/<id>]
+ *     [--concurrency 4] [--strict] [--update-debt]
+ *     [--verdict-md packages/app/test/ui-smoke/walkthrough/WALKTHROUGH_VERDICTS.md]
+ */
+
+import { existsSync } from "node:fs";
+import { mkdir, readdir, readFile, writeFile } from "node:fs/promises";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import {
+  aggregateVerdicts,
+  buildReviewPrompt,
+  gateFailures,
+  imageBlock,
+  parseVisionVerdict,
+} from "./review-lib.mjs";
+
+const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "../..");
+const MODEL = process.env.AI_QA_VISION_MODEL || "claude-haiku-4-5-20251001";
+const ANTHROPIC_VERSION = "2023-06-01";
+const DEFAULT_VERDICT_MD = join(
+  REPO_ROOT,
+  "packages/app/test/ui-smoke/walkthrough/WALKTHROUGH_VERDICTS.md",
+);
+
+function parseArgs(argv) {
+  const a = {
+    runDir: null,
+    concurrency: 4,
+    strict: false,
+    updateDebt: false,
+    verdictMd: DEFAULT_VERDICT_MD,
+  };
+  for (let i = 0; i < argv.length; i++) {
+    const arg = argv[i];
+    if (arg === "--run-dir") a.runDir = argv[++i];
+    else if (arg === "--concurrency") a.concurrency = Number(argv[++i]);
+    else if (arg === "--strict") a.strict = true;
+    else if (arg === "--update-debt") a.updateDebt = true;
+    else if (arg === "--verdict-md")
+      a.verdictMd = resolve(REPO_ROOT, argv[++i]);
+  }
+  return a;
+}
+
+async function latestRunDir() {
+  const base = join(REPO_ROOT, "reports", "walkthrough");
+  if (!existsSync(base)) return null;
+  const entries = (await readdir(base, { withFileTypes: true }))
+    .filter((e) => e.isDirectory())
+    .map((e) => e.name)
+    .sort();
+  return entries.length ? join(base, entries[entries.length - 1]) : null;
+}
+
+/** Collect every captured step (across viewports) that has a screenshot. */
+async function loadCaptures(runDir) {
+  const out = [];
+  for (const vp of ["desktop", "mobile"]) {
+    const stepsPath = join(runDir, vp, "steps.json");
+    if (!existsSync(stepsPath)) continue;
+    let parsed;
+    try {
+      parsed = JSON.parse(await readFile(stepsPath, "utf8"));
+    } catch {
+      continue;
+    }
+    for (const step of parsed.steps ?? []) {
+      if (step.skipped || !step.screenshotRelPath) continue;
+      const png = join(
+        runDir,
+        vp,
+        step.screenshotRelPath.replace(`${vp}/`, ""),
+      );
+      const pngPath = existsSync(png)
+        ? png
+        : join(runDir, step.screenshotRelPath);
+      if (!existsSync(pngPath)) continue;
+      out.push({
+        key: `${step.id}-${vp}`,
+        stepId: step.id,
+        stepN: step.n,
+        title: step.title,
+        expectation: step.expectation,
+        viewport: vp,
+        lane: step.lane,
+        routePath: step.url,
+        issues: [
+          ...(step.newConsoleErrors ?? []),
+          ...(step.newServerErrors ?? []),
+        ],
+        pngPath,
+      });
+    }
+  }
+  // Stable order: by viewport then step number.
+  out.sort((a, b) =>
+    a.viewport === b.viewport
+      ? a.stepN.localeCompare(b.stepN)
+      : a.viewport.localeCompare(b.viewport),
+  );
+  return out;
+}
+
+async function reviewOne(capture) {
+  try {
+    const base64 = (await readFile(capture.pngPath)).toString("base64");
+    const prompt = buildReviewPrompt({
+      label: `${capture.title} — ${capture.expectation}`,
+      path: capture.routePath,
+      viewport: capture.viewport,
+      theme: "light",
+      issues: capture.issues,
+    });
+    const res = await fetch("https://api.anthropic.com/v1/messages", {
+      method: "POST",
+      headers: {
+        "content-type": "application/json",
+        "x-api-key": process.env.ANTHROPIC_API_KEY,
+        "anthropic-version": ANTHROPIC_VERSION,
+      },
+      body: JSON.stringify({
+        model: MODEL,
+        max_tokens: 1024,
+        messages: [
+          {
+            role: "user",
+            content: [imageBlock(base64), { type: "text", text: prompt }],
+          },
+        ],
+      }),
+    });
+    if (!res.ok)
+      throw new Error(
+        `HTTP ${res.status}: ${(await res.text()).slice(0, 200)}`,
+      );
+    const body = await res.json();
+    const text = (body.content ?? [])
+      .filter((b) => b.type === "text")
+      .map((b) => b.text)
+      .join("");
+    const verdict = parseVisionVerdict(text);
+    return {
+      key: capture.key,
+      stepN: capture.stepN,
+      stepId: capture.stepId,
+      title: capture.title,
+      viewport: capture.viewport,
+      lane: capture.lane,
+      ...verdict,
+    };
+  } catch (err) {
+    return {
+      key: capture.key,
+      stepN: capture.stepN,
+      stepId: capture.stepId,
+      title: capture.title,
+      viewport: capture.viewport,
+      lane: capture.lane,
+      error: err?.message || String(err),
+    };
+  }
+}
+
+async function mapPool(items, n, fn) {
+  const out = new Array(items.length);
+  let cursor = 0;
+  await Promise.all(
+    Array.from({ length: Math.min(n, items.length) }, async () => {
+      while (cursor < items.length) {
+        const i = cursor++;
+        out[i] = await fn(items[i]);
+      }
+    }),
+  );
+  return out;
+}
+
+function buildVerdictMarkdown({ runId, model, lane, totals, results }) {
+  const lines = [];
+  lines.push("# Full Walkthrough — Vision Verdicts");
+  lines.push("");
+  lines.push(
+    "Generated by `scripts/ai-qa/review-walkthrough.mjs`. Each row is a per-step",
+    "screenshot from the continuous full-walkthrough run, scored by the vision",
+    "reviewer against the step's expectation row in `JOURNEY.md`. `broken` outside",
+    "the documented debt allowlist fails the gate.",
+  );
+  lines.push("");
+  lines.push(`- Run: \`${runId}\``);
+  lines.push(`- Lane: \`${lane}\``);
+  lines.push(`- Vision model: \`${model}\``);
+  lines.push(
+    `- Totals: ${totals.good} good · ${totals["needs-work"]} needs-work · ${totals.broken} broken · ${totals.error} error (of ${totals.total})`,
+  );
+  lines.push("");
+  lines.push("| Step | Viewport | Verdict | Notes |");
+  lines.push("| --- | --- | --- | --- |");
+  for (const r of results) {
+    const badge = r.error
+      ? "⚠️ error"
+      : r.verdict === "good"
+        ? "✅ good"
+        : r.verdict === "needs-work"
+          ? "🟡 needs-work"
+          : "❌ broken";
+    const notes = r.error
+      ? r.error
+      : [
+          ...(r.reasons ?? []),
+          ...(r.layoutIssues ?? []),
+          ...(r.brandViolations ?? []),
+        ]
+          .join("; ")
+          .slice(0, 200) || "—";
+    lines.push(
+      `| ${r.stepN} ${r.stepId} | ${r.viewport} | ${badge} | ${notes.replace(/\|/g, "\\|")} |`,
+    );
+  }
+  lines.push("");
+  return lines.join("\n");
+}
+
+async function main() {
+  const args = parseArgs(process.argv.slice(2));
+  if (!process.env.ANTHROPIC_API_KEY) {
+    console.log(
+      "[walkthrough-vision] ANTHROPIC_API_KEY not set — skipping (CI-safe no-op). Set it to run the content review.",
+    );
+    return;
+  }
+  const runDir = args.runDir
+    ? resolve(REPO_ROOT, args.runDir)
+    : await latestRunDir();
+  if (!runDir || !existsSync(runDir)) {
+    console.error(
+      "[walkthrough-vision] no walkthrough run dir found. Run the walkthrough spec first.",
+    );
+    process.exit(2);
+  }
+  const manifest = existsSync(join(runDir, "manifest.json"))
+    ? JSON.parse(await readFile(join(runDir, "manifest.json"), "utf8"))
+    : { runId: runDir.split("/").pop(), lane: "unknown" };
+
+  const captures = await loadCaptures(runDir);
+  if (!captures.length) {
+    console.error(`[walkthrough-vision] no step screenshots under ${runDir}`);
+    process.exit(2);
+  }
+  console.log(
+    `[walkthrough-vision] reviewing ${captures.length} step screenshots with ${MODEL} (concurrency ${args.concurrency})`,
+  );
+
+  const results = await mapPool(captures, args.concurrency, reviewOne);
+  const totals = aggregateVerdicts(results);
+
+  const debtPath = join(
+    dirname(fileURLToPath(import.meta.url)),
+    "walkthrough-vision-debt.json",
+  );
+  const debt = existsSync(debtPath)
+    ? JSON.parse(await readFile(debtPath, "utf8"))
+    : {};
+
+  await writeFile(
+    join(runDir, "vision-review.json"),
+    JSON.stringify(
+      { model: MODEL, runId: manifest.runId, totals, results },
+      null,
+      2,
+    ),
+  );
+
+  if (args.updateDebt) {
+    const next = {};
+    for (const r of results)
+      if (r.error || r.verdict === "broken")
+        next[r.key] = (r.reasons ?? []).join("; ") || String(r.error);
+    await writeFile(
+      debtPath,
+      JSON.stringify(next, Object.keys(next).sort(), 2),
+    );
+    console.log(
+      `[walkthrough-vision] debt updated (${Object.keys(next).length} entries)`,
+    );
+  }
+
+  const md = buildVerdictMarkdown({
+    runId: manifest.runId,
+    model: MODEL,
+    lane: manifest.lane,
+    totals,
+    results,
+  });
+  await mkdir(dirname(args.verdictMd), { recursive: true });
+  await writeFile(args.verdictMd, md);
+  console.log(`[walkthrough-vision] verdict markdown → ${args.verdictMd}`);
+
+  const failures = gateFailures(results, { debt, strict: args.strict });
+  console.log(
+    `[walkthrough-vision] ${JSON.stringify(totals)} | strict=${args.strict} | undebted failures=${failures.length}`,
+  );
+  if (failures.length) {
+    for (const f of failures.slice(0, 40))
+      console.error(
+        `  [${f.verdict}] ${f.key}\n      ${f.reasons.join(" | ")}`,
+      );
+    console.error(`\nReport: ${join(runDir, "vision-review.json")}`);
+    process.exit(1);
+  }
+  console.log(
+    `[walkthrough-vision] PASSED — ${join(runDir, "vision-review.json")}`,
+  );
+}
+
+main().catch((err) => {
+  console.error("[walkthrough-vision] fatal", err);
+  process.exit(1);
+});

scripts/ai-qa/route-catalog.ts

@@ -0,0 +1,121 @@
+/**
+ * Route catalog and shared types for the AI-QA screenshot tooling.
+ *
+ * Enumerates every dashboard route the QA scripts visit, the per-route
+ * ready-checks that gate a capture, the viewport sizes and themes to render,
+ * and the settings-section map. Built on top of the app-core dev route catalog
+ * and consumed by the review-screenshots / review-walkthrough scripts.
+ */
+import { buildRouteCatalog } from "../../packages/app-core/src/api/dev-route-catalog";
+
+export type ReadyCheck =
+  | { selector: string; text?: never }
+  | { selector?: never; text: string };
+
+export type ViewportName = "desktop" | "tablet" | "mobile";
+export type Theme = "light" | "dark";
+
+export type AiQaRoute = {
+  id: string;
+  path: string;
+  label: string;
+  readyChecks: readonly ReadyCheck[];
+  readyMode?: "all" | "any";
+  timeoutMs?: number;
+  viewports?: readonly ViewportName[];
+};
+
+export type SettingsSection = {
+  id: string;
+  label: string;
+  match: RegExp;
+};
+
+export const VIEWPORT_SIZES: Record<
+  ViewportName,
+  { width: number; height: number }
+> = {
+  desktop: { width: 1440, height: 1000 },
+  tablet: { width: 820, height: 1180 },
+  mobile: { width: 390, height: 844 },
+};
+
+const READY_CHECKS_BY_PATH: Record<string, readonly ReadyCheck[]> = {
+  "/onboarding": [{ selector: '[data-testid="first-run-runtime-cloud"]' }],
+  "/chat": [
+    { selector: '[data-testid="conversations-sidebar"]' },
+    { selector: '[data-testid="chat-composer-textarea"]' },
+  ],
+  // Settings/lazy-view routes: wait for the actual rendered view marker, not the
+  // always-present #root, so the screenshot is captured after the lazy chunk
+  // mounts and paints (a bare #root check races the chunk and yields a blank,
+  // one-solid-color capture that fails the screenshot-quality gate).
+  "/connectors": [{ selector: '[data-testid="settings-shell"]' }],
+  "/tutorial": [{ selector: '[data-testid="tutorial-launcher"]' }],
+  "/help": [{ selector: '[data-testid="help-view"]' }],
+  "/apps/transcripts": [{ selector: '[data-testid="transcripts-view"]' }],
+  "/apps": [{ selector: '[data-testid="apps-shell"]' }],
+  "/views": [{ text: "Views" }],
+  "/apps/lifeops": [{ selector: '[data-testid="lifeops-shell"]' }],
+  "/apps/plugins": [{ text: "Browser Workspace" }, { text: "AI Providers" }],
+  "/apps/skills": [{ selector: '[data-testid="skills-shell"]' }],
+  "/apps/fine-tuning": [{ selector: '[data-testid="fine-tuning-view"]' }],
+  "/apps/trajectories": [{ selector: '[data-testid="trajectories-view"]' }],
+  "/apps/relationships": [{ selector: '[data-testid="relationships-view"]' }],
+  "/apps/memories": [{ selector: '[data-testid="memory-viewer-view"]' }],
+  "/apps/inventory": [{ selector: '[data-testid="wallet-shell"]' }],
+  "/apps/runtime": [{ selector: '[data-testid="runtime-view"]' }],
+  "/apps/database": [{ selector: '[data-testid="database-view"]' }],
+  "/apps/logs": [{ selector: '[data-testid="logs-view"]' }],
+  "/apps/tasks": [{ selector: '[data-testid="tasks-view"]' }],
+  "/character": [{ selector: '[data-testid="character-editor-view"]' }],
+  "/character/select": [{ selector: '[data-testid="character-editor-view"]' }],
+  "/character/documents": [{ selector: '[data-testid="documents-view"]' }],
+  "/wallet": [{ selector: '[data-testid="wallet-shell"]' }],
+  "/browser": [{ selector: '[data-testid="browser-workspace-address-input"]' }],
+  "/stream": [{ text: "Stream Ready" }],
+  "/automations": [{ selector: '[data-testid="automations-shell"]' }],
+  "/settings": [{ selector: '[data-testid="settings-shell"]' }],
+  "/settings/voice": [{ selector: '[data-testid="settings-shell"]' }],
+  "/companion": [{ text: "Companion" }],
+  "/rolodex": [{ text: "Views" }],
+  "/desktop": [{ text: "Desktop workspace tools are only available" }],
+};
+
+const catalog = buildRouteCatalog(new Date("2026-01-01T00:00:00.000Z"));
+const extraAppWindowRoutes = [
+  {
+    tabId: "app-window-inventory",
+    path: "/apps/inventory",
+    label: "Inventory App Window",
+    platformGate: null,
+  },
+] as const;
+
+export const AI_QA_ROUTES: readonly AiQaRoute[] = [
+  ...catalog.routes,
+  ...extraAppWindowRoutes,
+]
+  .filter((route) => route.platformGate !== "android")
+  .map((route) => ({
+    id: route.tabId,
+    path: route.path,
+    label: route.label,
+    readyChecks: READY_CHECKS_BY_PATH[route.path] ?? [{ selector: "#root" }],
+    readyMode: route.path === "/chat" ? "all" : "any",
+    timeoutMs: 90_000,
+    viewports:
+      route.platformGate === "desktop"
+        ? (["desktop"] as const)
+        : (["desktop", "mobile"] as const),
+  }));
+
+export const SETTINGS_SECTIONS: readonly SettingsSection[] =
+  catalog.settingsSections.map((section) => ({
+    id: section.id,
+    label: section.label,
+    match: new RegExp(
+      `^${section.label.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}$`,
+      "i",
+    ),
+  }));

scripts/assert-agents-claude-identical.mjs

@@ -0,0 +1,102 @@
+#!/usr/bin/env node
+/**
+ * Verifies that every tracked agent guide is paired and byte-identical.
+ *
+ * The repository convention is to author `CLAUDE.md` and copy it to
+ * `AGENTS.md` in the same directory. This guard walks tracked files only, so
+ * generated build output and local worktree clutter cannot affect the result.
+ * It fails when a directory has only one of the two files or when a pair differs
+ * byte-for-byte.
+ */
+
+import { execFileSync } from "node:child_process";
+import { existsSync, readFileSync } from "node:fs";
+import { dirname } from "node:path";
+import { pathToFileURL } from "node:url";
+
+function git(args) {
+  return execFileSync("git", args, { encoding: "utf8", maxBuffer: 1 << 30 });
+}
+
+function guideFiles() {
+  return git([
+    "ls-files",
+    "-z",
+    "AGENTS.md",
+    "CLAUDE.md",
+    "**/AGENTS.md",
+    "**/CLAUDE.md",
+  ])
+    .split("\0")
+    .filter(Boolean)
+    .filter((file) => existsSync(file))
+    .filter(shouldCheckGuideFile);
+}
+
+export function shouldCheckGuideFile(file) {
+  const parts = file.replace(/\\/g, "/").split("/");
+  return !(
+    parts.includes("fixtures") ||
+    parts.includes("__tests__") ||
+    parts.includes(".archived")
+  );
+}
+
+function directoryFor(file) {
+  const dir = dirname(file);
+  return dir === "." ? "." : dir;
+}
+
+function main() {
+  const byDirectory = new Map();
+
+  for (const file of guideFiles()) {
+    const directory = directoryFor(file);
+    const entry = byDirectory.get(directory) ?? {};
+
+    if (file.endsWith("AGENTS.md")) entry.agents = file;
+    if (file.endsWith("CLAUDE.md")) entry.claude = file;
+
+    byDirectory.set(directory, entry);
+  }
+
+  const failures = [];
+  let pairs = 0;
+
+  for (const [directory, entry] of [...byDirectory.entries()].sort(([a], [b]) =>
+    a.localeCompare(b),
+  )) {
+    if (!entry.agents || !entry.claude) {
+      failures.push(
+        `${directory}: expected both CLAUDE.md and AGENTS.md, found ${
+          entry.claude ? "CLAUDE.md only" : "AGENTS.md only"
+        }`,
+      );
+      continue;
+    }
+
+    pairs += 1;
+    const agents = readFileSync(entry.agents);
+    const claude = readFileSync(entry.claude);
+
+    if (!agents.equals(claude)) {
+      failures.push(
+        `${directory}: CLAUDE.md and AGENTS.md differ; author CLAUDE.md, then copy it to AGENTS.md`,
+      );
+    }
+  }
+
+  if (failures.length > 0) {
+    console.error("[assert-agents-claude-identical] FAIL");
+    for (const failure of failures) console.error(`- ${failure}`);
+    process.exit(1);
+  }
+
+  console.log(
+    `[assert-agents-claude-identical] PASS: ${pairs} tracked CLAUDE.md/AGENTS.md pair(s) are byte-identical.`,
+  );
+}
+
+if (import.meta.url === pathToFileURL(process.argv[1]).href) {
+  main();
+}

scripts/assert-agents-claude-identical.test.mjs

@@ -0,0 +1,33 @@
+import assert from "node:assert/strict";
+import { describe, it } from "node:test";
+import { shouldCheckGuideFile } from "./assert-agents-claude-identical.mjs";
+
+describe("assert-agents-claude-identical guide filtering", () => {
+  it("checks authored guide pairs", () => {
+    assert.equal(shouldCheckGuideFile("AGENTS.md"), true);
+    assert.equal(shouldCheckGuideFile("CLAUDE.md"), true);
+    assert.equal(shouldCheckGuideFile("packages/app/AGENTS.md"), true);
+    assert.equal(shouldCheckGuideFile("plugins/plugin-openai/CLAUDE.md"), true);
+  });
+
+  it("excludes fixture, test, and archived sample trees", () => {
+    assert.equal(
+      shouldCheckGuideFile(
+        "packages/import-conversations/src/parsers/fixtures/openclaw-home/AGENTS.md",
+      ),
+      false,
+    );
+    assert.equal(
+      shouldCheckGuideFile(
+        "packages/elizaos/src/migrate/__tests__/fixtures/oc-home/CLAUDE.md",
+      ),
+      false,
+    );
+    assert.equal(
+      shouldCheckGuideFile(
+        "packages/benchmarks/openclaw-benchmark/.archived/opencode_setup/AGENTS.md",
+      ),
+      false,
+    );
+  });
+});

scripts/assert-comment-only-diff.mjs

@@ -0,0 +1,235 @@
+#!/usr/bin/env node
+/**
+ * Machine proof that a comment-cleanup change touches comments and whitespace
+ * only, never code.
+ *
+ * For every file changed against a base ref (default `origin/develop`), the
+ * base blob and the working-tree copy are parsed with the TypeScript parser and
+ * their ASTs are walked to the leaf-token level; comments (including JSDoc
+ * doc-comments, which the parser keeps as nodes rather than trivia) are
+ * excluded, and the two code-token streams are asserted identical. Any token, or
+ * any added / deleted / renamed / non-source changed file, fails the check and
+ * is reported with the first offending token. A comment edit cannot change the
+ * parsed code tokens, so an identical token stream is a sound proof that only
+ * comments moved.
+ *
+ * This script is the single non-comment change in the repo-wide comment cleanup
+ * (parent elizaOS/eliza#12181, Work Item 2): it lets each batch PR prove "zero
+ * functional diff" mechanically instead of by reviewer trust. Wired as the root
+ * `check:comment-only` npm script; run per batch PR alongside `bun run verify`.
+ *
+ * Usage:
+ *   node scripts/assert-comment-only-diff.mjs [base-ref]   # default origin/develop
+ *   node scripts/assert-comment-only-diff.mjs --self-test  # planted-diff self-check
+ */
+
+import { execFileSync } from "node:child_process";
+import { readFileSync } from "node:fs";
+import { extname } from "node:path";
+import ts from "typescript";
+
+const SOURCE_EXT = new Set([
+  ".ts",
+  ".tsx",
+  ".mts",
+  ".cts",
+  ".js",
+  ".jsx",
+  ".mjs",
+  ".cjs",
+]);
+
+function git(args) {
+  return execFileSync("git", args, { encoding: "utf8", maxBuffer: 1 << 30 });
+}
+
+function isSource(file) {
+  return SOURCE_EXT.has(extname(file));
+}
+
+function scriptKind(fileName) {
+  if (fileName.endsWith(".tsx")) return ts.ScriptKind.TSX;
+  if (fileName.endsWith(".jsx")) return ts.ScriptKind.JSX;
+  if (
+    fileName.endsWith(".js") ||
+    fileName.endsWith(".mjs") ||
+    fileName.endsWith(".cjs")
+  )
+    return ts.ScriptKind.JS;
+  return ts.ScriptKind.TS;
+}
+
+/**
+ * Non-trivia token stream: `<kind>:<raw source text>` per leaf token, in order.
+ *
+ * The file is fully parsed (`createSourceFile`) and its AST is walked to its
+ * leaf tokens — the raw scanner is deliberately NOT used, because a scanner has
+ * no parse context and cannot tell a regex literal (`/…/`) from division or a
+ * template literal from a backtick inside a regex; a regex such as
+ * `` /=`([^`]+)`/ `` would make the scanner mis-open a template literal that
+ * swallows following comments, producing false divergences on comment-only
+ * edits. The parser resolves all of that. Comments are trivia and are not
+ * nodes, so they are excluded; string/template/regex literals and JSX text are
+ * single leaf tokens whose text is compared, so any code or literal edit
+ * surfaces as a divergence.
+ */
+function tokenStream(text, fileName) {
+  const sf = ts.createSourceFile(
+    fileName,
+    text,
+    ts.ScriptTarget.Latest,
+    /* setParentNodes */ false,
+    scriptKind(fileName),
+  );
+  const tokens = [];
+  const visit = (node) => {
+    // `/** … */` doc-comments are parsed into JSDoc nodes (not trivia); they are
+    // comments and are editable, so skip the whole JSDoc subtree.
+    if (
+      node.kind >= ts.SyntaxKind.FirstJSDocNode &&
+      node.kind <= ts.SyntaxKind.LastJSDocNode
+    ) {
+      return;
+    }
+    const children = node.getChildren(sf);
+    if (children.length === 0) {
+      if (node.kind === ts.SyntaxKind.EndOfFileToken) return;
+      tokens.push(`${node.kind}:${node.getText(sf)}`);
+      return;
+    }
+    for (const child of children) visit(child);
+  };
+  visit(sf);
+  return tokens;
+}
+
+/** First divergent index, or -1 if the two streams are identical. */
+function firstDivergence(a, b) {
+  const n = Math.min(a.length, b.length);
+  for (let i = 0; i < n; i++) {
+    if (a[i] !== b[i]) return i;
+  }
+  return a.length === b.length ? -1 : n;
+}
+
+function selfTest() {
+  const before = `const answer = 40 + 2; // old note\nexport default answer;\n`;
+  const commentsOnly = `/** Header prose. */\nconst answer = 40 + 2; // new note, rewritten\nexport default answer;\n`;
+  const oneToken = `const answer = 40 + 3; // old note\nexport default answer;\n`;
+
+  const failures = [];
+  if (
+    firstDivergence(
+      tokenStream(before, "x.ts"),
+      tokenStream(commentsOnly, "x.ts"),
+    ) !== -1
+  ) {
+    failures.push(
+      "comments-only plant was flagged as a code change (false positive)",
+    );
+  }
+  if (
+    firstDivergence(
+      tokenStream(before, "x.ts"),
+      tokenStream(oneToken, "x.ts"),
+    ) === -1
+  ) {
+    failures.push(
+      "one-token code plant (40+2 -> 40+3) slipped through (false negative)",
+    );
+  }
+  if (failures.length) {
+    for (const f of failures)
+      console.error(`[assert-comment-only-diff] self-test FAIL: ${f}`);
+    process.exit(1);
+  }
+  console.log(
+    "[assert-comment-only-diff] self-test PASS: comments-only accepted, one-token change rejected.",
+  );
+  process.exit(0);
+}
+
+function main() {
+  const argv = process.argv.slice(2);
+  if (argv.includes("--self-test")) return selfTest();
+
+  const base = argv[0] || "origin/develop";
+  let mergeBase;
+  try {
+    mergeBase = git(["merge-base", base, "HEAD"]).trim();
+  } catch {
+    console.error(
+      `[assert-comment-only-diff] cannot resolve merge-base of ${base} and HEAD.`,
+    );
+    process.exit(2);
+  }
+
+  // Diff merge-base against the working tree: catches committed and uncommitted
+  // changes on this branch without flagging what develop moved on its own.
+  const raw = git(["diff", "--name-status", "-z", mergeBase]);
+  const parts = raw.split("\0").filter(Boolean);
+
+  const violations = [];
+  let checked = 0;
+
+  for (let i = 0; i < parts.length; ) {
+    const status = parts[i++];
+    const code = status[0];
+    const file = parts[i++];
+    // Renames/copies carry a second path field.
+    const dest = code === "R" || code === "C" ? parts[i++] : file;
+
+    if (code !== "M") {
+      violations.push(
+        `${dest}: ${code} — comment cleanup must modify existing files only, not add/delete/rename`,
+      );
+      continue;
+    }
+    if (!isSource(dest)) {
+      violations.push(
+        `${dest}: changed non-source file — comment cleanup touches source files only`,
+      );
+      continue;
+    }
+
+    let baseText;
+    try {
+      baseText = git(["show", `${mergeBase}:${file}`]);
+    } catch {
+      violations.push(`${dest}: cannot read base blob at ${mergeBase}`);
+      continue;
+    }
+    const headText = readFileSync(dest, "utf8");
+
+    const baseTokens = tokenStream(baseText, dest);
+    const headTokens = tokenStream(headText, dest);
+    const idx = firstDivergence(baseTokens, headTokens);
+    checked++;
+    if (idx !== -1) {
+      const b =
+        baseTokens[idx]?.split(":").slice(1).join(":") ?? "<end of file>";
+      const h =
+        headTokens[idx]?.split(":").slice(1).join(":") ?? "<end of file>";
+      violations.push(
+        `${dest}: code token #${idx} diverges — base \`${b}\` vs head \`${h}\``,
+      );
+    }
+  }
+
+  if (violations.length) {
+    console.error(
+      `[assert-comment-only-diff] FAIL — ${violations.length} file(s) changed code, not just comments:\n`,
+    );
+    for (const v of violations) console.error(`  ✗ ${v}`);
+    console.error(
+      `\nComment cleanup requires zero functional diff. Fix the files above.`,
+    );
+    process.exit(1);
+  }
+
+  console.log(
+    `[assert-comment-only-diff] OK — ${checked} source file(s) changed; every code token identical to ${base}. Comments only.`,
+  );
+}
+
+main();

scripts/benchmark/drift-harness.ts

@@ -0,0 +1,942 @@
+#!/usr/bin/env bun
+/**
+ * Conversation-compaction drift harness.
+ *
+ * Drives a synthetic multi-turn conversation with planted facts, forces a
+ * selected compaction strategy on a fixed cadence, and emits reproducible
+ * JSONL events consumed by packages/benchmarks/context-bench.
+ */
+
+import {
+  compactors,
+  findSafeCompactionBoundary,
+} from "../../packages/agent/src/runtime/conversation-compactor.ts";
+import {
+  approxCountTokens,
+  type CompactionArtifact,
+  type CompactorMessage,
+  type CompactorModelCall,
+  type CompactorTranscript,
+  countTranscriptTokens,
+} from "../../packages/agent/src/runtime/conversation-compactor.types.ts";
+
+const KNOWN_STRATEGIES = [
+  "none",
+  "prompt-stripping",
+  "naive-summary",
+  "structured-state",
+  "hierarchical-summary",
+  "hybrid-ledger",
+] as const;
+
+type StrategyName = (typeof KNOWN_STRATEGIES)[number];
+
+type ReasoningEffort = "low" | "medium" | "high";
+
+type Args = {
+  strategy: StrategyName;
+  turns: number;
+  compactEvery: number;
+  plantFacts: number;
+  seed: number;
+  output?: string;
+  dryRun: boolean;
+  agentModel: string;
+  judgeModel: string;
+  compactorModel: string;
+  agentReasoningEffort: ReasoningEffort;
+  judgeReasoningEffort: ReasoningEffort;
+  compactorReasoningEffort: ReasoningEffort;
+  probeMaxTokens: number;
+  realisticSystemPrompt: boolean;
+  withToolCalls: boolean;
+};
+
+type Fact = {
+  id: string;
+  kind: string;
+  expected: string;
+  question: string;
+  plantedTurn: number;
+};
+
+type ProbeEvent = {
+  event: "probe";
+  atTurn: number;
+  factId: string;
+  plantedTurn: number;
+  kind: string;
+  expected: string;
+  actual: string;
+  correct: boolean;
+  judgeReasoning: string;
+  phase: "post-compact" | "final";
+};
+
+const FACT_KINDS = [
+  "aws_account",
+  "person_name",
+  "address",
+  "code",
+  "book_title",
+  "project_codename",
+  "isbn",
+  "date_iso",
+  "birthday",
+  "flight_number",
+  "uuid",
+  "zipcode",
+] as const;
+
+function usage(): string {
+  return [
+    "usage: bun run scripts/benchmark/drift-harness.ts --strategy <name> [options]",
+    "",
+    `strategies: ${KNOWN_STRATEGIES.join(", ")}`,
+    "",
+    "options:",
+    "  --turns <n>                         default 50",
+    "  --compact-every <n>                 default 10",
+    "  --plant-facts <n>                   default 5",
+    "  --seed <n>                          default 1337",
+    "  --output <path>                     write JSONL to path",
+    "  --dry-run                          deterministic local model; no API calls",
+    "  --agent-model <id>                  default env AGENT_MODEL or gpt-oss-120b",
+    "  --judge-model <id>                  default env JUDGE_MODEL or agent model",
+    "  --compactor-model <id>              default env COMPACTOR_MODEL or agent model",
+    "  --agent-reasoning-effort <level>     low | medium | high, default medium",
+    "  --judge-reasoning-effort <level>     low | medium | high, default medium",
+    "  --compactor-reasoning-effort <level> low | medium | high, default low",
+    "  --probe-max-tokens <n>              default 600",
+    "  --realistic-system-prompt           use a larger Eliza-style system prompt",
+    "  --with-tool-calls                   add synthetic tool-call/result probes",
+  ].join("\n");
+}
+
+function parseReasoningEffort(value: string): ReasoningEffort {
+  if (value === "low" || value === "medium" || value === "high") return value;
+  throw new Error(`invalid reasoning effort ${value}`);
+}
+
+function readFlag(argv: string[], index: number): string {
+  const value = argv[index + 1];
+  if (!value || value.startsWith("--")) {
+    throw new Error(`missing value for ${argv[index]}`);
+  }
+  return value;
+}
+
+function parseArgs(argv: string[]): Args {
+  const agentModel = process.env.AGENT_MODEL ?? "gpt-oss-120b";
+  const args: Args = {
+    strategy: "none",
+    turns: 50,
+    compactEvery: 10,
+    plantFacts: 5,
+    seed: 1337,
+    dryRun: false,
+    agentModel,
+    judgeModel: process.env.JUDGE_MODEL ?? agentModel,
+    compactorModel: process.env.COMPACTOR_MODEL ?? agentModel,
+    agentReasoningEffort: "medium",
+    judgeReasoningEffort: "medium",
+    compactorReasoningEffort: "low",
+    probeMaxTokens: 600,
+    realisticSystemPrompt: false,
+    withToolCalls: false,
+  };
+
+  for (let i = 0; i < argv.length; i++) {
+    const flag = argv[i];
+    switch (flag) {
+      case "--help":
+      case "-h":
+        process.stdout.write(`${usage()}\n`);
+        process.exit(0);
+        break;
+      case "--strategy": {
+        const value = readFlag(argv, i);
+        if (!KNOWN_STRATEGIES.includes(value as StrategyName)) {
+          throw new Error(
+            `unknown strategy ${value}; expected ${KNOWN_STRATEGIES.join(", ")}`,
+          );
+        }
+        args.strategy = value as StrategyName;
+        i++;
+        break;
+      }
+      case "--turns":
+        args.turns = Number(readFlag(argv, i));
+        i++;
+        break;
+      case "--compact-every":
+        args.compactEvery = Number(readFlag(argv, i));
+        i++;
+        break;
+      case "--plant-facts":
+        args.plantFacts = Number(readFlag(argv, i));
+        i++;
+        break;
+      case "--seed":
+        args.seed = Number(readFlag(argv, i));
+        i++;
+        break;
+      case "--output":
+        args.output = readFlag(argv, i);
+        i++;
+        break;
+      case "--dry-run":
+        args.dryRun = true;
+        break;
+      case "--agent-model":
+        args.agentModel = readFlag(argv, i);
+        i++;
+        break;
+      case "--judge-model":
+        args.judgeModel = readFlag(argv, i);
+        i++;
+        break;
+      case "--compactor-model":
+        args.compactorModel = readFlag(argv, i);
+        i++;
+        break;
+      case "--agent-reasoning-effort":
+        args.agentReasoningEffort = parseReasoningEffort(readFlag(argv, i));
+        i++;
+        break;
+      case "--judge-reasoning-effort":
+        args.judgeReasoningEffort = parseReasoningEffort(readFlag(argv, i));
+        i++;
+        break;
+      case "--compactor-reasoning-effort":
+        args.compactorReasoningEffort = parseReasoningEffort(readFlag(argv, i));
+        i++;
+        break;
+      case "--probe-max-tokens":
+        args.probeMaxTokens = Number(readFlag(argv, i));
+        i++;
+        break;
+      case "--realistic-system-prompt":
+        args.realisticSystemPrompt = true;
+        break;
+      case "--with-tool-calls":
+        args.withToolCalls = true;
+        break;
+      default:
+        throw new Error(`unknown argument ${flag}`);
+    }
+  }
+
+  if (!Number.isInteger(args.turns) || args.turns < 1) {
+    throw new Error("--turns must be a positive integer");
+  }
+  if (!Number.isInteger(args.compactEvery) || args.compactEvery < 1) {
+    throw new Error("--compact-every must be a positive integer");
+  }
+  if (!Number.isInteger(args.plantFacts) || args.plantFacts < 0) {
+    throw new Error("--plant-facts must be a non-negative integer");
+  }
+  if (!Number.isInteger(args.seed)) {
+    throw new Error("--seed must be an integer");
+  }
+  if (!Number.isInteger(args.probeMaxTokens) || args.probeMaxTokens < 1) {
+    throw new Error("--probe-max-tokens must be a positive integer");
+  }
+
+  return args;
+}
+
+function seededInt(seed: number, index: number, modulo: number): number {
+  let x = (seed ^ Math.imul(index + 1, 0x9e3779b1)) >>> 0;
+  x ^= x << 13;
+  x ^= x >>> 17;
+  x ^= x << 5;
+  return (x >>> 0) % modulo;
+}
+
+function padded(seed: number, index: number, digits: number): string {
+  const max = 10 ** Math.min(digits, 12);
+  const n = seededInt(seed, index, max);
+  return String(n).padStart(digits, "0").slice(0, digits);
+}
+
+function uuidFor(seed: number, index: number): string {
+  const parts = [8, 4, 4, 4, 12].map((digits, part) =>
+    padded(seed + part * 101, index, digits).replace(/[0-9]/g, (d) =>
+      Number(d).toString(16),
+    ),
+  );
+  return parts.join("-");
+}
+
+function valueForKind(kind: string, seed: number, index: number): string {
+  const names = ["Mira Valen", "Talia Rune", "Niko Ardent", "Sana Holt"];
+  const books = [
+    "The Glass Atlas",
+    "Lanterns Under Ice",
+    "The Quiet Relay",
+    "Signal Garden",
+  ];
+  const codenames = [
+    "EMBER-VAULT",
+    "POLARIS-NINE",
+    "CITRINE-LAB",
+    "ORBIT-MINT",
+  ];
+  switch (kind) {
+    case "aws_account":
+      return padded(seed, index, 12);
+    case "person_name":
+      return names[seededInt(seed, index, names.length)];
+    case "address":
+      return `${100 + seededInt(seed, index, 899)} Meridian Ave, Unit ${1 + seededInt(seed, index + 1, 40)}`;
+    case "code":
+      return `CTX-${padded(seed, index, 6)}`;
+    case "book_title":
+      return books[seededInt(seed, index, books.length)];
+    case "project_codename":
+      return codenames[seededInt(seed, index, codenames.length)];
+    case "isbn":
+      return `978-1-${padded(seed, index, 3)}-${padded(seed, index + 1, 5)}-${seededInt(seed, index + 2, 10)}`;
+    case "date_iso":
+      return `2026-${String(1 + seededInt(seed, index, 12)).padStart(2, "0")}-${String(1 + seededInt(seed, index + 1, 28)).padStart(2, "0")}`;
+    case "birthday":
+      return `${String(1 + seededInt(seed, index, 12)).padStart(2, "0")}/${String(1 + seededInt(seed, index + 1, 28)).padStart(2, "0")}/199${seededInt(seed, index + 2, 10)}`;
+    case "flight_number":
+      return `EL${padded(seed, index, 4)}`;
+    case "uuid":
+      return uuidFor(seed, index);
+    case "zipcode":
+      return padded(seed, index, 5);
+    default:
+      return `VALUE-${padded(seed, index, 6)}`;
+  }
+}
+
+function questionFor(kind: string, id: string): string {
+  switch (kind) {
+    case "aws_account":
+      return `What AWS account number was planted for ${id}?`;
+    case "person_name":
+      return `What person name was planted for ${id}?`;
+    case "address":
+      return `What address was planted for ${id}?`;
+    case "code":
+      return `What code was planted for ${id}?`;
+    case "book_title":
+      return `What book title was planted for ${id}?`;
+    case "project_codename":
+      return `What project codename was planted for ${id}?`;
+    case "isbn":
+      return `What ISBN was planted for ${id}?`;
+    case "date_iso":
+      return `What ISO date was planted for ${id}?`;
+    case "birthday":
+      return `What birthday was planted for ${id}?`;
+    case "flight_number":
+      return `What flight number was planted for ${id}?`;
+    case "uuid":
+      return `What UUID was planted for ${id}?`;
+    case "zipcode":
+      return `What ZIP code was planted for ${id}?`;
+    default:
+      return `What value was planted for ${id}?`;
+  }
+}
+
+function buildFacts(args: Args): Fact[] {
+  const facts: Fact[] = [];
+  const usedTurns = new Set<number>();
+  for (let i = 0; i < args.plantFacts; i++) {
+    const id = `fact_${i + 1}`;
+    const kind = FACT_KINDS[(i + args.seed) % FACT_KINDS.length];
+    let plantedTurn = Math.max(
+      1,
+      Math.min(
+        args.turns,
+        Math.floor(((i + 1) * args.turns) / (args.plantFacts + 1)),
+      ),
+    );
+    while (usedTurns.has(plantedTurn) && plantedTurn < args.turns)
+      plantedTurn++;
+    usedTurns.add(plantedTurn);
+    const expected = valueForKind(kind, args.seed, i);
+    facts.push({
+      id,
+      kind,
+      expected,
+      question: questionFor(kind, id),
+      plantedTurn,
+    });
+  }
+  return facts;
+}
+
+function defaultSystemPrompt(args: Args): string {
+  if (!args.realisticSystemPrompt) {
+    return "You are an Eliza agent. Track user-provided facts exactly and answer later recall probes with the exact value.";
+  }
+  return [
+    "You are an Eliza agent running inside elizaOS.",
+    "Maintain continuity across long conversations, respect tool results, and keep exact identifiers intact.",
+    "Available actions include REPLY, TASKS, RUNTIME, SHELL, and NONE.",
+    "Providers include recent messages, user profile, workspace state, runtime settings, and active integrations.",
+    "When a user gives a durable fact, treat the exact value as load-bearing context for later turns.",
+    "Never invent identifiers. If a value appears in prior conversation history, reproduce it exactly.",
+    "Plugin descriptions: shell command execution, dashboard runtime management, cloud usage reporting, and local workspace inspection.",
+    "This synthetic prompt intentionally resembles a larger production prompt so compaction has non-conversation context to preserve.",
+  ].join("\n\n");
+}
+
+function renderTranscript(messages: CompactorMessage[]): string {
+  return messages
+    .map((message) => {
+      const extras: string[] = [];
+      if (message.toolCalls) {
+        extras.push(` toolCalls=${JSON.stringify(message.toolCalls)}`);
+      }
+      if (message.toolCallId) extras.push(` toolCallId=${message.toolCallId}`);
+      if (message.toolName) extras.push(` toolName=${message.toolName}`);
+      return `[${message.role}]${extras.join("")} ${message.content}`;
+    })
+    .join("\n");
+}
+
+function extractFacts(text: string): string[] {
+  const out: string[] = [];
+  for (const match of text.matchAll(/FACT\s+(fact_\d+):\s+([^\n]+)/g)) {
+    out.push(`${match[1]}: ${match[2].trim()}`);
+  }
+  for (const match of text.matchAll(
+    /\[tool_result:([^\]]+)\]\s+([^\n]+?)\s+\(turn=(\d+)\)/g,
+  )) {
+    out.push(`tool ${match[1]} turn ${match[3]}: ${match[2].trim()}`);
+  }
+  return Array.from(new Set(out));
+}
+
+function extractExpectedAnswer(text: string, fact: Fact): string | null {
+  const factLine = new RegExp(
+    `FACT\\s+${fact.id}:\\s+[^\\n]*expected=${escapeRegExp(fact.expected)}(?:\\s|$)`,
+  );
+  if (factLine.test(text)) return fact.expected;
+  if (text.includes(fact.expected)) return fact.expected;
+  return null;
+}
+
+function escapeRegExp(value: string): string {
+  return value.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
+}
+
+function dryRunModelCall(): CompactorModelCall {
+  return async ({ systemPrompt, messages }) => {
+    const text = messages.map((message) => message.content).join("\n");
+    const facts = extractFacts(text);
+    if (
+      systemPrompt.includes('"facts"') &&
+      systemPrompt.includes("JSON object")
+    ) {
+      return JSON.stringify({
+        facts,
+        decisions: [],
+        pending_actions: [],
+        forbidden_behaviors: [],
+        entities: {},
+      });
+    }
+    if (systemPrompt.includes("hybrid ledger")) {
+      return JSON.stringify({
+        state: {
+          facts,
+          decisions: [],
+          pending_actions: [],
+          forbidden_behaviors: [],
+          entities: {},
+        },
+        ledger: facts.map((fact, index) => ({ index, note: fact })),
+      });
+    }
+    return facts.length > 0
+      ? facts.map((fact) => `Preserved ${fact}`).join("\n")
+      : "No durable planted facts were present in the compacted region.";
+  };
+}
+
+async function chatCompletion(args: {
+  model: string;
+  systemPrompt: string;
+  messages: CompactorMessage[];
+  maxOutputTokens: number;
+  reasoningEffort: ReasoningEffort;
+}): Promise<string> {
+  const apiKey =
+    process.env.CEREBRAS_API_KEY ??
+    process.env.OPENAI_API_KEY ??
+    process.env.OPENAI_COMPAT_API_KEY;
+  if (!apiKey) {
+    throw new Error(
+      "real drift runs require CEREBRAS_API_KEY, OPENAI_API_KEY, or OPENAI_COMPAT_API_KEY; use --dry-run for local smoke tests",
+    );
+  }
+  const baseUrl =
+    process.env.OPENAI_COMPAT_BASE_URL ??
+    process.env.CEREBRAS_BASE_URL ??
+    "https://api.cerebras.ai/v1";
+  const response = await fetch(
+    `${baseUrl.replace(/\/$/, "")}/chat/completions`,
+    {
+      method: "POST",
+      headers: {
+        Authorization: `Bearer ${apiKey}`,
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify({
+        model: args.model,
+        messages: [
+          { role: "system", content: args.systemPrompt },
+          ...args.messages.map((message) => ({
+            role: message.role === "tool" ? "user" : message.role,
+            content: message.content,
+          })),
+        ],
+        max_tokens: args.maxOutputTokens,
+        reasoning_effort: args.reasoningEffort,
+      }),
+    },
+  );
+  if (!response.ok) {
+    throw new Error(
+      `chat completion failed (${response.status}): ${await response.text()}`,
+    );
+  }
+  const payload = (await response.json()) as {
+    choices?: Array<{ message?: { content?: string } }>;
+  };
+  return payload.choices?.[0]?.message?.content?.trim() ?? "";
+}
+
+function realModelCall(args: Args): CompactorModelCall {
+  return ({ systemPrompt, messages, maxOutputTokens }) =>
+    chatCompletion({
+      model: args.compactorModel,
+      systemPrompt,
+      messages,
+      maxOutputTokens: maxOutputTokens ?? 800,
+      reasoningEffort: args.compactorReasoningEffort,
+    });
+}
+
+function promptStripCompact(
+  transcript: CompactorTranscript,
+  preserveTailMessages: number,
+): CompactionArtifact {
+  const startedAt = Date.now();
+  const messages = transcript.messages;
+  const boundary = findSafeCompactionBoundary(messages, preserveTailMessages);
+  const systemPrefix = messages.filter((message, index) => {
+    return (
+      index < boundary &&
+      (message.role === "system" || message.role === "developer")
+    );
+  });
+  const protectedCount = systemPrefix.length;
+  const region = messages.slice(protectedCount, boundary);
+  const preservedTail = messages.slice(boundary);
+  const stripped = region
+    .map((message) => message.content)
+    .join("\n")
+    .replace(/\([^)]*internal thought:[^)]*\)/gi, "")
+    .replace(/\([^)]*actions:[^)]*\)/gi, "")
+    .replace(/\b[0-9a-f]{8}-[0-9a-f-]{27,}\b/gi, "")
+    .replace(/\n{3,}/g, "\n\n")
+    .trim();
+  const replacementMessages: CompactorMessage[] = stripped
+    ? [
+        {
+          role: "assistant",
+          content: `[conversation prompt-stripping]\n${stripped}`,
+          tags: ["compactor:prompt-stripping"],
+        },
+      ]
+    : [];
+  const compacted = [...systemPrefix, ...replacementMessages, ...preservedTail];
+  return {
+    replacementMessages,
+    stats: {
+      originalMessageCount: messages.length,
+      compactedMessageCount: compacted.length,
+      originalTokens: countTranscriptTokens(transcript),
+      compactedTokens: countTranscriptTokens({ messages: compacted }),
+      latencyMs: Date.now() - startedAt,
+      extra: { regionSize: region.length },
+    },
+  };
+}
+
+async function applyStrategy(args: {
+  strategy: StrategyName;
+  transcript: CompactorTranscript;
+  callModel: CompactorModelCall;
+}): Promise<{
+  transcript: CompactorTranscript;
+  stats: CompactionArtifact["stats"];
+}> {
+  const preserveTailMessages = 6;
+  if (args.strategy === "none") {
+    const tokens = countTranscriptTokens(args.transcript);
+    return {
+      transcript: args.transcript,
+      stats: {
+        originalMessageCount: args.transcript.messages.length,
+        compactedMessageCount: args.transcript.messages.length,
+        originalTokens: tokens,
+        compactedTokens: tokens,
+        latencyMs: 0,
+      },
+    };
+  }
+
+  const artifact =
+    args.strategy === "prompt-stripping"
+      ? promptStripCompact(args.transcript, preserveTailMessages)
+      : await compactors[args.strategy].compact(args.transcript, {
+          targetTokens: 700,
+          preserveTailMessages,
+          countTokens: approxCountTokens,
+          callModel: args.callModel,
+          summarizationModel: "drift-harness",
+        });
+
+  const messages = args.transcript.messages;
+  const systemPrefixLength = messages.findIndex(
+    (message) => message.role !== "system" && message.role !== "developer",
+  );
+  const prefixLength =
+    systemPrefixLength === -1 ? messages.length : systemPrefixLength;
+  const boundary = findSafeCompactionBoundary(messages, preserveTailMessages);
+  return {
+    transcript: {
+      messages: [
+        ...messages.slice(0, prefixLength),
+        ...artifact.replacementMessages,
+        ...messages.slice(boundary),
+      ],
+      metadata: args.transcript.metadata,
+    },
+    stats: artifact.stats,
+  };
+}
+
+function emitTurn(
+  lines: string[],
+  turn: number,
+  role: "user" | "assistant",
+  content: string,
+  factId?: string,
+): void {
+  lines.push(
+    JSON.stringify({
+      event: "turn",
+      turn,
+      role,
+      contentLen: content.length,
+      tokens: approxCountTokens(content),
+      ...(factId ? { factId } : {}),
+    }),
+  );
+}
+
+async function answerProbe(args: {
+  harnessArgs: Args;
+  transcript: CompactorTranscript;
+  fact: Fact;
+}): Promise<string> {
+  const transcriptText = renderTranscript(args.transcript.messages);
+  if (args.harnessArgs.dryRun) {
+    return (
+      extractExpectedAnswer(transcriptText, args.fact) ?? "I don't recall."
+    );
+  }
+  return chatCompletion({
+    model: args.harnessArgs.agentModel,
+    systemPrompt:
+      "Answer the recall probe using only the supplied transcript. Return the exact value if present.",
+    messages: [
+      {
+        role: "user",
+        content: `Transcript:\n${transcriptText}\n\nProbe: ${args.fact.question}`,
+      },
+    ],
+    maxOutputTokens: args.harnessArgs.probeMaxTokens,
+    reasoningEffort: args.harnessArgs.agentReasoningEffort,
+  });
+}
+
+async function judgeProbe(args: {
+  harnessArgs: Args;
+  fact: Fact;
+  actual: string;
+}): Promise<{ correct: boolean; reasoning: string }> {
+  const exact = args.actual.includes(args.fact.expected);
+  if (args.harnessArgs.dryRun || exact) {
+    return {
+      correct: exact,
+      reasoning: exact
+        ? "exact-match: expected substring present"
+        : "exact-match: expected substring missing",
+    };
+  }
+  const verdict = await chatCompletion({
+    model: args.harnessArgs.judgeModel,
+    systemPrompt:
+      'Grade whether ACTUAL contains the same exact value as EXPECTED. Respond JSON: {"correct":boolean,"reasoning":"..."}.',
+    messages: [
+      {
+        role: "user",
+        content: `EXPECTED: ${args.fact.expected}\nACTUAL: ${args.actual}`,
+      },
+    ],
+    maxOutputTokens: 200,
+    reasoningEffort: args.harnessArgs.judgeReasoningEffort,
+  });
+  try {
+    const parsed = JSON.parse(verdict) as {
+      correct?: boolean;
+      reasoning?: string;
+    };
+    return {
+      correct: parsed.correct === true,
+      reasoning: parsed.reasoning ?? "judge-json",
+    };
+  } catch {
+    return {
+      correct: exact,
+      reasoning: `judge-unparseable; exact-match=${exact}`,
+    };
+  }
+}
+
+async function probeFacts(args: {
+  harnessArgs: Args;
+  transcript: CompactorTranscript;
+  facts: Fact[];
+  lines: string[];
+  atTurn: number;
+  phase: ProbeEvent["phase"];
+}): Promise<ProbeEvent[]> {
+  const events: ProbeEvent[] = [];
+  for (const fact of args.facts.filter(
+    (item) => item.plantedTurn <= args.atTurn,
+  )) {
+    const actual = await answerProbe({
+      harnessArgs: args.harnessArgs,
+      transcript: args.transcript,
+      fact,
+    });
+    const judgment = await judgeProbe({
+      harnessArgs: args.harnessArgs,
+      fact,
+      actual,
+    });
+    const event: ProbeEvent = {
+      event: "probe",
+      atTurn: args.atTurn,
+      factId: fact.id,
+      plantedTurn: fact.plantedTurn,
+      kind: fact.kind,
+      expected: fact.expected,
+      actual,
+      correct: judgment.correct,
+      judgeReasoning: judgment.reasoning,
+      phase: args.phase,
+    };
+    args.lines.push(JSON.stringify(event));
+    events.push(event);
+  }
+  return events;
+}
+
+function addToolCallMessages(
+  messages: CompactorMessage[],
+  lines: string[],
+  turn: number,
+  facts: Fact[],
+  seed: number,
+): void {
+  const toolName = "lookup_context";
+  const expected = `TOOL-${padded(seed, turn, 6)}`;
+  const fact: Fact = {
+    id: `tool_${turn}`,
+    kind: "tool_result",
+    expected,
+    plantedTurn: turn,
+    question: `What did ${toolName} return at turn ${turn}?`,
+  };
+  facts.push(fact);
+  const assistant: CompactorMessage = {
+    role: "assistant",
+    content: `[tool_call:${toolName}] requesting synthetic value for turn ${turn}`,
+    toolCalls: [{ id: `call_${turn}`, name: toolName, arguments: { turn } }],
+  };
+  const tool: CompactorMessage = {
+    role: "tool",
+    toolCallId: `call_${turn}`,
+    toolName,
+    content: `[tool_result:${toolName}] ${expected} (turn=${turn})`,
+  };
+  messages.push(assistant, tool);
+  emitTurn(lines, turn, "assistant", assistant.content);
+  lines.push(
+    JSON.stringify({
+      event: "turn",
+      turn,
+      role: "tool",
+      contentLen: tool.content.length,
+      tokens: approxCountTokens(tool.content),
+      factId: fact.id,
+    }),
+  );
+}
+
+async function main(): Promise<number> {
+  const args = parseArgs(Bun.argv.slice(2));
+  const facts = buildFacts(args);
+  const factsByTurn = new Map<number, Fact[]>();
+  for (const fact of facts) {
+    const items = factsByTurn.get(fact.plantedTurn) ?? [];
+    items.push(fact);
+    factsByTurn.set(fact.plantedTurn, items);
+  }
+
+  const lines: string[] = [];
+  const transcript: CompactorTranscript = {
+    messages: [{ role: "system", content: defaultSystemPrompt(args) }],
+    metadata: { seed: args.seed, strategy: args.strategy },
+  };
+  const callModel = args.dryRun ? dryRunModelCall() : realModelCall(args);
+  let totalCompactions = 0;
+  let totalTokensSaved = 0;
+  const probes: ProbeEvent[] = [];
+
+  for (let turn = 1; turn <= args.turns; turn++) {
+    const planted = factsByTurn.get(turn) ?? [];
+    const durableFacts = planted.map(
+      (fact) =>
+        `FACT ${fact.id}: kind=${fact.kind} expected=${fact.expected} question="${fact.question}"`,
+    );
+    const filler = `Turn ${turn}: continue the planning conversation and keep prior durable facts available.`;
+    const userContent = [filler, ...durableFacts].join("\n");
+    transcript.messages.push({ role: "user", content: userContent });
+    emitTurn(lines, turn, "user", userContent, planted[0]?.id);
+
+    const assistantContent = planted.length
+      ? `Acknowledged ${planted.map((fact) => fact.id).join(", ")}.`
+      : `Acknowledged turn ${turn}.`;
+    transcript.messages.push({ role: "assistant", content: assistantContent });
+    emitTurn(lines, turn, "assistant", assistantContent);
+
+    if (args.withToolCalls && turn % 5 === 0) {
+      addToolCallMessages(transcript.messages, lines, turn, facts, args.seed);
+    }
+
+    if (turn % args.compactEvery === 0 && args.strategy !== "none") {
+      const result = await applyStrategy({
+        strategy: args.strategy,
+        transcript,
+        callModel,
+      });
+      transcript.messages = result.transcript.messages;
+      totalCompactions++;
+      totalTokensSaved += Math.max(
+        0,
+        result.stats.originalTokens - result.stats.compactedTokens,
+      );
+      lines.push(
+        JSON.stringify({
+          event: "compact",
+          atTurn: turn,
+          strategy: args.strategy,
+          originalTokens: result.stats.originalTokens,
+          compactedTokens: result.stats.compactedTokens,
+          latencyMs: result.stats.latencyMs,
+        }),
+      );
+      probes.push(
+        ...(await probeFacts({
+          harnessArgs: args,
+          transcript,
+          facts,
+          lines,
+          atTurn: turn,
+          phase: "post-compact",
+        })),
+      );
+    }
+  }
+
+  probes.push(
+    ...(await probeFacts({
+      harnessArgs: args,
+      transcript,
+      facts,
+      lines,
+      atTurn: args.turns,
+      phase: "final",
+    })),
+  );
+
+  const totalCorrect = probes.filter((probe) => probe.correct).length;
+  const perKind = new Map<string, { correct: number; total: number }>();
+  for (const probe of probes) {
+    const stats = perKind.get(probe.kind) ?? { correct: 0, total: 0 };
+    stats.total++;
+    if (probe.correct) stats.correct++;
+    perKind.set(probe.kind, stats);
+  }
+  const perKindAccuracy = Object.fromEntries(
+    Array.from(perKind.entries()).map(([kind, stats]) => [
+      kind,
+      {
+        correct: stats.correct,
+        total: stats.total,
+        accuracy: stats.total > 0 ? stats.correct / stats.total : 0,
+      },
+    ]),
+  );
+
+  lines.push(
+    JSON.stringify({
+      event: "summary",
+      strategy: args.strategy,
+      overallAccuracy: probes.length > 0 ? totalCorrect / probes.length : 0,
+      totalCompactions,
+      totalTokensSaved,
+      totalProbes: probes.length,
+      totalCorrect,
+      seed: args.seed,
+      turns: args.turns,
+      compactEvery: args.compactEvery,
+      plantFacts: args.plantFacts,
+      valid: true,
+      skipped: false,
+      perKindAccuracy,
+    }),
+  );
+
+  const output = `${lines.join("\n")}\n`;
+  if (args.output) {
+    await Bun.write(args.output, output);
+  }
+  process.stdout.write(output);
+  return 0;
+}
+
+main().then(
+  (code) => process.exit(code),
+  (error: unknown) => {
+    process.stderr.write(
+      `${error instanceof Error ? error.message : String(error)}\n\n${usage()}\n`,
+    );
+    process.exit(1);
+  },
+);

scripts/build-riscv64-artifacts.sh

@@ -0,0 +1,252 @@
+#!/usr/bin/env bash
+# build-riscv64-artifacts.sh — build-driver for every riscv64 cross-build
+# the QEMU smoke harness consumes.
+#
+# Walks the same artifact list as scripts/check-riscv64-artifacts.sh and
+# runs the relevant cross-build for each artifact that's missing. Each
+# step is idempotent: if the output already exists and is rv64 ELF, the
+# step is skipped (so callers can re-run cheaply).
+#
+# Gated on ELIZA_RISCV64_SMOKE=1 by default (same posture as the smoke
+# harness). Unset = skip all builds.
+#
+# Tooling requirements (caller's job to install):
+#   - zig 0.14+        (Zig toolchain; provides riscv64-linux-musl — every
+#                       cross-build here is zig/musl, so no Android NDK is needed)
+#   - cmake 3.21+      (drives every package's cross-build)
+#   - node 20+         (drives compile-libllama.mjs)
+#
+# Usage:
+#   ELIZA_RISCV64_SMOKE=1 bash scripts/build-riscv64-artifacts.sh
+#   ELIZA_RISCV64_SMOKE=1 bash scripts/build-riscv64-artifacts.sh --force         # rebuild even if present
+#
+# Exit code:
+#   0 — every reachable build succeeded (or already present)
+#   1 — at least one build failed
+#   2 — missing required toolchain (zig / cmake) — caller must install
+
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$repo_root"
+
+JOBS="${JOBS:-$(nproc 2>/dev/null || echo 4)}"
+FORCE=0
+
+while [ $# -gt 0 ]; do
+    case "$1" in
+        --jobs) JOBS="$2"; shift 2;;
+        --force) FORCE=1; shift;;
+        -h|--help)
+            awk '/^# /{print substr($0,3)} /^#$/{print ""} !/^#/{exit}' "$0"
+            exit 0;;
+        *) echo "unknown argument: $1" >&2; exit 2;;
+    esac
+done
+
+if [ "${ELIZA_RISCV64_SMOKE:-0}" != "1" ]; then
+    echo "[build-riscv64-artifacts] ELIZA_RISCV64_SMOKE not set; nothing to do."
+    echo "[build-riscv64-artifacts] To run: ELIZA_RISCV64_SMOKE=1 bun run build:riscv64-artifacts"
+    exit 0
+fi
+
+# ── Toolchain pre-flight ─────────────────────────────────────────────
+ZIG_BIN="${ZIG_BIN:-$(command -v zig || true)}"
+if [ -z "$ZIG_BIN" ]; then
+    cat >&2 <<'EOF'
+[build-riscv64-artifacts] zig not on PATH.
+
+Install Zig 0.14+ from https://ziglang.org/download/ — every cross-build in
+this harness drives `zig cc --target=riscv64-linux-musl` directly.
+
+EOF
+    exit 2
+fi
+ZIG_VERSION="$($ZIG_BIN version)"
+ZIG_MAJOR_MINOR="$(printf '%s' "$ZIG_VERSION" | awk -F. '{ print $1"."$2 }')"
+export ZIG_BIN
+
+if ! command -v cmake >/dev/null 2>&1; then
+    echo "[build-riscv64-artifacts] cmake not on PATH. Install cmake 3.21+." >&2
+    exit 2
+fi
+
+NODE_BIN="${NODE_BIN:-$(command -v node || true)}"
+if [ -z "$NODE_BIN" ]; then
+    echo "[build-riscv64-artifacts] node not on PATH. Install node 20+." >&2
+    exit 2
+fi
+RM_PATH_RECURSIVE="$repo_root/packages/scripts/rm-path-recursive.mjs"
+
+remove_path_recursive() {
+    "$NODE_BIN" "$RM_PATH_RECURSIVE" "$@"
+}
+
+echo "[build-riscv64-artifacts] zig=$ZIG_VERSION  cmake=$(cmake --version | head -1)  node=$($NODE_BIN --version)"
+echo "[build-riscv64-artifacts] jobs=$JOBS  force=$FORCE"
+
+# ── Helpers ──────────────────────────────────────────────────────────
+FAIL_N=0
+
+is_riscv64_elf() {
+    local f="$1"
+    [ -f "$f" ] || return 1
+    file -b "$f" 2>/dev/null | grep -q "UCB RISC-V"
+}
+
+should_build() {
+    # $1 = sentinel path. Returns 0 if we should build, 1 if we should skip.
+    local sentinel="$1"
+    if [ "$FORCE" = "1" ]; then return 0; fi
+    if [ -e "$sentinel" ]; then
+        echo "  → up-to-date: $sentinel"
+        return 1
+    fi
+    return 0
+}
+
+build_native_plugin() {
+    local pkg="$1"; local extra_flag="${2:-}"
+    local pkgdir="$repo_root/packages/native/plugins/$pkg"
+    local builddir="$pkgdir/build/riscv64"
+    if [ ! -f "$pkgdir/CMakeLists.txt" ]; then
+        echo "  ✗ $pkg: $pkgdir/CMakeLists.txt missing"
+        FAIL_N=$((FAIL_N+1))
+        return
+    fi
+    local sentinel_a="$builddir/lib${pkg%-cpu}.a"
+    case "$pkg" in
+        silero-vad-cpp) sentinel_a="$builddir/libsilero_vad.a";;
+        voice-classifier-cpp) sentinel_a="$builddir/libvoice_classifier.a";;
+        wakeword-cpp) sentinel_a="$builddir/libwakeword.a";;
+        yolo-cpp) sentinel_a="$builddir/libyolo.a";;
+        face-cpp) sentinel_a="$builddir/libface.a";;
+        doctr-cpp) sentinel_a="$builddir/libdoctr.a";;
+        polarquant-cpu) sentinel_a="$builddir/libpolarquant.a";;
+        turboquant-cpu) sentinel_a="$builddir/libturboquant.a";;
+        qjl-cpu) sentinel_a="$builddir/libqjl.a";;
+    esac
+    if ! should_build "$sentinel_a"; then return; fi
+    echo "→ Building $pkg (riscv64) …"
+    remove_path_recursive "$builddir"
+    mkdir -p "$builddir"
+    local config_log="$builddir.config.log"
+    local build_log="$builddir.build.log"
+    if [ -n "$extra_flag" ]; then
+        if ! cmake -S "$pkgdir" -B "$builddir" \
+            -DCMAKE_TOOLCHAIN_FILE="$repo_root/packages/native/cmake/toolchain-riscv64-linux-musl.cmake" \
+            "$extra_flag" >"$config_log" 2>&1; then
+            echo "  ✗ $pkg: cmake configure failed (see $config_log)"
+            FAIL_N=$((FAIL_N+1)); return
+        fi
+    else
+        if ! cmake -S "$pkgdir" -B "$builddir" \
+            -DCMAKE_TOOLCHAIN_FILE="$repo_root/packages/native/cmake/toolchain-riscv64-linux-musl.cmake" \
+            >"$config_log" 2>&1; then
+            echo "  ✗ $pkg: cmake configure failed (see $config_log)"
+            FAIL_N=$((FAIL_N+1)); return
+        fi
+    fi
+    if ! cmake --build "$builddir" -j"$JOBS" >"$build_log" 2>&1; then
+        echo "  ✗ $pkg: cmake build failed (see $build_log)"
+        FAIL_N=$((FAIL_N+1)); return
+    fi
+    echo "  ✓ $pkg: $builddir"
+}
+
+# RVV flag escapes for Zig 0.13 (same logic as
+# scripts/verify-riscv64-buildpaths.sh). TurboQuant uses the generic
+# CPU (no zvl* attribute) so LLVM's loop vectoriser emits portable
+# vsetvli loops; a named core like sifive_x280 advertises VLEN=512
+# via zvl512b and produces code that silently truncates at smaller
+# VLEN (qemu-user reports VLEN=128, the RVV-spec minimum).
+QJL_RVV=""; POLAR_RVV=""; TBQ_RVV=""
+if [ "$ZIG_MAJOR_MINOR" = "0.13" ]; then
+    QJL_RVV="-DQJL_RVV_COMPILE_OPTIONS=-mcpu=sifive_x280;-mabi=lp64d"
+    POLAR_RVV="-DPOLARQUANT_RVV_COMPILE_OPTIONS=-mcpu=sifive_x280;-mabi=lp64d"
+    TBQ_RVV="-DTURBOQUANT_RVV_FLAGS=-mcpu=generic_rv64+v+m+a+f+d+c"
+fi
+
+# ── Native plugins ───────────────────────────────────────────────────
+echo
+echo "── Step 1: native plugins (CPU-side kernels + speech/vision libs) ──"
+build_native_plugin qjl-cpu              "$QJL_RVV"
+build_native_plugin polarquant-cpu       "$POLAR_RVV"
+build_native_plugin turboquant-cpu       "$TBQ_RVV"
+build_native_plugin silero-vad-cpp       ""
+build_native_plugin voice-classifier-cpp ""
+build_native_plugin wakeword-cpp         ""
+build_native_plugin yolo-cpp             ""
+build_native_plugin face-cpp             ""
+build_native_plugin doctr-cpp            ""
+
+# ── libllama family (MTP) ─────────────────────────────────────────
+echo
+echo "── Step 2: libllama / libggml family (MTP) ──"
+COMPILE_LIBLLAMA="$repo_root/packages/app-core/scripts/aosp/compile-libllama.mjs"
+if [ ! -f "$COMPILE_LIBLLAMA" ]; then
+    echo "  ✗ compile-libllama.mjs missing at $COMPILE_LIBLLAMA"
+    FAIL_N=$((FAIL_N+1))
+else
+    # compile-libllama.mjs exposes exactly one riscv64 target: android-riscv64-cpu.
+    # Despite the name it produces a Linux/musl shared object — zig
+    # `riscv64-linux-musl` driver, -DCMAKE_SYSTEM_NAME=Linux, no Android NDK in
+    # the compile, strip via `zig objcopy` (NDK llvm-strip is only a non-fatal
+    # last-resort fallback). So it is the canonical riscv64 libllama for both the
+    # APK and bare Linux riscv64, and it does NOT require the NDK. (There is no
+    # separate `linux-riscv64-cpu` libllama target; the earlier invocation of one
+    # always threw "unsupported --target".)
+    #
+    # repoRoot inside compile-libllama resolves to the consumer root, so
+    # its default --assets-dir/--src-dir would land in the wrong tree. Pin both
+    # to this eliza checkout so the .so installs where check-riscv64-artifacts.sh
+    # scans for it and the build uses the in-repo llama.cpp submodule.
+    libllama_assets_dir="$repo_root/packages/app-core/platforms/android/app/src/main/assets/agent"
+    libllama_sentinel="$libllama_assets_dir/riscv64/libllama.so"
+    libllama_src_args=()
+    if [ -f "$repo_root/plugins/plugin-local-inference/native/llama.cpp/CMakeLists.txt" ]; then
+        libllama_src_args=(--src-dir "$repo_root/plugins/plugin-local-inference/native/llama.cpp")
+    fi
+    if should_build "$libllama_sentinel"; then
+        echo "→ Building libllama (riscv64, zig/musl) …"
+        mkdir -p "$repo_root/build"
+        if "$NODE_BIN" "$COMPILE_LIBLLAMA" --target android-riscv64-cpu \
+            --assets-dir "$libllama_assets_dir" \
+            "${libllama_src_args[@]}" \
+            >"$repo_root/build/libllama-riscv64.log" 2>&1; then
+            echo "  ✓ libllama riscv64"
+        else
+            echo "  ✗ libllama riscv64 (see build/libllama-riscv64.log)"
+            FAIL_N=$((FAIL_N+1))
+        fi
+    fi
+fi
+
+# ── sigsys-handler-riscv64 ───────────────────────────────────────────
+echo
+echo "── Step 3: libsigsys-handler-riscv64 (Bun seccomp shim) ──"
+COMPILE_SHIM="$repo_root/packages/app-core/scripts/aosp/compile-shim.mjs"
+if [ ! -f "$COMPILE_SHIM" ]; then
+    echo "  ✗ compile-shim.mjs missing at $COMPILE_SHIM"
+    FAIL_N=$((FAIL_N+1))
+else
+    shim_sentinel="${HOME}/.cache/eliza-android-agent/seccomp-shim/riscv64/libsigsys-handler.so"
+    if should_build "$shim_sentinel"; then
+        echo "→ Building libsigsys-handler (riscv64) …"
+        mkdir -p "$repo_root/build"
+        if "$NODE_BIN" "$COMPILE_SHIM" --abi riscv64 >"$repo_root/build/libsigsys-handler-riscv64.log" 2>&1; then
+            echo "  ✓ libsigsys-handler riscv64"
+        else
+            echo "  ✗ libsigsys-handler riscv64 (see build/libsigsys-handler-riscv64.log)"
+            FAIL_N=$((FAIL_N+1))
+        fi
+    fi
+fi
+
+echo
+if [ "$FAIL_N" -gt 0 ]; then
+    echo "[build-riscv64-artifacts] $FAIL_N build(s) failed."
+    exit 1
+fi
+echo "[build-riscv64-artifacts] All riscv64 artifacts built (or already present)."
+exit 0

scripts/certification/check-commit-drift.mjs

@@ -0,0 +1,241 @@
+#!/usr/bin/env node
+/**
+ * Commit-drift check for the develop→main certification gate (#14547). A
+ * certification signs one exact commit, but a promotion branch may pick up
+ * non-source commits after signing (README and docs touch-ups). This
+ * helper decides whether the PR head is still covered by the certification:
+ * pass iff head == cert.commit, or cert.commit is an ancestor of head and
+ * every path in `git diff cert.commit..head` matches the docs-only allowlist.
+ *
+ * The allowlist is deliberately narrow: docs/**, packages/docs/**, any *.md,
+ * and the certification artifacts. GitHub workflow/policy/config drift must
+ * force re-certification, never ride the docs allowlist.
+ *
+ * Consumed by .github/workflows/certification-verify.yml; it reads the
+ * certification only to extract `commit` — all cryptographic and schema
+ * verification stays in `packages/evidence certify:verify`.
+ */
+
+import { execFileSync } from "node:child_process";
+import { appendFileSync, readFileSync, writeFileSync } from "node:fs";
+import { pathToFileURL } from "node:url";
+
+/** Paths that may change after certification without re-certifying. */
+export function isAllowedDriftPath(filePath) {
+  // Trust anchor and gate plumbing: never allowed as post-certification drift.
+  if (filePath.startsWith(".github/certification/")) return false;
+  if (filePath === ".github/workflows/certification-verify.yml") return false;
+  if (filePath.startsWith("scripts/certification/")) return false;
+
+  // The certification artifacts themselves necessarily land AFTER the
+  // certified commit (the signature covers the commit sha, so the commit that
+  // adds the file can never be the signed one). Their integrity is enforced
+  // cryptographically — signature over the payload, bundleSha over the bundle
+  // — not by the drift rule, so allowing them here weakens nothing.
+  if (filePath === "certification.json") return true;
+  if (filePath.startsWith("evidence/bundle/")) return true;
+
+  if (filePath.startsWith("docs/")) return true;
+  if (filePath.startsWith("packages/docs/")) return true;
+  if (filePath.endsWith(".md")) return true;
+  return false;
+}
+
+const SHA_RE = /^[0-9a-f]{40}$/;
+
+function git(repoDir, args) {
+  return execFileSync("git", args, {
+    cwd: repoDir,
+    encoding: "utf8",
+    stdio: ["ignore", "pipe", "pipe"],
+  });
+}
+
+/** Read `commit` out of a certification.json without validating anything else. */
+export function readCertCommit(certPath) {
+  let raw;
+  try {
+    raw = readFileSync(certPath, "utf8");
+  } catch (error) {
+    return { error: `certification unreadable: ${error.message}` };
+  }
+  let parsed;
+  try {
+    parsed = JSON.parse(raw);
+  } catch (error) {
+    return { error: `certification is not valid JSON: ${error.message}` };
+  }
+  const commit = parsed?.commit;
+  if (typeof commit !== "string" || !SHA_RE.test(commit)) {
+    return {
+      error: `certification \`commit\` is not a full 40-hex sha: ${JSON.stringify(commit)}`,
+    };
+  }
+  return { commit };
+}
+
+/**
+ * Evaluate drift between the certified commit and the PR head.
+ * Result shape: { result, certCommit, headCommit, driftPaths, disallowedPaths, detail }
+ * where result is one of:
+ *   match | allowed-drift             → covered by the certification
+ *   cert-commit-unknown | not-ancestor | disallowed-drift  → not covered
+ */
+export function evaluateCommitDrift({ certCommit, headCommit, repoDir }) {
+  const base = {
+    certCommit,
+    headCommit,
+    driftPaths: [],
+    disallowedPaths: [],
+  };
+  if (certCommit === headCommit) {
+    return {
+      ...base,
+      result: "match",
+      detail: "certification commit equals the PR head",
+    };
+  }
+
+  try {
+    git(repoDir, ["cat-file", "-e", `${certCommit}^{commit}`]);
+  } catch {
+    // A cert for a commit this repository has never seen (or outside the
+    // fetched history window) cannot cover this head. Shallow clones can
+    // produce this for very old certs — that fails safe: stale certs must
+    // re-certify anyway.
+    return {
+      ...base,
+      result: "cert-commit-unknown",
+      detail: `certified commit ${certCommit} is not present in this repository's fetched history`,
+    };
+  }
+
+  let isAncestor = true;
+  try {
+    git(repoDir, ["merge-base", "--is-ancestor", certCommit, headCommit]);
+  } catch (error) {
+    if (error.status === 1) {
+      isAncestor = false;
+    } else {
+      throw error;
+    }
+  }
+  if (!isAncestor) {
+    return {
+      ...base,
+      result: "not-ancestor",
+      detail: `certified commit ${certCommit} is not an ancestor of head ${headCommit} — the certified tree was never part of this branch`,
+    };
+  }
+
+  const driftPaths = git(repoDir, [
+    "diff",
+    "--name-only",
+    "-z",
+    certCommit,
+    headCommit,
+  ])
+    .split("\0")
+    .filter((entry) => entry.length > 0);
+  const disallowedPaths = driftPaths.filter(
+    (entry) => !isAllowedDriftPath(entry),
+  );
+  if (disallowedPaths.length > 0) {
+    return {
+      ...base,
+      driftPaths,
+      disallowedPaths,
+      result: "disallowed-drift",
+      detail: `${disallowedPaths.length} of ${driftPaths.length} drifted path(s) fall outside the docs-only allowlist — re-certify at the current head`,
+    };
+  }
+  return {
+    ...base,
+    driftPaths,
+    result: "allowed-drift",
+    detail: `${driftPaths.length} drifted path(s), all inside the docs-only allowlist`,
+  };
+}
+
+function parseArgs(argv) {
+  const args = { repoDir: process.cwd() };
+  for (let index = 0; index < argv.length; index += 1) {
+    const flag = argv[index];
+    const value = () => {
+      const next = argv[index + 1];
+      if (next === undefined) {
+        console.error(`${flag} requires a value`);
+        process.exit(2);
+      }
+      index += 1;
+      return next;
+    };
+    if (flag === "--cert") args.certPath = value();
+    else if (flag === "--head") args.headCommit = value();
+    else if (flag === "--repo") args.repoDir = value();
+    else if (flag === "--json-out") args.jsonOut = value();
+    else if (flag === "--github-output") args.githubOutput = value();
+    else {
+      console.error(`unknown argument: ${flag}`);
+      process.exit(2);
+    }
+  }
+  if (args.certPath === undefined || args.headCommit === undefined) {
+    console.error(
+      "Usage: check-commit-drift.mjs --cert <certification.json> --head <sha> [--repo <dir>] [--json-out <file>] [--github-output <file>]",
+    );
+    process.exit(2);
+  }
+  if (!SHA_RE.test(args.headCommit)) {
+    console.error(`--head must be a full 40-hex sha, got: ${args.headCommit}`);
+    process.exit(2);
+  }
+  return args;
+}
+
+function main() {
+  const args = parseArgs(process.argv.slice(2));
+  const certRead = readCertCommit(args.certPath);
+  const outcome =
+    "error" in certRead
+      ? {
+          result: "cert-unreadable",
+          certCommit: null,
+          headCommit: args.headCommit,
+          driftPaths: [],
+          disallowedPaths: [],
+          detail: certRead.error,
+        }
+      : evaluateCommitDrift({
+          certCommit: certRead.commit,
+          headCommit: args.headCommit,
+          repoDir: args.repoDir,
+        });
+
+  if (args.jsonOut !== undefined) {
+    writeFileSync(args.jsonOut, `${JSON.stringify(outcome, null, 2)}\n`);
+  }
+  if (args.githubOutput !== undefined) {
+    appendFileSync(
+      args.githubOutput,
+      `cert-commit=${outcome.certCommit ?? ""}\nresult=${outcome.result}\n`,
+    );
+  }
+
+  const ok = outcome.result === "match" || outcome.result === "allowed-drift";
+  console.log(`[check-commit-drift] ${outcome.result}: ${outcome.detail}`);
+  for (const entry of outcome.driftPaths) {
+    const marker = outcome.disallowedPaths.includes(entry)
+      ? "DISALLOWED"
+      : "allowed   ";
+    console.log(`  ${marker} ${entry}`);
+  }
+  process.exit(ok ? 0 : 1);
+}
+
+if (
+  process.argv[1] &&
+  import.meta.url === pathToFileURL(process.argv[1]).href
+) {
+  main();
+}

scripts/certification/check-commit-drift.test.mjs

@@ -0,0 +1,244 @@
+/**
+ * Tests for the certification commit-drift check. Fixture repositories are
+ * real `git init` repos built in a temp dir per test — no mocked git — so the
+ * ancestor and diff semantics under test are exactly what CI runs.
+ */
+
+import assert from "node:assert/strict";
+import { execFileSync } from "node:child_process";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join } from "node:path";
+import { after, describe, it } from "node:test";
+import {
+  evaluateCommitDrift,
+  isAllowedDriftPath,
+  readCertCommit,
+} from "./check-commit-drift.mjs";
+
+const roots = [];
+after(() => {
+  for (const root of roots) rmSync(root, { recursive: true, force: true });
+});
+
+function git(repoDir, args) {
+  return execFileSync("git", args, {
+    cwd: repoDir,
+    encoding: "utf8",
+    stdio: ["ignore", "pipe", "pipe"],
+  }).trim();
+}
+
+function makeRepo() {
+  const repoDir = mkdtempSync(join(tmpdir(), "cert-drift-test-"));
+  roots.push(repoDir);
+  git(repoDir, ["init", "-b", "main"]);
+  git(repoDir, ["config", "user.email", "drift-test@example.invalid"]);
+  git(repoDir, ["config", "user.name", "drift test"]);
+  return repoDir;
+}
+
+function commitFiles(repoDir, message, files) {
+  for (const [relPath, content] of Object.entries(files)) {
+    const filePath = join(repoDir, ...relPath.split("/"));
+    mkdirSync(dirname(filePath), { recursive: true });
+    writeFileSync(filePath, content);
+  }
+  git(repoDir, ["add", "-A"]);
+  git(repoDir, ["commit", "-m", message]);
+  return git(repoDir, ["rev-parse", "HEAD"]);
+}
+
+describe("isAllowedDriftPath", () => {
+  const allowed = [
+    "docs/promotion-notes.md",
+    "docs/nested/deep/guide.txt",
+    "packages/docs/src/pages/index.tsx",
+    "README.md",
+    "packages/core/CHANGELOG.md",
+    ".github/pull_request_template.md",
+    // The certification artifacts land after the signed commit by necessity;
+    // signature + bundleSha protect their content, not the drift rule.
+    "certification.json",
+    "evidence/bundle/manifest.json",
+    "evidence/bundle/lanes/e2e/result.json",
+  ];
+  const disallowed = [
+    "packages/core/src/runtime.ts",
+    "scripts/run-anything.mjs",
+    ".github/certification/certification-public-key.pem",
+    ".github/certification/README.md",
+    ".github/workflows/certification-verify.yml",
+    ".github/workflows/ci.yaml",
+    ".github/actions/setup/action.yml",
+    ".github/CODEOWNERS",
+    ".github/ISSUE_TEMPLATE/config.yml",
+    "scripts/certification/check-commit-drift.mjs",
+    "package.json",
+    "bun.lock",
+    "docsish/trap.ts",
+    "nested/certification.json",
+    "evidence/other/file.json",
+  ];
+  for (const path of allowed) {
+    it(`allows ${path}`, () => assert.equal(isAllowedDriftPath(path), true));
+  }
+  for (const path of disallowed) {
+    it(`rejects ${path}`, () => assert.equal(isAllowedDriftPath(path), false));
+  }
+});
+
+describe("readCertCommit", () => {
+  it("extracts a full 40-hex commit", () => {
+    const dir = mkdtempSync(join(tmpdir(), "cert-drift-cert-"));
+    roots.push(dir);
+    const sha = "a".repeat(40);
+    const certPath = join(dir, "certification.json");
+    writeFileSync(certPath, JSON.stringify({ schema: 1, commit: sha }));
+    assert.deepEqual(readCertCommit(certPath), { commit: sha });
+  });
+
+  it("reports a missing file instead of throwing", () => {
+    const read = readCertCommit(join(tmpdir(), "does-not-exist-cert.json"));
+    assert.match(read.error, /certification unreadable/);
+  });
+
+  it("reports malformed JSON", () => {
+    const dir = mkdtempSync(join(tmpdir(), "cert-drift-cert-"));
+    roots.push(dir);
+    const certPath = join(dir, "certification.json");
+    writeFileSync(certPath, "{not json");
+    assert.match(readCertCommit(certPath).error, /not valid JSON/);
+  });
+
+  it("rejects short or missing commit fields", () => {
+    const dir = mkdtempSync(join(tmpdir(), "cert-drift-cert-"));
+    roots.push(dir);
+    const certPath = join(dir, "certification.json");
+    writeFileSync(certPath, JSON.stringify({ commit: "abc123" }));
+    assert.match(readCertCommit(certPath).error, /40-hex/);
+    writeFileSync(certPath, JSON.stringify({ schema: 1 }));
+    assert.match(readCertCommit(certPath).error, /40-hex/);
+  });
+});
+
+describe("evaluateCommitDrift", () => {
+  it("matches when the certification commit equals head", () => {
+    const repoDir = makeRepo();
+    const sha = commitFiles(repoDir, "base", {
+      "packages/core/src/index.ts": "export {};\n",
+    });
+    const outcome = evaluateCommitDrift({
+      certCommit: sha,
+      headCommit: sha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "match");
+    assert.deepEqual(outcome.driftPaths, []);
+  });
+
+  it("allows docs-only drift after the certified commit", () => {
+    const repoDir = makeRepo();
+    const certSha = commitFiles(repoDir, "certified tree", {
+      "packages/core/src/index.ts": "export {};\n",
+    });
+    const headSha = commitFiles(repoDir, "docs drift", {
+      "docs/promotion-notes.md": "# notes\n",
+      "packages/docs/guide.mdx": "guide\n",
+      "README.md": "# readme\n",
+      ".github/pull_request_template.md": "# template\n",
+    });
+    const outcome = evaluateCommitDrift({
+      certCommit: certSha,
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "allowed-drift");
+    assert.equal(outcome.driftPaths.length, 4);
+    assert.deepEqual(outcome.disallowedPaths, []);
+  });
+
+  it("fails when drift touches source paths, listing the offenders", () => {
+    const repoDir = makeRepo();
+    const certSha = commitFiles(repoDir, "certified tree", {
+      "packages/core/src/index.ts": "export {};\n",
+    });
+    const headSha = commitFiles(repoDir, "mixed drift", {
+      "docs/ok.md": "fine\n",
+      "packages/core/src/runtime.ts": "export const x = 1;\n",
+    });
+    const outcome = evaluateCommitDrift({
+      certCommit: certSha,
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "disallowed-drift");
+    assert.deepEqual(outcome.disallowedPaths, ["packages/core/src/runtime.ts"]);
+  });
+
+  it("fails when drift touches GitHub workflow or policy paths", () => {
+    const repoDir = makeRepo();
+    const certSha = commitFiles(repoDir, "certified tree", {
+      "packages/core/src/index.ts": "export {};\n",
+    });
+    const headSha = commitFiles(repoDir, "github policy drift", {
+      ".github/workflows/ci.yaml": "name: ci\n",
+      ".github/CODEOWNERS": "* @elizaOS/core\n",
+    });
+    const outcome = evaluateCommitDrift({
+      certCommit: certSha,
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "disallowed-drift");
+    assert.deepEqual([...outcome.disallowedPaths].sort(), [
+      ".github/CODEOWNERS",
+      ".github/workflows/ci.yaml",
+    ]);
+  });
+
+  it("fails when drift swaps the trusted public key (the attack the base-key rule exists for)", () => {
+    const repoDir = makeRepo();
+    const certSha = commitFiles(repoDir, "certified tree", {
+      "packages/core/src/index.ts": "export {};\n",
+    });
+    const headSha = commitFiles(repoDir, "key swap", {
+      ".github/certification/certification-public-key.pem": "FAKE KEY\n",
+    });
+    const outcome = evaluateCommitDrift({
+      certCommit: certSha,
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "disallowed-drift");
+    assert.deepEqual(outcome.disallowedPaths, [
+      ".github/certification/certification-public-key.pem",
+    ]);
+  });
+
+  it("fails when the certified commit is not an ancestor of head", () => {
+    const repoDir = makeRepo();
+    commitFiles(repoDir, "trunk base", { "a.txt": "a\n" });
+    git(repoDir, ["checkout", "-b", "side"]);
+    const sideSha = commitFiles(repoDir, "side work", { "side.txt": "s\n" });
+    git(repoDir, ["checkout", "main"]);
+    const headSha = commitFiles(repoDir, "trunk head", { "b.txt": "b\n" });
+    const outcome = evaluateCommitDrift({
+      certCommit: sideSha,
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "not-ancestor");
+  });
+
+  it("fails when the certified commit does not exist in the repository", () => {
+    const repoDir = makeRepo();
+    const headSha = commitFiles(repoDir, "only commit", { "a.txt": "a\n" });
+    const outcome = evaluateCommitDrift({
+      certCommit: "f".repeat(40),
+      headCommit: headSha,
+      repoDir,
+    });
+    assert.equal(outcome.result, "cert-commit-unknown");
+  });
+});

scripts/certification/render-check-summary.mjs

@@ -0,0 +1,220 @@
+#!/usr/bin/env node
+/**
+ * Render the certification-verify gate result for humans. Consumes the
+ * `certify:verify --json` report plus the commit-drift outcome and emits
+ * either GitHub step-summary markdown (`--mode summary`) or `::error`
+ * workflow-command annotations (`--mode annotations`). Display only — every
+ * pass/fail decision was already made by `certify:verify` and
+ * check-commit-drift.mjs; this file must never turn a red input green.
+ *
+ * The point of the summary is the trust handoff: the human merging to main
+ * sees WHO certified WHAT (reviewer identity is part of the signed payload
+ * precisely because a signature proves custody, not diligence).
+ */
+
+import { createHash, createPublicKey } from "node:crypto";
+import { readFileSync } from "node:fs";
+import { pathToFileURL } from "node:url";
+
+/**
+ * Display-only fingerprint (first 16 hex of sha256 over the SPKI DER) so the
+ * summary names which trusted key was used. The canonical implementation the
+ * verifier trusts lives in packages/evidence/src/certify/keys.ts.
+ */
+export function fingerprintPem(pem) {
+  const der = createPublicKey(pem).export({ type: "spki", format: "der" });
+  return createHash("sha256").update(der).digest("hex").slice(0, 16);
+}
+
+function readJsonIfPossible(filePath) {
+  if (filePath === undefined) return undefined;
+  try {
+    return JSON.parse(readFileSync(filePath, "utf8"));
+  } catch {
+    return undefined;
+  }
+}
+
+function formatAge(createdAt, now = Date.now()) {
+  const createdMs = Date.parse(createdAt);
+  if (Number.isNaN(createdMs)) return "unparseable createdAt";
+  const hours = (now - createdMs) / 3_600_000;
+  return `${hours.toFixed(1)}h old`;
+}
+
+function escapeCell(text) {
+  return String(text).replace(/\|/g, "\\|").replace(/\r?\n/g, " ");
+}
+
+const VERDICT_ICONS = { pass: "✅", fail: "❌", waived: "⚠️" };
+
+export function renderSummary({ report, drift, trustedFingerprint, certPath }) {
+  const lines = ["## certification-verify", ""];
+  const verifyOk = report?.ok === true;
+  // No drift outcome supplied (e.g. the selftest's direct render) means drift
+  // was not part of this evaluation; the workflow always passes one.
+  const driftOk =
+    drift === undefined ||
+    drift.result === "match" ||
+    drift.result === "allowed-drift";
+  lines.push(
+    `**Result:** ${verifyOk && driftOk ? "✅ certification verified" : "❌ certification rejected"}`,
+    "",
+  );
+  lines.push(`- Certification: \`${certPath}\``);
+  lines.push(
+    `- Trusted public key (from the base branch): fingerprint \`${trustedFingerprint}\``,
+  );
+
+  const payload = report?.payload;
+  if (payload !== undefined) {
+    const reviewer = payload.reviewer ?? {};
+    const model = reviewer.model !== undefined ? ` (${reviewer.model})` : "";
+    lines.push(
+      `- Reviewer: **${reviewer.kind}:${reviewer.id}**${model}`,
+      `- Tier: \`${payload.tier}\` · Commit: \`${payload.commit}\` (\`${payload.branch}\` → \`${payload.baseRef}\`)`,
+      `- Created: \`${payload.createdAt}\` (${formatAge(payload.createdAt)})`,
+    );
+  } else {
+    lines.push(
+      "- Reviewer/tier/commit: unavailable — the certification payload did not parse",
+    );
+  }
+  if (report !== undefined) {
+    lines.push(
+      report.bundle !== undefined
+        ? `- Bundle: re-hashed ${report.bundle.artifactCount} artifact(s); rollup completeness re-derived`
+        : "- Bundle: not supplied to verifier — promotion gate requires `evidence/bundle` and should fail before this point",
+    );
+  }
+
+  if (drift !== undefined) {
+    lines.push("", "### Commit drift", "");
+    lines.push(
+      `- ${driftOk ? "✅" : "❌"} \`${drift.result}\`: ${drift.detail}`,
+    );
+    if ((drift.driftPaths ?? []).length > 0) {
+      lines.push("", "<details><summary>Drifted paths</summary>", "");
+      for (const entry of drift.driftPaths) {
+        const bad = (drift.disallowedPaths ?? []).includes(entry);
+        lines.push(`- ${bad ? "❌" : "✅"} \`${entry}\``);
+      }
+      lines.push("", "</details>");
+    }
+  }
+
+  if ((payload?.verdicts ?? []).length > 0) {
+    lines.push("", "### Signed verdicts", "");
+    lines.push("| Subject | Verdict | Notes |", "| --- | --- | --- |");
+    for (const verdict of payload.verdicts) {
+      lines.push(
+        `| \`${escapeCell(verdict.subject)}\` | ${VERDICT_ICONS[verdict.verdict] ?? ""} ${escapeCell(verdict.verdict)} | ${escapeCell(verdict.notes ?? "")} |`,
+      );
+    }
+  }
+
+  const failures = report?.failures ?? [];
+  if (failures.length > 0) {
+    lines.push("", "### Verification failures", "");
+    for (const failure of failures) {
+      lines.push(`- \`${failure.code}\`: ${failure.message}`);
+    }
+  }
+  if (report === undefined) {
+    lines.push(
+      "",
+      "### Verification failures",
+      "",
+      "- verify report missing — `certify:verify` did not produce output; see the job log",
+    );
+  }
+  lines.push("");
+  return lines.join("\n");
+}
+
+export function renderAnnotations({ report, drift }) {
+  const lines = [];
+  for (const failure of report?.failures ?? []) {
+    lines.push(
+      `::error title=certification-verify (${failure.code})::${failure.message}`,
+    );
+  }
+  if (report === undefined) {
+    lines.push(
+      "::error title=certification-verify::certify:verify produced no report; see the job log",
+    );
+  }
+  if (
+    drift !== undefined &&
+    drift.result !== "match" &&
+    drift.result !== "allowed-drift"
+  ) {
+    lines.push(
+      `::error title=certification-verify (${drift.result})::${drift.detail}`,
+    );
+  }
+  return lines.join("\n");
+}
+
+function main() {
+  const argv = process.argv.slice(2);
+  const args = {};
+  for (let index = 0; index < argv.length; index += 1) {
+    const flag = argv[index];
+    const value = () => {
+      const next = argv[index + 1];
+      if (next === undefined) {
+        console.error(`${flag} requires a value`);
+        process.exit(2);
+      }
+      index += 1;
+      return next;
+    };
+    if (flag === "--report") args.reportPath = value();
+    else if (flag === "--drift") args.driftPath = value();
+    else if (flag === "--pubkey") args.pubkeyPath = value();
+    else if (flag === "--cert-path") args.certPath = value();
+    else if (flag === "--mode") args.mode = value();
+    else {
+      console.error(`unknown argument: ${flag}`);
+      process.exit(2);
+    }
+  }
+  if (args.mode !== "summary" && args.mode !== "annotations") {
+    console.error(
+      "Usage: render-check-summary.mjs --mode <summary|annotations> --report <verify.json> --drift <drift.json> --pubkey <pem> --cert-path <display path>",
+    );
+    process.exit(2);
+  }
+
+  const report = readJsonIfPossible(args.reportPath);
+  const drift = readJsonIfPossible(args.driftPath);
+  let trustedFingerprint = "unavailable";
+  if (args.pubkeyPath !== undefined) {
+    try {
+      trustedFingerprint = fingerprintPem(
+        readFileSync(args.pubkeyPath, "utf8"),
+      );
+    } catch {
+      trustedFingerprint = "unreadable public key";
+    }
+  }
+
+  const output =
+    args.mode === "summary"
+      ? renderSummary({
+          report,
+          drift,
+          trustedFingerprint,
+          certPath: args.certPath ?? "certification.json",
+        })
+      : renderAnnotations({ report, drift });
+  if (output.length > 0) process.stdout.write(`${output}\n`);
+}
+
+if (
+  process.argv[1] &&
+  import.meta.url === pathToFileURL(process.argv[1]).href
+) {
+  main();
+}

scripts/certification/selftest.mjs

@@ -0,0 +1,345 @@
+#!/usr/bin/env node
+/**
+ * End-to-end self-test of the certification gate plumbing, used by the
+ * `certification-verify-selftest` CI job and runnable locally. It exercises
+ * the REAL CLIs (`certify:keygen` → `bundle:create` → `certify:rollup` →
+ * `certify:sign` → `certify:verify`) against a throwaway keypair and a
+ * throwaway fixture repo — no reimplementation of any verification logic —
+ * and asserts:
+ *
+ *   1. a freshly signed certification verifies green (exit 0, ok:true),
+ *   2. a tampered payload fails `bad-signature`,
+ *   3. a missing committed bundle fails `bundle-tampered`,
+ *   4. a tampered bundle artifact fails `bundle-tampered`,
+ *   5. verification against a different trusted key fails `wrong-key`,
+ *   6. an out-of-window certification fails `stale`,
+ *   7. the commit-drift helper passes on match and rejects source drift,
+ *   8. the summary renderer produces markdown from the real report.
+ *
+ * The throwaway private key exists only in this process's memory and child
+ * env (ELIZA_CERT_SIGNING_KEY) — never on disk — matching the custody rules
+ * in .github/certification/README.md.
+ */
+
+import { execFileSync, spawnSync } from "node:child_process";
+import {
+  appendFileSync,
+  mkdirSync,
+  mkdtempSync,
+  readdirSync,
+  readFileSync,
+  rmSync,
+  writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..");
+const EVIDENCE_PKG = join(REPO_ROOT, "packages", "evidence");
+
+const failures = [];
+function step(name, ok, detail) {
+  const marker = ok ? "ok  " : "FAIL";
+  console.log(`[selftest] ${marker} ${name}${detail ? ` — ${detail}` : ""}`);
+  if (!ok) failures.push(name);
+}
+
+function runCli(scriptName, args, { env = {}, expectExit = 0 } = {}) {
+  const result = spawnSync(
+    "bun",
+    ["run", "--cwd", EVIDENCE_PKG, scriptName, "--", ...args],
+    { encoding: "utf8", env: { ...process.env, ...env } },
+  );
+  if (result.error) throw result.error;
+  const label = `${scriptName} ${args.join(" ")}`;
+  console.log(`[selftest] $ bun run --cwd packages/evidence ${label}`);
+  if (result.status !== expectExit) {
+    console.log(result.stdout);
+    console.error(result.stderr);
+  }
+  return result;
+}
+
+function failureCodes(stdout) {
+  return JSON.parse(stdout).failures.map((failure) => failure.code);
+}
+
+function git(cwd, args) {
+  return execFileSync("git", args, {
+    cwd,
+    encoding: "utf8",
+    stdio: ["ignore", "pipe", "pipe"],
+  }).trim();
+}
+
+const work = mkdtempSync(join(tmpdir(), "certification-selftest-"));
+try {
+  // -- keygen: throwaway trusted keypair + a second (wrong) keypair ---------
+  const pubkeyPath = join(work, "trusted-public-key.pem");
+  const keygen = runCli("certify:keygen", [
+    "--print-private-key",
+    "--pubkey-out",
+    pubkeyPath,
+  ]);
+  const privatePem = keygen.stdout.slice(
+    keygen.stdout.indexOf("-----BEGIN PRIVATE KEY-----"),
+  );
+  step("keygen produced a private key", privatePem.includes("PRIVATE KEY"));
+  const signingEnv = {
+    ELIZA_CERT_SIGNING_KEY: Buffer.from(privatePem).toString("base64"),
+  };
+  const wrongPubkeyPath = join(work, "wrong-public-key.pem");
+  runCli("certify:keygen", ["--pubkey-out", wrongPubkeyPath]);
+
+  // -- fixture repo: one lane result the e2e-recordings silo will ingest ----
+  const fixtureRepo = join(work, "fixture-repo");
+  mkdirSync(join(fixtureRepo, "e2e-recordings"), { recursive: true });
+  writeFileSync(
+    join(fixtureRepo, "e2e-recordings", "result.json"),
+    JSON.stringify({ passed: 3, failed: 0, skipped: 0 }),
+  );
+  writeFileSync(join(fixtureRepo, "src.ts"), "export {};\n");
+  git(fixtureRepo, ["init", "-b", "main"]);
+  git(fixtureRepo, ["config", "user.email", "selftest@example.invalid"]);
+  git(fixtureRepo, ["config", "user.name", "certification selftest"]);
+  git(fixtureRepo, ["add", "-A"]);
+  git(fixtureRepo, ["commit", "-m", "fixture tree"]);
+  const fixtureCommit = git(fixtureRepo, ["rev-parse", "HEAD"]);
+
+  // -- bundle + rollup + sign ------------------------------------------------
+  const runsDir = join(work, "runs");
+  const create = runCli("bundle:create", [
+    "--tier",
+    "full",
+    "--repo-root",
+    fixtureRepo,
+    "--out",
+    runsDir,
+  ]);
+  step("bundle:create succeeded", create.status === 0);
+  const bundleDir = join(runsDir, readdirSync(runsDir)[0]);
+
+  const verdictsPath = join(work, "verdicts.json");
+  const rollup = runCli("certify:rollup", [
+    "--bundle",
+    bundleDir,
+    "--out",
+    verdictsPath,
+  ]);
+  step("certify:rollup succeeded", rollup.status === 0);
+
+  const sign = runCli(
+    "certify:sign",
+    [
+      "--bundle",
+      bundleDir,
+      "--verdicts",
+      verdictsPath,
+      "--reviewer-id",
+      "certification-selftest",
+      "--reviewer-kind",
+      "agent",
+    ],
+    { env: signingEnv },
+  );
+  step("certify:sign succeeded", sign.status === 0);
+  const certPath = join(bundleDir, "certification.json");
+
+  const baseVerifyArgs = [
+    "--cert",
+    certPath,
+    "--pubkey",
+    pubkeyPath,
+    "--expected-commit",
+    fixtureCommit,
+    "--max-age-hours",
+    "72",
+    "--required-tier",
+    "full",
+    "--bundle",
+    bundleDir,
+    "--json",
+  ];
+
+  // -- 1. green path ----------------------------------------------------------
+  const green = runCli("certify:verify", baseVerifyArgs);
+  step("valid certification verifies (exit 0)", green.status === 0);
+  step("report.ok is true", JSON.parse(green.stdout).ok === true);
+  const greenReportPath = join(work, "verify-report.json");
+  writeFileSync(greenReportPath, green.stdout);
+
+  // -- 2. tampered payload → bad-signature ------------------------------------
+  const tamperedCertPath = join(work, "tampered-certification.json");
+  const cert = JSON.parse(readFileSync(certPath, "utf8"));
+  writeFileSync(
+    tamperedCertPath,
+    JSON.stringify({ ...cert, branch: "attacker-branch" }),
+  );
+  const tampered = runCli(
+    "certify:verify",
+    ["--cert", tamperedCertPath, "--pubkey", pubkeyPath, "--json"],
+    { expectExit: 1 },
+  );
+  step("tampered payload exits 1", tampered.status === 1);
+  step(
+    "tampered payload reports bad-signature",
+    failureCodes(tampered.stdout).includes("bad-signature"),
+    failureCodes(tampered.stdout).join(","),
+  );
+
+  // -- 3. missing committed bundle → bundle-tampered ---------------------------
+  const missingBundle = runCli(
+    "certify:verify",
+    [
+      "--cert",
+      certPath,
+      "--pubkey",
+      pubkeyPath,
+      "--expected-commit",
+      fixtureCommit,
+      "--max-age-hours",
+      "72",
+      "--required-tier",
+      "full",
+      "--bundle",
+      join(work, "missing-bundle"),
+      "--json",
+    ],
+    { expectExit: 1 },
+  );
+  step("missing committed bundle exits 1", missingBundle.status === 1);
+  step(
+    "missing committed bundle reports bundle-tampered",
+    failureCodes(missingBundle.stdout).includes("bundle-tampered"),
+    failureCodes(missingBundle.stdout).join(","),
+  );
+
+  // -- 4. tampered bundle artifact → bundle-tampered ---------------------------
+  appendFileSync(join(bundleDir, "lanes", "e2e", "result.json"), " ");
+  const bundleTampered = runCli("certify:verify", baseVerifyArgs, {
+    expectExit: 1,
+  });
+  step("tampered bundle exits 1", bundleTampered.status === 1);
+  step(
+    "tampered bundle reports bundle-tampered",
+    failureCodes(bundleTampered.stdout).includes("bundle-tampered"),
+    failureCodes(bundleTampered.stdout).join(","),
+  );
+
+  // -- 5. different trusted key → wrong-key ------------------------------------
+  const wrongKey = runCli(
+    "certify:verify",
+    ["--cert", certPath, "--pubkey", wrongPubkeyPath, "--json"],
+    { expectExit: 1 },
+  );
+  step("wrong trusted key exits 1", wrongKey.status === 1);
+  step(
+    "wrong trusted key reports wrong-key",
+    failureCodes(wrongKey.stdout).includes("wrong-key"),
+    failureCodes(wrongKey.stdout).join(","),
+  );
+
+  // -- 6. out-of-window certification → stale ----------------------------------
+  await new Promise((resolveSleep) => setTimeout(resolveSleep, 1500));
+  const stale = runCli(
+    "certify:verify",
+    [
+      "--cert",
+      certPath,
+      "--pubkey",
+      pubkeyPath,
+      // ~0.36s window: the 1.5s sleep above guarantees the cert is outside it.
+      "--max-age-hours",
+      "0.0001",
+      "--json",
+    ],
+    { expectExit: 1 },
+  );
+  step("out-of-window certification exits 1", stale.status === 1);
+  step(
+    "out-of-window certification reports stale",
+    failureCodes(stale.stdout).includes("stale"),
+    failureCodes(stale.stdout).join(","),
+  );
+
+  // -- 7. drift helper: match passes, source drift fails ------------------------
+  const driftScript = join(
+    REPO_ROOT,
+    "scripts",
+    "certification",
+    "check-commit-drift.mjs",
+  );
+  const driftMatch = spawnSync(
+    "node",
+    [
+      driftScript,
+      "--cert",
+      certPath,
+      "--head",
+      fixtureCommit,
+      "--repo",
+      fixtureRepo,
+    ],
+    { encoding: "utf8" },
+  );
+  step("drift helper passes on commit match", driftMatch.status === 0);
+
+  writeFileSync(join(fixtureRepo, "src.ts"), "export const drift = 1;\n");
+  git(fixtureRepo, ["add", "-A"]);
+  git(fixtureRepo, ["commit", "-m", "source drift"]);
+  const driftedHead = git(fixtureRepo, ["rev-parse", "HEAD"]);
+  const driftBad = spawnSync(
+    "node",
+    [
+      driftScript,
+      "--cert",
+      certPath,
+      "--head",
+      driftedHead,
+      "--repo",
+      fixtureRepo,
+    ],
+    { encoding: "utf8" },
+  );
+  step("drift helper rejects source drift", driftBad.status === 1);
+
+  // -- 8. renderer produces markdown from the real report -----------------------
+  const renderScript = join(
+    REPO_ROOT,
+    "scripts",
+    "certification",
+    "render-check-summary.mjs",
+  );
+  const summary = spawnSync(
+    "node",
+    [
+      renderScript,
+      "--mode",
+      "summary",
+      "--report",
+      greenReportPath,
+      "--pubkey",
+      pubkeyPath,
+      "--cert-path",
+      "selftest certification.json",
+    ],
+    { encoding: "utf8" },
+  );
+  step(
+    "renderer emits the verdict table",
+    summary.status === 0 &&
+      summary.stdout.includes("certification verified") &&
+      summary.stdout.includes("lane:e2e"),
+  );
+  console.log("\n[selftest] rendered summary preview:\n");
+  console.log(summary.stdout);
+} finally {
+  rmSync(work, { recursive: true, force: true });
+}
+
+if (failures.length > 0) {
+  console.error(`\n[selftest] FAILED: ${failures.join("; ")}`);
+  process.exit(1);
+}
+console.log("\n[selftest] all certification gate plumbing checks passed");

scripts/check-even-research-audit.mjs

@@ -0,0 +1,218 @@
+#!/usr/bin/env node
+/**
+ * CI gate that the Even Realities smartglasses upstream research is complete:
+ * every requested reference repo is checked out under research/even-realities,
+ * the upstream-audit and smartglasses docs exist, and the implemented facewear
+ * surface files are present. Exits non-zero on any gap. `--self-test` exercises
+ * the gate logic against synthetic inputs.
+ */
+import { existsSync, readFileSync } from "node:fs";
+import { resolve } from "node:path";
+
+const repoRoot = resolve(import.meta.dirname, "..");
+const researchRoot = "research/even-realities";
+const upstreamAuditPath =
+  "plugins/plugin-facewear/docs/smartglasses-upstream-audit.md";
+const smartglassesDocsPath = "plugins/plugin-facewear/docs/smartglasses.md";
+const requestedRepos = [
+  ["fabioglimb/even-toolkit", "even-toolkit"],
+  ["BxNxM/even-dev", "even-dev"],
+  ["emingenc/even_glasses", "even_glasses"],
+  ["binarythinktank/eveng1_python_sdk", "eveng1_python_sdk"],
+  ["meyskens/fahrplan", "fahrplan"],
+  ["nickustinov/weather-even-g2", "weather-even-g2"],
+  ["jappyjan/even-realities", "even-realities"],
+  ["emingenc/g1_flutter_blue_plus", "g1_flutter_blue_plus"],
+  ["nickustinov/tesla-even-g2", "tesla-even-g2"],
+  ["galfaroth/awesome-even-realities-g1", "awesome-even-realities-g1"],
+  ["even-realities/EvenDemoApp", "EvenDemoApp"],
+  ["Mentra-Community/MentraOS", "MentraOS"],
+];
+const implementedSurfacePaths = [
+  "plugins/plugin-facewear/src/protocol/smartglasses.ts",
+  "plugins/plugin-facewear/src/services/smartglasses-service.ts",
+  "plugins/plugin-facewear/src/actions/facewear-control.ts",
+  "plugins/plugin-facewear/src/providers/smartglasses-status.ts",
+  "plugins/plugin-facewear/src/transport/web-bluetooth.ts",
+  "plugins/plugin-facewear/src/transport/noble.ts",
+  "plugins/plugin-facewear/src/transport/even-bridge.ts",
+  "packages/examples/smartglasses/hardware-smoke.ts",
+  "packages/examples/smartglasses/noble-hardware-smoke.ts",
+  "packages/examples/smartglasses/hardware-evidence.ts",
+  "packages/examples/smartglasses/evenhub-smoke.ts",
+  "packages/examples/smartglasses/simulator-automation-smoke.ts",
+];
+
+if (process.argv.includes("--self-test")) {
+  runSelfTest();
+  process.exit(0);
+}
+
+const failures = [];
+const gitignore = read(".gitignore");
+const audit = read(upstreamAuditPath);
+const smartglassesDocs = read(smartglassesDocsPath);
+
+failures.push(...gitignoreFailures(gitignore));
+failures.push(...implementedSurfaceFailures(audit));
+failures.push(...docLinkFailures(smartglassesDocs));
+
+if (!existsSync(resolve(repoRoot, researchRoot))) {
+  failures.push(`${researchRoot}: missing research root`);
+}
+
+for (const [slug, dir] of requestedRepos) {
+  const relPath = `${researchRoot}/${dir}`;
+  if (!existsSync(resolve(repoRoot, relPath))) {
+    failures.push(`${relPath}: missing checkout for ${slug}`);
+    continue;
+  }
+  if (!existsSync(resolve(repoRoot, relPath, ".git"))) {
+    failures.push(`${relPath}: checkout for ${slug} has no .git metadata`);
+  }
+  failures.push(...auditRowFailures(audit, slug, relPath));
+}
+
+if (failures.length > 0) {
+  console.error(JSON.stringify({ ok: false, failures }, null, 2));
+  process.exit(1);
+}
+
+console.log(
+  JSON.stringify(
+    {
+      ok: true,
+      researchRoot,
+      repositories: requestedRepos.length,
+      audit: upstreamAuditPath,
+    },
+    null,
+    2,
+  ),
+);
+
+function read(relPath) {
+  return readFileSync(resolve(repoRoot, relPath), "utf8");
+}
+
+function gitignoreFailures(source) {
+  return source.includes(`${researchRoot}/`)
+    ? []
+    : [`.gitignore: missing ${researchRoot}/`];
+}
+
+function auditRowFailures(auditSource, slug, relPath) {
+  const result = [];
+  if (!auditSource.includes(`\`${slug}\``)) {
+    result.push(`${upstreamAuditPath}: missing source row for ${slug}`);
+  }
+  if (!auditSource.includes(`\`${relPath}\``)) {
+    result.push(`${upstreamAuditPath}: missing checkout path for ${relPath}`);
+  }
+  return result;
+}
+
+function implementedSurfaceFailures(auditSource) {
+  const result = [];
+  for (const relPath of implementedSurfacePaths) {
+    if (!existsSync(resolve(repoRoot, relPath))) {
+      result.push(`${relPath}: implemented surface path is missing`);
+    }
+  }
+  for (const expected of [
+    "src/providers/smartglasses-status.ts",
+    "src/transport/even-bridge.ts",
+    "src/transport/web-bluetooth.ts",
+    "src/transport/noble.ts",
+    "packages/examples/smartglasses/hardware-evidence.ts",
+    "packages/examples/smartglasses/simulator-automation-smoke.ts",
+  ]) {
+    if (!auditSource.includes(expected)) {
+      result.push(
+        `${upstreamAuditPath}: missing implemented surface reference ${expected}`,
+      );
+    }
+  }
+  if (auditSource.includes("src/providers/status.ts")) {
+    result.push(
+      `${upstreamAuditPath}: stale provider path src/providers/status.ts`,
+    );
+  }
+  return result;
+}
+
+function docLinkFailures(docsSource) {
+  const result = [];
+  if (!docsSource.includes("docs/smartglasses-upstream-audit.md")) {
+    result.push(
+      `${smartglassesDocsPath}: missing link to docs/smartglasses-upstream-audit.md`,
+    );
+  }
+  if (docsSource.includes("docs/upstream-audit.md")) {
+    result.push(
+      `${smartglassesDocsPath}: stale link to docs/upstream-audit.md`,
+    );
+  }
+  return result;
+}
+
+function runSelfTest() {
+  const [slug, dir] = requestedRepos[0];
+  const relPath = `${researchRoot}/${dir}`;
+  const failures = [];
+
+  const ignored = gitignoreFailures(`${researchRoot}/\n`);
+  if (ignored.length > 0) failures.push(`valid gitignore failed: ${ignored}`);
+
+  const missingIgnore = gitignoreFailures("");
+  if (!missingIgnore.some((failure) => failure.includes(researchRoot))) {
+    failures.push("missing gitignore fixture was not detected");
+  }
+
+  const validAudit = `| \`${slug}\` | \`${relPath}\` | files | carried |`;
+  const validAuditFailures = auditRowFailures(validAudit, slug, relPath);
+  if (validAuditFailures.length > 0) {
+    failures.push(`valid audit row failed: ${validAuditFailures}`);
+  }
+
+  const missingSource = auditRowFailures(
+    `| missing/source | \`${relPath}\` | files | carried |`,
+    slug,
+    relPath,
+  );
+  if (
+    !missingSource.some((failure) => failure.includes("missing source row"))
+  ) {
+    failures.push("missing source row fixture was not detected");
+  }
+
+  const missingPath = auditRowFailures(
+    `| \`${slug}\` | missing/path | files | carried |`,
+    slug,
+    relPath,
+  );
+  if (
+    !missingPath.some((failure) => failure.includes("missing checkout path"))
+  ) {
+    failures.push("missing checkout path fixture was not detected");
+  }
+
+  const staleSurface = implementedSurfaceFailures("src/providers/status.ts");
+  if (
+    !staleSurface.some((failure) => failure.includes("stale provider path"))
+  ) {
+    failures.push("stale provider fixture was not detected");
+  }
+
+  const staleLink = docLinkFailures("docs/upstream-audit.md");
+  if (!staleLink.some((failure) => failure.includes("stale link"))) {
+    failures.push("stale doc link fixture was not detected");
+  }
+
+  if (failures.length > 0) {
+    console.error(JSON.stringify({ ok: false, failures }, null, 2));
+    process.exit(1);
+  }
+
+  console.log(JSON.stringify({ ok: true, fixtures: 6 }, null, 2));
+}

scripts/check-markdown-links.mjs

@@ -0,0 +1,177 @@
+#!/usr/bin/env node
+
+import { execFileSync } from "node:child_process";
+import { existsSync, readFileSync, statSync } from "node:fs";
+import path from "node:path";
+
+const ROOT = path.resolve(import.meta.dirname, "..");
+const RETIRED_REPO_EVIDENCE_PREFIX = `${[".github", ["issue", "evidence"].join("-")].join("/")}/`;
+
+// Historical evidence, fixture, vendored, and prototype documentation contains
+// intentionally stale sample paths. Keep the first gate on maintained
+// contributor-facing docs; shrink this list as those trees are cleaned up.
+const EXCLUDED_PREFIXES = [
+  RETIRED_REPO_EVIDENCE_PREFIX,
+  "docs/",
+  "packages/app-core/",
+  "packages/benchmarks/",
+  "packages/cloud/",
+  "packages/docs/apps/",
+  "packages/docs/build-and-release.md",
+  "packages/docs/connectors/",
+  "packages/docs/dashboard/",
+  "packages/docs/electrobun-startup.md",
+  "packages/docs/plugin-resolution-and-node-path.md",
+  "packages/docs/plugins/",
+  "packages/docs/runtime/",
+  "packages/elizaos/src/commands/",
+  "packages/examples/",
+  "packages/feed/",
+  "packages/security/docs/",
+  "packages/skills/",
+  "packages/training/",
+  "packages/ui/src/services/local-inference/",
+  "packages/app-core/test/",
+  "plugins/plugin-agent-orchestrator/docs/",
+  "plugins/plugin-computeruse/",
+  "plugins/plugin-local-inference/",
+  "plugins/plugin-wallet/src/chains/solana/",
+  "plugins/plugin-xai/",
+];
+
+const EXCLUDED_NAMES = new Set(["CHANGELOG.md"]);
+
+function trackedMarkdownFiles() {
+  return execFileSync("git", ["ls-files", "*.md"], {
+    cwd: ROOT,
+    encoding: "utf8",
+  })
+    .split("\n")
+    .filter(Boolean)
+    .filter((file) => !EXCLUDED_NAMES.has(path.basename(file)))
+    .filter(
+      (file) => !EXCLUDED_PREFIXES.some((prefix) => file.startsWith(prefix)),
+    );
+}
+
+function stripAnchor(href) {
+  const hashIndex = href.indexOf("#");
+  return hashIndex === -1 ? href : href.slice(0, hashIndex);
+}
+
+function candidateTargets(sourceFile, href) {
+  const withoutQuery = href.split("?")[0];
+  if (withoutQuery.startsWith("/")) {
+    const docsTarget = path.join(
+      ROOT,
+      "packages/docs",
+      withoutQuery.replace(/^\/+/, ""),
+    );
+    return [
+      path.join(ROOT, withoutQuery),
+      docsTarget,
+      `${docsTarget}.md`,
+      `${docsTarget}.mdx`,
+      path.join(docsTarget, "README.md"),
+      path.join(docsTarget, "index.md"),
+      path.join(docsTarget, "index.mdx"),
+    ];
+  }
+
+  const target = path.resolve(ROOT, path.dirname(sourceFile), withoutQuery);
+  return [
+    target,
+    `${target}.md`,
+    `${target}.mdx`,
+    path.join(target, "README.md"),
+    path.join(target, "index.md"),
+    path.join(target, "index.mdx"),
+  ];
+}
+
+function isRelativeLink(href) {
+  return (
+    href &&
+    !href.startsWith("#") &&
+    !href.startsWith("mailto:") &&
+    !href.startsWith("tel:") &&
+    !href.startsWith("http://") &&
+    !href.startsWith("https://") &&
+    !href.startsWith("ftp://") &&
+    !href.startsWith("file://") &&
+    !href.startsWith("data:")
+  );
+}
+
+function decodeHref(href) {
+  try {
+    return decodeURIComponent(href);
+  } catch {
+    return href;
+  }
+}
+
+function targetExists(sourceFile, rawHref) {
+  const href = decodeHref(stripAnchor(rawHref).trim());
+  if (!isRelativeLink(href)) return true;
+
+  for (const target of candidateTargets(sourceFile, href)) {
+    if (!target.startsWith(ROOT)) continue;
+    if (!existsSync(target)) continue;
+    if (statSync(target).isDirectory()) {
+      if (
+        existsSync(path.join(target, "README.md")) ||
+        existsSync(path.join(target, "index.md")) ||
+        existsSync(path.join(target, "index.mdx"))
+      ) {
+        return true;
+      }
+      continue;
+    }
+    return true;
+  }
+  return false;
+}
+
+function stripCode(markdown) {
+  return markdown
+    .replace(/```[\s\S]*?```/g, "")
+    .replace(/~~~[\s\S]*?~~~/g, "")
+    .replace(/`[^`\n]+`/g, "");
+}
+
+function markdownLinks(markdown) {
+  const links = [];
+  const searchable = stripCode(markdown);
+  const inlinePattern = /!?\[[^\]\n]*\]\(([^)\s]+)(?:\s+"[^"]*")?\)/g;
+  const referencePattern = /^\s*\[[^\]\n]+]:\s*(\S+)/gm;
+  for (const match of searchable.matchAll(inlinePattern)) {
+    links.push(match[1]);
+  }
+  for (const match of searchable.matchAll(referencePattern)) {
+    links.push(match[1]);
+  }
+  return links;
+}
+
+const failures = [];
+for (const file of trackedMarkdownFiles()) {
+  const markdown = readFileSync(path.join(ROOT, file), "utf8");
+  for (const href of markdownLinks(markdown)) {
+    if (!targetExists(file, href)) {
+      failures.push(`${file}: missing relative link target ${href}`);
+    }
+  }
+}
+
+if (failures.length > 0) {
+  console.error(
+    `[check-markdown-links] ${failures.length} missing relative link target(s):`,
+  );
+  for (const failure of failures) {
+    console.error(`- ${failure}`);
+  }
+  process.exit(1);
+}
+
+console.log("[check-markdown-links] PASS: relative Markdown links resolve.");

scripts/check-pr-evidence.mjs

@@ -0,0 +1,627 @@
+#!/usr/bin/env node
+/**
+ * Mechanical PR evidence gate for the evidence rows in the pull request
+ * template. The template keeps stable HTML markers above each required row so
+ * this checker can ignore row prose churn while still failing closed when a row
+ * is blank, checkbox-only, or removed.
+ */
+
+import { readFileSync } from "node:fs";
+import { pathToFileURL } from "node:url";
+
+export const REQUIRED_EVIDENCE_ROWS = [
+  { id: "before-screenshots", label: "Before screenshots" },
+  { id: "after-screenshots", label: "After screenshots" },
+  { id: "walkthrough-video", label: "Walkthrough video" },
+  { id: "backend-logs", label: "Backend logs" },
+  { id: "frontend-logs", label: "Frontend console/network logs" },
+  { id: "llm-trajectory", label: "Real-LLM trajectory" },
+  { id: "domain-artifacts", label: "Domain artifacts" },
+];
+
+export const SURFACE_EVIDENCE_LABELS = ["ui", "frontend", "native"];
+export const SURFACE_ARTIFACT_ROW_IDS = [
+  "before-screenshots",
+  "after-screenshots",
+  "walkthrough-video",
+];
+export const SURFACE_OCR_EVIDENCE_ROW = {
+  id: "ocr-review",
+  label: "OCR visual text review",
+};
+
+/**
+ * A changed file forces surface artifacts when it is a rendered-UI source file
+ * — labels are advisory and agents routinely omit them, so the gate cannot rely
+ * on them alone. Detection is deliberately narrow: a visual EXTENSION (`.tsx`,
+ * CSS family, `.svg`, `.html`, `.vue`) under a UI-bearing PACKAGE, excluding
+ * test/story/fixture files that render nothing a user sees. This is why editing
+ * a real component in `packages/ui` or `packages/app` demands screenshots while
+ * editing its `*.test.tsx` or `*.stories.tsx` does not.
+ */
+const SURFACE_PATH_RE =
+  /(^|\/)(packages\/(app|ui|tui|homepage)|apps\/app|packages\/cloud\/frontend|packages\/os\/landing)\//i;
+const SURFACE_VISUAL_EXT_RE = /\.(tsx|jsx|css|scss|sass|less|svg|html|vue)$/i;
+const SURFACE_NON_VISUAL_RE =
+  /(\.(test|spec|stories|story|bench)\.|\.d\.ts$|(^|\/)(__tests__|__e2e__|__mocks__|__fixtures__|test|tests|e2e|stories)\/)/i;
+
+/**
+ * True when any changed file is a rendered-UI source file (see `SURFACE_PATH_RE`
+ * rationale). Backslash paths from a Windows runner are normalized so the same
+ * diff classifies identically on either OS.
+ */
+export function requiresSurfaceArtifactsFromFiles(files) {
+  return surfaceFiles(files).length > 0;
+}
+
+/** The rendered-UI source files within a changed-file list. */
+export function surfaceFiles(files) {
+  return parseChangedFiles(files)
+    .map((raw) => raw.replaceAll("\\", "/"))
+    .filter(
+      (file) =>
+        SURFACE_PATH_RE.test(file) &&
+        SURFACE_VISUAL_EXT_RE.test(file) &&
+        !SURFACE_NON_VISUAL_RE.test(file),
+    );
+}
+
+/**
+ * A "before" screenshot is impossible when the ENTIRE touched UI surface is new
+ * — every rendered-UI file in the diff was ADDED, none modified. In that case
+ * the before-screenshots row may be an honest `N/A - <reason>` instead of
+ * media. Determined mechanically from the added-files list (`git diff
+ * --diff-filter=A`), never from the reason text, so it cannot be gamed by
+ * prose.
+ */
+export function beforeScreenshotImpossible(changedFiles, addedFiles) {
+  const surface = surfaceFiles(changedFiles);
+  if (surface.length === 0) return false;
+  const added = new Set(surfaceFiles(addedFiles));
+  return surface.every((file) => added.has(file));
+}
+const OCR_EVIDENCE_RE =
+  /\bOCR\b|ocr-triage|mvp:visual-verify|audit:app:verify|tesseract|text readout/i;
+
+const MARKER_RE = /<!--\s*evidence-row:([a-z0-9-]+)\s*-->/gi;
+const RETIRED_REPO_EVIDENCE_PATH = [
+  ".github",
+  ["issue", "evidence"].join("-"),
+].join("/");
+const RETIRED_REPO_EVIDENCE_RE = new RegExp(
+  `${RETIRED_REPO_EVIDENCE_PATH.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}\\/\\S+`,
+  "i",
+);
+
+export function parseLabels(value) {
+  if (Array.isArray(value)) {
+    return value
+      .flatMap((label) => parseLabels(label))
+      .filter((label, index, labels) => labels.indexOf(label) === index);
+  }
+  return String(value ?? "")
+    .split(/[\n,]/)
+    .map((label) => label.trim().toLowerCase())
+    .filter(Boolean);
+}
+
+export function requiresSurfaceArtifacts(labels) {
+  const labelSet = new Set(parseLabels(labels));
+  return SURFACE_EVIDENCE_LABELS.some((label) => labelSet.has(label));
+}
+
+export function hasOcrEvidenceReference(rows) {
+  for (const rowText of rows.values()) {
+    // OCR proof must reference real evidence, not a page link: either an
+    // actual media artifact or a linked report/log file next to OCR keywords.
+    if (
+      OCR_EVIDENCE_RE.test(rowText) &&
+      (hasVisualArtifactReference(rowText) ||
+        hasEvidenceFileReference(rowText))
+    ) {
+      return true;
+    }
+  }
+  return false;
+}
+
+// Linked non-media evidence file (an OCR report/JSON/log) — still stricter
+// than "any URL": the link target must look like a file, not a web page.
+const EVIDENCE_FILE_RE = /\.(json|txt|log|csv|md)(\?\S*)?(\s|$|\)|"|')/i;
+
+export function hasEvidenceFileReference(text) {
+  const value = String(text ?? "");
+  return (
+    (EVIDENCE_FILE_RE.test(value) || GITHUB_ATTACHMENT_RE.test(value)) &&
+    !RETIRED_REPO_EVIDENCE_RE.test(value)
+  );
+}
+
+export function hasNaWithReason(text) {
+  const match = text.match(/\bN\/?A\b\s*[-:\u2013\u2014]\s*(\S[\s\S]*?)$/im);
+  if (!match) return false;
+  const reason = match[1].trim();
+  if (reason.length < 3) return false;
+  const stripped = reason.replace(/[`*_]+/g, "").trim();
+  return !/^<[^>]*>[.\s]*$/.test(stripped);
+}
+
+export function hasArtifactReference(text) {
+  const markdownLinks = [
+    ...String(text ?? "").matchAll(/\[[^\]]+\]\(\s*(\S+)\s*\)/g),
+  ];
+  if (markdownLinks.some((match) => !RETIRED_REPO_EVIDENCE_RE.test(match[1]))) {
+    return true;
+  }
+  if (/https?:\/\/\S+/i.test(text)) return true;
+  if (
+    /user-images\.githubusercontent\.com|github\.com\/[^)\s]+\/assets\//i.test(
+      text,
+    )
+  ) {
+    return true;
+  }
+  return false;
+}
+
+// A GitHub-uploaded attachment (drag-and-drop) lands on one of these hosts;
+// these URLs are the unforgeable signal that a real image/video is embedded.
+const GITHUB_ATTACHMENT_RE =
+  /(https?:\/\/)?(github\.com\/user-attachments\/assets\/|user-images\.githubusercontent\.com\/|github\.com\/[^/\s)]+\/[^/\s)]+\/assets\/)/i;
+// A URL/path whose tail is an image or video file — a directly-linked media file.
+const MEDIA_EXT_RE =
+  /\.(png|jpe?g|gif|webp|apng|avif|bmp|svg|mp4|mov|webm|m4v|ogg)(\?\S*)?(\s|$|\)|"|')/i;
+
+/**
+ * Strict media check for the VISUAL evidence rows (screenshots, walkthrough
+ * video, OCR readout). Unlike `hasArtifactReference`, a bare link to a page —
+ * the PR itself, a `/checks` tab, a commit, a job log — does NOT count: those
+ * are how an author games the loose check while attaching no pixels. Only a
+ * real embedded/linked image or video satisfies it: a GitHub attachment-host
+ * URL, a markdown image embed `![](…)`, an `<img>`/`<video>` tag, or a URL/path
+ * ending in a known media extension. This is the difference between "here is a
+ * link" and "here is the picture".
+ */
+export function hasVisualArtifactReference(text) {
+  const value = String(text ?? "");
+  if (GITHUB_ATTACHMENT_RE.test(value)) return true;
+  if (/!\[[^\]]*\]\(\s*\S+\s*\)/.test(value)) return true; // ![alt](url) embed
+  if (/<(img|video|source|picture)\b[^>]*>/i.test(value)) return true;
+  if (MEDIA_EXT_RE.test(value) && !RETIRED_REPO_EVIDENCE_RE.test(value)) {
+    return true;
+  }
+  return false;
+}
+
+export function parseChangedFiles(value) {
+  if (Array.isArray(value))
+    return value.flatMap((entry) => parseChangedFiles(entry));
+  return String(value ?? "")
+    .split(/\r?\n/)
+    .map((entry) => entry.trim())
+    .filter(Boolean);
+}
+
+export function findRetiredRepoEvidenceFiles(files) {
+  return parseChangedFiles(files).filter((file) =>
+    file.replaceAll("\\", "/").startsWith(`${RETIRED_REPO_EVIDENCE_PATH}/`),
+  );
+}
+
+export function isChecked(rowText) {
+  return /^\s*[-*]\s*\[\s*[xX]\s*\]/m.test(rowText);
+}
+
+export function isRowSatisfied(rowText) {
+  return hasNaWithReason(rowText) || hasArtifactReference(rowText);
+}
+
+export function isRowSatisfiedForContext(
+  rowText,
+  { artifactRequired = false } = {},
+) {
+  if (artifactRequired) return hasArtifactReference(rowText);
+  return isRowSatisfied(rowText);
+}
+
+export function boundRowBlock(block) {
+  const lines = block.split(/\r?\n/);
+  const out = [];
+  let started = false;
+  for (const line of lines) {
+    if (!started) {
+      if (line.trim() === "") continue;
+      started = true;
+      out.push(line);
+      continue;
+    }
+
+    const trimmed = line.trim();
+    if (trimmed === "") break;
+    if (/^#/.test(trimmed)) break;
+    if (/<!--\s*evidence-row:/i.test(trimmed)) break;
+    if (/^[-*]\s/.test(line) && !/^\s/.test(line)) break;
+    out.push(line);
+  }
+  return out.join("\n").trim();
+}
+
+export function extractEvidenceRows(body) {
+  const source = body ?? "";
+  const rows = new Map();
+  const matches = [];
+  for (const match of source.matchAll(MARKER_RE)) {
+    const start = match.index;
+    matches.push({
+      id: match[1].toLowerCase(),
+      start,
+      end: start + match[0].length,
+    });
+  }
+
+  for (let i = 0; i < matches.length; i += 1) {
+    const current = matches[i];
+    const next = matches[i + 1];
+    const sliceEnd = next ? next.start : source.length;
+    const rowText = boundRowBlock(source.slice(current.end, sliceEnd));
+    if (!rows.has(current.id) || rowText.length > 0) {
+      rows.set(current.id, rowText);
+    }
+  }
+  return rows;
+}
+
+export function evaluatePrEvidence(
+  body,
+  requiredRows = REQUIRED_EVIDENCE_ROWS,
+  options = {},
+) {
+  const rows = extractEvidenceRows(body ?? "");
+  // When a changed-file list is available, path detection is the sole surface
+  // trigger: the auto-labeler applies `ui` to ANY packages/ui path, so the
+  // label alone forces screenshots onto non-visual .ts changes. The label
+  // trigger survives only for label-only invocations (no file list), where it
+  // is the best signal available.
+  const surfaceArtifactsRequired =
+    parseChangedFiles(options.changedFiles).length > 0
+      ? requiresSurfaceArtifactsFromFiles(options.changedFiles)
+      : requiresSurfaceArtifacts(options.labels);
+  // A wholly-new surface (every touched UI file was ADDED) has no "before"
+  // state to photograph; that one row may be N/A-with-reason.
+  const beforeNaAllowed = beforeScreenshotImpossible(
+    options.changedFiles,
+    options.addedFiles,
+  );
+  const findings = requiredRows.map(({ id, label }) => {
+    if (!rows.has(id)) return { id, label, status: "missing" };
+    const rowText = rows.get(id);
+    if (rowText.length === 0) return { id, label, status: "blank" };
+    const artifactRequired =
+      surfaceArtifactsRequired &&
+      SURFACE_ARTIFACT_ROW_IDS.includes(id) &&
+      !(id === "before-screenshots" && beforeNaAllowed && hasNaWithReason(rowText));
+    // Visual rows on a surface PR demand REAL media (attachment/embed/media
+    // URL) — a link to the PR page or a /checks tab is not a screenshot.
+    if (artifactRequired && !hasVisualArtifactReference(rowText)) {
+      return { id, label, status: "artifact-required" };
+    }
+    return {
+      id,
+      label,
+      status:
+        artifactRequired || isRowSatisfied(rowText) ? "ok" : "blank",
+    };
+  });
+  if (surfaceArtifactsRequired && !hasOcrEvidenceReference(rows)) {
+    findings.push({ ...SURFACE_OCR_EVIDENCE_ROW, status: "ocr-required" });
+  }
+
+  return {
+    ok: findings.every((finding) => finding.status === "ok"),
+    findings,
+  };
+}
+
+function readBody(args) {
+  const idx = args.indexOf("--body-file");
+  if (idx !== -1) {
+    const file = args[idx + 1];
+    if (!file) {
+      console.error("--body-file requires a path argument");
+      process.exit(2);
+    }
+    return readFileSync(file, "utf8");
+  }
+
+  try {
+    return readFileSync(0, "utf8");
+  } catch {
+    return "";
+  }
+}
+
+function readFileListArg(args, flag) {
+  const idx = args.indexOf(flag);
+  if (idx === -1) return [];
+
+  const file = args[idx + 1];
+  if (!file) {
+    console.error(`${flag} requires a path argument`);
+    process.exit(2);
+  }
+  return parseChangedFiles(readFileSync(file, "utf8"));
+}
+
+function usage() {
+  console.log(`Usage: node scripts/check-pr-evidence.mjs [options]
+
+Options:
+  --body-file <path>  Read the PR body from a file (default: stdin).
+  --labels <labels>   Comma-separated PR labels; ui/frontend/native require
+                      concrete screenshot/video artifacts and linked OCR proof.
+  --changed-files-file <path>
+                      Reject committed files under retired repo evidence paths,
+                      AND require concrete screenshot/video/OCR artifacts when a
+                      rendered-UI source file is in the diff (labels optional).
+  --added-files-file <path>
+                      Newline list of ADDED files (git diff --diff-filter=A).
+                      When every rendered-UI file in the diff is newly added,
+                      the before-screenshots row may be 'N/A - <reason>' (a
+                      brand-new surface has no before state).
+  --json              Print machine-readable findings JSON.
+  --self-test         Run the planted-fixture self-check.
+  --help, -h          Show this help.
+`);
+}
+
+function buildFixtureBody(overrides = {}) {
+  const defaults = {
+    "before-screenshots":
+      "- [ ] Before screenshots `N/A - backend-only change, no UI surface`.",
+    "after-screenshots":
+      "- [ ] After screenshots `N/A - backend-only change, no UI surface`.",
+    "walkthrough-video":
+      "- [x] A video walkthrough: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000000",
+    "backend-logs":
+      "- [ ] Backend logs: [backend.txt](https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000001)",
+    "frontend-logs": "- [ ] Frontend logs `N/A - no frontend change`.",
+    "llm-trajectory":
+      "- [ ] Real-LLM trajectory: [report](https://example.com/report.json)",
+    "domain-artifacts":
+      "- [ ] Domain artifacts: OCR report https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000007",
+  };
+  const merged = { ...defaults, ...overrides };
+  return REQUIRED_EVIDENCE_ROWS.map(
+    ({ id }) => `<!-- evidence-row:${id} -->\n${merged[id] ?? ""}`,
+  ).join("\n\n");
+}
+
+function runSelfTest() {
+  const failures = [];
+
+  {
+    const { ok } = evaluatePrEvidence(buildFixtureBody());
+    if (!ok) failures.push("all-filled fixture should pass");
+  }
+
+  {
+    const { ok, findings } = evaluatePrEvidence(
+      buildFixtureBody({
+        "backend-logs":
+          "- [ ] Backend logs show the real code path firing end to end, or are marked `N/A - <reason>`.",
+      }),
+    );
+    const blank = findings.find((finding) => finding.id === "backend-logs");
+    if (ok) failures.push("blank fixture should fail");
+    if (blank?.status !== "blank") {
+      failures.push("blank row should be reported blank");
+    }
+  }
+
+  {
+    const { ok, findings } = evaluatePrEvidence(
+      buildFixtureBody({
+        "backend-logs": "- [x] Backend logs attached.",
+      }),
+    );
+    const blank = findings.find((finding) => finding.id === "backend-logs");
+    if (ok) failures.push("checked-without-artifact fixture should fail");
+    if (blank?.status !== "blank") {
+      failures.push("checked-without-artifact row should be reported blank");
+    }
+  }
+
+  {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    const { ok } = evaluatePrEvidence(body);
+    if (!ok) failures.push("all-N/A-with-reason fixture should pass");
+  }
+
+  {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      labels: "ui",
+    });
+    if (ok) failures.push("ui-labeled all-N/A fixture should fail");
+    const screenshots = findings.filter((finding) =>
+      SURFACE_ARTIFACT_ROW_IDS.includes(finding.id),
+    );
+    if (screenshots.some((finding) => finding.status !== "artifact-required")) {
+      failures.push(
+        "ui-labeled screenshot/video rows should require artifacts",
+      );
+    }
+    const ocr = findings.find((finding) => finding.id === "ocr-review");
+    if (ocr?.status !== "ocr-required") {
+      failures.push("ui-labeled evidence should require OCR proof");
+    }
+  }
+
+  {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      changedFiles: ["packages/ui/src/components/Foo.tsx"],
+    });
+    if (ok) {
+      failures.push("UI-file diff with all-N/A rows should fail (no labels)");
+    }
+    if (
+      findings.find((finding) => finding.id === "before-screenshots")
+        ?.status !== "artifact-required"
+    ) {
+      failures.push("UI-file diff should require screenshot artifacts");
+    }
+  }
+
+  {
+    const { ok } = evaluatePrEvidence(buildFixtureBody(), REQUIRED_EVIDENCE_ROWS, {
+      changedFiles: [
+        "packages/ui/src/components/Foo.test.tsx",
+        "packages/app-core/src/services/thing.ts",
+        "packages/ui/src/components/Foo.stories.tsx",
+      ],
+    });
+    if (!ok) {
+      failures.push(
+        "test/story/server-only diff should not trigger surface artifacts",
+      );
+    }
+  }
+
+  {
+    const { ok } = evaluatePrEvidence(
+      buildFixtureBody({ "backend-logs": "- [ ] Backend logs N/A" }),
+    );
+    if (ok) failures.push("bare N/A should fail");
+  }
+
+  {
+    const { ok, findings } = evaluatePrEvidence(
+      buildFixtureBody({
+        "backend-logs": `- [ ] Backend logs: ${RETIRED_REPO_EVIDENCE_PATH}/13676-backend.txt`,
+      }),
+    );
+    const backend = findings.find((finding) => finding.id === "backend-logs");
+    if (ok) failures.push("retired repo evidence-only row should fail");
+    if (backend?.status !== "blank") {
+      failures.push("retired repo evidence-only row should be reported blank");
+    }
+  }
+
+  {
+    const retired = findRetiredRepoEvidenceFiles([
+      "packages/app/test-results/report.json",
+      `${RETIRED_REPO_EVIDENCE_PATH}/13676-backend.txt`,
+    ]);
+    if (retired.length !== 1) {
+      failures.push("retired repo evidence changed file should be rejected");
+    }
+  }
+
+  {
+    const body = REQUIRED_EVIDENCE_ROWS.slice(1)
+      .map(
+        ({ id }) =>
+          `<!-- evidence-row:${id} -->\n- [ ] N/A - covered elsewhere`,
+      )
+      .join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body);
+    const missing = findings.find(
+      (finding) => finding.id === "before-screenshots",
+    );
+    if (ok) failures.push("missing-marker fixture should fail");
+    if (missing?.status !== "missing") {
+      failures.push("absent row should be reported missing");
+    }
+  }
+
+  if (failures.length > 0) {
+    console.error("check-pr-evidence self-test FAILED:");
+    for (const failure of failures) console.error(`  - ${failure}`);
+    process.exit(1);
+  }
+  console.log("check-pr-evidence self-test passed (11 cases).");
+}
+
+function main() {
+  const args = process.argv.slice(2);
+  if (args.includes("--help") || args.includes("-h")) {
+    usage();
+    process.exit(0);
+  }
+  if (args.includes("--self-test")) {
+    runSelfTest();
+    return;
+  }
+
+  const body = readBody(args);
+  const labelsIdx = args.indexOf("--labels");
+  const labels = labelsIdx === -1 ? "" : (args[labelsIdx + 1] ?? "");
+  const changedFiles = readFileListArg(args, "--changed-files-file");
+  const addedFiles = readFileListArg(args, "--added-files-file");
+  const retiredEvidenceFiles = findRetiredRepoEvidenceFiles(changedFiles);
+  const { ok, findings } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+    labels,
+    changedFiles,
+    addedFiles,
+  });
+  const allOk = ok && retiredEvidenceFiles.length === 0;
+
+  if (args.includes("--json")) {
+    console.log(
+      JSON.stringify({ ok: allOk, findings, retiredEvidenceFiles }, null, 2),
+    );
+  } else {
+    for (const finding of findings) {
+      const symbol = finding.status === "ok" ? "ok  " : "FAIL";
+      console.log(
+        `  [${symbol}] ${finding.label} (${finding.id}): ${finding.status}`,
+      );
+    }
+    if (retiredEvidenceFiles.length > 0) {
+      console.log("  [FAIL] Retired repo evidence files:");
+      for (const file of retiredEvidenceFiles) console.log(`    - ${file}`);
+    }
+  }
+
+  if (!allOk) {
+    const bad = findings.filter((finding) => finding.status !== "ok");
+    console.error(
+      `\nEvidence gate FAILED: ${bad.length} row(s) need attention, ${retiredEvidenceFiles.length} retired repo evidence file(s) changed.
+
+How to fix (fastest path):
+  1. bun run evidence:doctor            # install any missing capture tool
+  2. capture: bun run --cwd packages/app audit:app  (screenshots + OCR)
+     and/or the fixtures under packages/ui/src/components/shell/__e2e__/
+  3. attach + patch rows in ONE command:
+     node scripts/pr-evidence.mjs rows <pr> \\
+       --row after-screenshots=shot.jpg --row walkthrough-video=walk.mp4 \\
+       --row ocr-review=ocr.txt --row frontend-logs=e2e.log ...
+     (uploads to the pr-evidence release and verifies this gate locally)
+
+Rules: visual rows on UI-touching PRs need REAL media (an uploaded image/video,
+not a link to the PR or /checks page); every other row needs an artifact link
+or 'N/A - <reason>'. A wholly-new surface may N/A the before-screenshots row.
+Worked example: https://github.com/elizaOS/eliza/pull/15171
+Full standard: CONTRIBUTING.md § Evidence.`,
+    );
+    process.exit(1);
+  }
+  console.log("\nEvidence gate passed: all required rows satisfied.");
+}
+
+if (
+  process.argv[1] &&
+  import.meta.url === pathToFileURL(process.argv[1]).href
+) {
+  main();
+}

scripts/check-pr-evidence.test.mjs

@@ -0,0 +1,584 @@
+/**
+ * Tests for the pull request evidence checker. The fixtures model the PR body
+ * instead of shelling out so the gate's parsing rules stay deterministic and
+ * cheap to run in PR workflows.
+ */
+
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import { dirname, join } from "node:path";
+import { describe, it } from "node:test";
+import { fileURLToPath } from "node:url";
+import {
+  boundRowBlock,
+  evaluatePrEvidence,
+  extractEvidenceRows,
+  findRetiredRepoEvidenceFiles,
+  hasArtifactReference,
+  hasNaWithReason,
+  hasOcrEvidenceReference,
+  hasVisualArtifactReference,
+  isChecked,
+  isRowSatisfied,
+  isRowSatisfiedForContext,
+  parseLabels,
+  REQUIRED_EVIDENCE_ROWS,
+  requiresSurfaceArtifacts,
+  requiresSurfaceArtifactsFromFiles,
+  SURFACE_ARTIFACT_ROW_IDS,
+} from "./check-pr-evidence.mjs";
+
+const HERE = dirname(fileURLToPath(import.meta.url));
+const TEMPLATE_PATH = join(HERE, "..", ".github", "pull_request_template.md");
+const RETIRED_REPO_EVIDENCE_PATH = [
+  ".github",
+  ["issue", "evidence"].join("-"),
+].join("/");
+
+function buildBody(overrides = {}) {
+  const defaults = {
+    "before-screenshots":
+      "- [ ] Before screenshots `N/A - backend-only change, no UI surface`.",
+    "after-screenshots":
+      "- [ ] After screenshots `N/A - backend-only change, no UI surface`.",
+    "walkthrough-video":
+      "- [x] A video walkthrough: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000000",
+    "backend-logs":
+      "- [ ] Backend logs: [backend.txt](https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000001)",
+    "frontend-logs": "- [ ] Frontend logs `N/A - no frontend change`.",
+    "llm-trajectory":
+      "- [ ] Real-LLM trajectory: [report](https://example.com/report.json)",
+    "domain-artifacts":
+      "- [ ] Domain artifacts: OCR report https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000007",
+  };
+  const merged = { ...defaults, ...overrides };
+  return REQUIRED_EVIDENCE_ROWS.map(
+    ({ id }) => `<!-- evidence-row:${id} -->\n${merged[id] ?? ""}`,
+  ).join("\n\n");
+}
+
+describe("check-pr-evidence parser", () => {
+  it("passes when every evidence row has an artifact or N/A reason", () => {
+    const { ok, findings } = evaluatePrEvidence(buildBody());
+    assert.equal(ok, true);
+    assert.ok(findings.every((finding) => finding.status === "ok"));
+  });
+
+  it("fails on a single blank row", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "backend-logs":
+          "- [ ] Backend logs show the real code path firing end to end, or are marked `N/A - <reason>`.",
+      }),
+    );
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "backend-logs").status,
+      "blank",
+    );
+    assert.equal(
+      findings.filter((finding) => finding.status === "ok").length,
+      REQUIRED_EVIDENCE_ROWS.length - 1,
+    );
+  });
+
+  it("fails when a row is checked without an artifact or N/A reason", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "backend-logs": "- [x] Backend logs attached.",
+      }),
+    );
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "backend-logs").status,
+      "blank",
+    );
+  });
+
+  it("fails when every required row is checked without artifacts or N/A reasons", () => {
+    const checkedRows = Object.fromEntries(
+      REQUIRED_EVIDENCE_ROWS.map(({ id, label }) => [
+        id,
+        `- [x] ${label} attached.`,
+      ]),
+    );
+    const { ok, findings } = evaluatePrEvidence(buildBody(checkedRows));
+    assert.equal(ok, false);
+    assert.ok(findings.every((finding) => finding.status === "blank"));
+  });
+
+  it("passes when every row is marked `N/A - <reason>`", () => {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    assert.equal(evaluatePrEvidence(body).ok, true);
+  });
+
+  it("fails UI-labeled PRs when screenshot/video rows are only N/A", () => {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      labels: "ui",
+    });
+    assert.equal(ok, false);
+    for (const id of SURFACE_ARTIFACT_ROW_IDS) {
+      assert.equal(
+        findings.find((finding) => finding.id === id).status,
+        "artifact-required",
+      );
+    }
+    assert.equal(
+      findings.find((finding) => finding.id === "ocr-review").status,
+      "ocr-required",
+    );
+  });
+
+  it("passes UI-labeled PRs when screenshot/video rows and OCR proof have concrete artifacts", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "before-screenshots":
+          "- [ ] Before screenshots: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000002",
+        "after-screenshots":
+          "- [ ] After screenshots: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000003",
+        "walkthrough-video":
+          "- [ ] Walkthrough video: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000004",
+        "domain-artifacts":
+          "- [ ] OCR text readout: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000008",
+      }),
+      REQUIRED_EVIDENCE_ROWS,
+      { labels: "frontend" },
+    );
+    assert.equal(ok, true);
+    assert.ok(findings.every((finding) => finding.status === "ok"));
+  });
+
+  it("fails UI-labeled PRs when screenshot/video artifacts omit OCR proof", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "before-screenshots":
+          "- [ ] Before screenshots: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000002",
+        "after-screenshots":
+          "- [ ] After screenshots: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000003",
+        "walkthrough-video":
+          "- [ ] Walkthrough video: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000004",
+        "domain-artifacts":
+          "- [ ] Domain artifacts `N/A - no domain artifacts produced`.",
+      }),
+      REQUIRED_EVIDENCE_ROWS,
+      { labels: "ui" },
+    );
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "ocr-review").status,
+      "ocr-required",
+    );
+  });
+
+  it("fails on a bare `N/A` with no reason", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({ "backend-logs": "- [ ] Backend logs N/A" }),
+    );
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "backend-logs").status,
+      "blank",
+    );
+  });
+
+  it("reports a required row as missing when its marker is absent", () => {
+    const body = REQUIRED_EVIDENCE_ROWS.slice(1)
+      .map(
+        ({ id }) =>
+          `<!-- evidence-row:${id} -->\n- [ ] N/A - covered elsewhere`,
+      )
+      .join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body);
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "before-screenshots").status,
+      "missing",
+    );
+  });
+
+  it("treats an empty body as all-missing", () => {
+    const { ok, findings } = evaluatePrEvidence("");
+    assert.equal(ok, false);
+    assert.ok(findings.every((finding) => finding.status === "missing"));
+  });
+});
+
+describe("check-pr-evidence row primitives", () => {
+  it("accepts N/A separators with a real reason", () => {
+    assert.equal(hasNaWithReason("N/A - backend-only, no UI"), true);
+    assert.equal(hasNaWithReason("N/A: nothing to show here"), true);
+    assert.equal(hasNaWithReason("N/A \u2014 no domain artifacts"), true);
+    assert.equal(hasNaWithReason("NA - agent change only"), true);
+  });
+
+  it("rejects bare or placeholder N/A reasons", () => {
+    assert.equal(hasNaWithReason("N/A"), false);
+    assert.equal(hasNaWithReason("N/A -"), false);
+    assert.equal(hasNaWithReason("N/A - "), false);
+    assert.equal(hasNaWithReason("N/A - <reason>."), false);
+  });
+
+  it("detects links and URLs", () => {
+    assert.equal(hasArtifactReference("[report](https://x/y.json)"), true);
+    assert.equal(
+      hasArtifactReference("see https://github.com/o/r/assets/1"),
+      true,
+    );
+    assert.equal(
+      hasArtifactReference(
+        "see https://user-images.githubusercontent.com/1/a.jpg",
+      ),
+      true,
+    );
+    assert.equal(hasArtifactReference("just words, no artifact"), false);
+  });
+
+  it("detects linked OCR evidence without accepting keyword-only prose", () => {
+    const rows = new Map([
+      [
+        "domain-artifacts",
+        "- [ ] OCR report: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000009",
+      ],
+    ]);
+    assert.equal(hasOcrEvidenceReference(rows), true);
+    assert.equal(
+      hasOcrEvidenceReference(
+        new Map([["domain-artifacts", "- [ ] OCR report was reviewed"]]),
+      ),
+      false,
+    );
+  });
+
+  it("rejects retired repo-local evidence paths", () => {
+    assert.equal(
+      hasArtifactReference(
+        `committed under ${RETIRED_REPO_EVIDENCE_PATH}/13676-a.png`,
+      ),
+      false,
+    );
+    assert.equal(
+      hasArtifactReference(
+        `[proof](${RETIRED_REPO_EVIDENCE_PATH}/13676-a.png)`,
+      ),
+      false,
+    );
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "backend-logs": `- [ ] Backend logs: ${RETIRED_REPO_EVIDENCE_PATH}/13676-backend.txt`,
+      }),
+    );
+    assert.equal(ok, false);
+    assert.equal(
+      findings.find((finding) => finding.id === "backend-logs").status,
+      "blank",
+    );
+  });
+
+  it("rejects changed files under the retired repo-local evidence path", () => {
+    assert.deepEqual(
+      findRetiredRepoEvidenceFiles([
+        "packages/app/test-results/report.json",
+        `${RETIRED_REPO_EVIDENCE_PATH}/13676-backend.txt`,
+        String.raw`.github\issue-evidence\13676-windows-path.txt`,
+      ]),
+      [
+        `${RETIRED_REPO_EVIDENCE_PATH}/13676-backend.txt`,
+        String.raw`.github\issue-evidence\13676-windows-path.txt`,
+      ],
+    );
+  });
+
+  it("accepts non-repo evidence directories in the diff", () => {
+    assert.deepEqual(
+      findRetiredRepoEvidenceFiles([
+        "packages/evidence/src/schema.ts",
+        "packages/app/test-results/issue-evidence/report.json",
+      ]),
+      [],
+    );
+  });
+
+  it("detects checked checkboxes without treating them as evidence", () => {
+    assert.equal(isChecked("- [x] done"), true);
+    assert.equal(isChecked("- [X] done"), true);
+    assert.equal(isChecked("- [ ] not done"), false);
+    assert.equal(isRowSatisfied("- [x] done"), false);
+  });
+
+  it("requires real media on visual rows — page links do not count", () => {
+    // The gaming vector: linking the PR itself or its /checks tab.
+    assert.equal(
+      hasVisualArtifactReference(
+        "- [ ] Before: https://github.com/elizaOS/eliza/pull/15178/checks",
+      ),
+      false,
+    );
+    assert.equal(
+      hasVisualArtifactReference(
+        "- [ ] After: https://github.com/elizaOS/eliza/pull/15178",
+      ),
+      false,
+    );
+    // Real media forms all count.
+    assert.equal(
+      hasVisualArtifactReference(
+        "https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000001",
+      ),
+      true,
+    );
+    assert.equal(
+      hasVisualArtifactReference("![after](https://example.com/x/after)"),
+      true,
+    );
+    assert.equal(
+      hasVisualArtifactReference(
+        '<img src="https://example.com/shot" width="400">',
+      ),
+      true,
+    );
+    assert.equal(
+      hasVisualArtifactReference("see https://example.com/walkthrough.mp4"),
+      true,
+    );
+  });
+
+  it("fails a UI-labeled PR whose visual rows link only to the PR/checks page", () => {
+    const { ok, findings } = evaluatePrEvidence(
+      buildBody({
+        "before-screenshots":
+          "- [ ] Before screenshots: https://github.com/elizaOS/eliza/pull/15178",
+        "after-screenshots":
+          "- [ ] After screenshots: https://github.com/elizaOS/eliza/pull/15178/checks",
+        "walkthrough-video":
+          "- [ ] Walkthrough video: https://github.com/elizaOS/eliza/pull/15178/checks",
+      }),
+      REQUIRED_EVIDENCE_ROWS,
+      { labels: "ui" },
+    );
+    assert.equal(ok, false);
+    for (const id of SURFACE_ARTIFACT_ROW_IDS) {
+      assert.equal(
+        findings.find((finding) => finding.id === id).status,
+        "artifact-required",
+      );
+    }
+  });
+
+  it("detects rendered-UI source files in the diff", () => {
+    assert.equal(
+      requiresSurfaceArtifactsFromFiles(["packages/ui/src/components/Foo.tsx"]),
+      true,
+    );
+    assert.equal(
+      requiresSurfaceArtifactsFromFiles(["packages/app/src/views/Home.css"]),
+      true,
+    );
+    assert.equal(
+      requiresSurfaceArtifactsFromFiles([
+        String.raw`apps\app\src\Panel.tsx`,
+      ]),
+      true,
+    );
+    // Tests, stories, and server code render nothing a user sees.
+    assert.equal(
+      requiresSurfaceArtifactsFromFiles([
+        "packages/ui/src/components/Foo.test.tsx",
+        "packages/ui/src/components/Foo.stories.tsx",
+        "packages/app-core/src/services/thing.ts",
+        "packages/app/README.md",
+      ]),
+      false,
+    );
+  });
+
+  it("forces surface artifacts from a UI diff even without labels", () => {
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - not applicable to this change\`.`,
+    ).join("\n\n");
+    const { ok, findings } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      changedFiles: ["packages/ui/src/components/NotificationRow.tsx"],
+    });
+    assert.equal(ok, false);
+    for (const id of SURFACE_ARTIFACT_ROW_IDS) {
+      assert.equal(
+        findings.find((finding) => finding.id === id).status,
+        "artifact-required",
+      );
+    }
+    assert.equal(
+      findings.find((finding) => finding.id === "ocr-review").status,
+      "ocr-required",
+    );
+  });
+
+  it("allows N/A before-screenshots when the whole UI surface is newly added", () => {
+    const media = (id, n) =>
+      `- [x] ![${id}](https://github.com/user-attachments/assets/00000000-0000-0000-0000-00000000000${n})`;
+    const body = buildBody({
+      "before-screenshots":
+        "- [ ] Before screenshots `N/A - the settings section is new in this PR; no prior surface existed`.",
+      "after-screenshots": media("after", 2),
+      "walkthrough-video":
+        "- [x] https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000003",
+      "domain-artifacts":
+        "- [ ] OCR text readout: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000008",
+    });
+    const changedFiles = [
+      "packages/ui/src/components/settings/NewSection.tsx",
+      "packages/ui/src/components/settings/new-section.test.ts",
+    ];
+    // Whole surface added → before may be N/A-with-reason.
+    const allNew = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      changedFiles,
+      addedFiles: changedFiles,
+    });
+    assert.equal(allNew.ok, true);
+    // Same body but the surface file was MODIFIED → before still needs media.
+    const modified = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      changedFiles,
+      addedFiles: [],
+    });
+    assert.equal(modified.ok, false);
+    assert.equal(
+      modified.findings.find((f) => f.id === "before-screenshots").status,
+      "artifact-required",
+    );
+    // A bare N/A with no reason never qualifies, even for a new surface.
+    const bare = evaluatePrEvidence(
+      buildBody({
+        "before-screenshots": "- [ ] Before screenshots N/A",
+        "after-screenshots": media("after", 2),
+        "walkthrough-video":
+          "- [x] https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000003",
+        "domain-artifacts":
+          "- [ ] OCR text readout: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000008",
+      }),
+      REQUIRED_EVIDENCE_ROWS,
+      { changedFiles, addedFiles: changedFiles },
+    );
+    assert.equal(bare.ok, false);
+  });
+
+  it("path detection overrides the coarse ui label when files are known", () => {
+    // The auto-labeler applies `ui` to any packages/ui path; a non-visual .ts
+    // change must not be forced to attach screenshots.
+    const body = REQUIRED_EVIDENCE_ROWS.map(
+      ({ id }) =>
+        `<!-- evidence-row:${id} -->\n- [ ] row \`N/A - non-visual .ts module change\`.`,
+    ).join("\n\n");
+    const { ok } = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      labels: "ui",
+      changedFiles: ["packages/ui/src/navigation/index.ts"],
+    });
+    assert.equal(ok, true);
+    // But a rendered-UI file still forces artifacts regardless of labels.
+    const forced = evaluatePrEvidence(body, REQUIRED_EVIDENCE_ROWS, {
+      labels: "",
+      changedFiles: ["packages/ui/src/navigation/index.ts", "packages/ui/src/components/Foo.tsx"],
+    });
+    assert.equal(forced.ok, false);
+  });
+
+  it("normalizes labels and detects surface labels", () => {
+    assert.deepEqual(parseLabels("bug, UI\nNative"), ["bug", "ui", "native"]);
+    assert.equal(requiresSurfaceArtifacts("testing,backend"), false);
+    assert.equal(requiresSurfaceArtifacts(["ci", "Frontend"]), true);
+  });
+
+  it("requires N/A-reason or artifact to satisfy a row", () => {
+    assert.equal(isRowSatisfied("- [ ] `N/A - not applicable`"), true);
+    assert.equal(isRowSatisfied("- [ ] [proof](https://e/x.png)"), true);
+    assert.equal(
+      isRowSatisfied(
+        "- [ ] Before screenshots are attached, or marked `N/A - <reason>`.",
+      ),
+      false,
+    );
+  });
+
+  it("requires artifacts when artifact-required mode is enabled", () => {
+    assert.equal(
+      isRowSatisfiedForContext("- [ ] `N/A - no UI`", {
+        artifactRequired: true,
+      }),
+      false,
+    );
+    assert.equal(
+      isRowSatisfiedForContext(
+        "- [ ] https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000005",
+        {
+          artifactRequired: true,
+        },
+      ),
+      true,
+    );
+  });
+});
+
+describe("check-pr-evidence marker extraction", () => {
+  it("captures the checkbox line plus indented continuation lines", () => {
+    const body = [
+      "<!-- evidence-row:backend-logs -->",
+      "- [ ] Backend logs show the real code path firing end to end,",
+      "      or are marked `N/A - no backend path in this change`.",
+      "",
+      "<!-- evidence-row:frontend-logs -->",
+      "- [ ] Frontend logs: https://github.com/user-attachments/assets/00000000-0000-0000-0000-000000000006",
+    ].join("\n");
+    const rows = extractEvidenceRows(body);
+    assert.ok(rows.get("backend-logs").includes("N/A - no backend path"));
+    assert.ok(rows.get("frontend-logs").includes("user-attachments/assets"));
+  });
+
+  it("bounds the last row so trailing links do not bleed in", () => {
+    const body = [
+      "<!-- evidence-row:domain-artifacts -->",
+      "- [ ] Domain artifacts are attached where applicable, or marked `N/A - <reason>`.",
+      "",
+      "# Evidence Details",
+      "",
+      "See [the runner](https://example.com/report.json).",
+    ].join("\n");
+    const rows = extractEvidenceRows(body);
+    assert.ok(!rows.get("domain-artifacts").includes("example.com"));
+    assert.equal(isRowSatisfied(rows.get("domain-artifacts")), false);
+  });
+
+  it("boundRowBlock stops at the first blank line or heading", () => {
+    const block = [
+      "- [ ] row text",
+      "      continued indented line",
+      "",
+      "# not part of the row",
+      "https://example.com/should-not-be-captured",
+    ].join("\n");
+    const bounded = boundRowBlock(block);
+    assert.ok(bounded.includes("continued indented line"));
+    assert.ok(!bounded.includes("example.com"));
+  });
+});
+
+describe("check-pr-evidence against the real PR template", () => {
+  it("carries a marker for every required evidence row", () => {
+    const template = readFileSync(TEMPLATE_PATH, "utf8");
+    const rows = extractEvidenceRows(template);
+    for (const { id } of REQUIRED_EVIDENCE_ROWS) {
+      assert.ok(rows.has(id), `template is missing marker evidence-row:${id}`);
+    }
+  });
+
+  it("fails the unedited template", () => {
+    const template = readFileSync(TEMPLATE_PATH, "utf8");
+    const { ok, findings } = evaluatePrEvidence(template);
+    assert.equal(ok, false);
+    assert.ok(findings.every((finding) => finding.status === "blank"));
+  });
+});

scripts/check-riscv64-artifacts.sh

@@ -0,0 +1,448 @@
+#!/usr/bin/env bash
+# check-riscv64-artifacts.sh — QEMU-user-mode smoke for every riscv64
+# native artifact this repo cross-compiles.
+#
+# One-shot harness: walks the known cross-build outputs (native plugins
+# + libllama / libggml family + libsigsys-handler-riscv64),
+# confirms each is an ELF UCB RISC-V
+# 64-bit double-float-ABI object, and exercises every executable smoke
+# under qemu-riscv64-static. Shared libraries are dlopen-verified via a
+# tiny C harness (also run under QEMU).
+#
+# Default posture: this is *gated* on ELIZA_RISCV64_SMOKE=1 because
+# the toolchain dependencies (qemu-user-static, the actual riscv64
+# artifacts) are heavy and may not be available on every CI box. With
+# the env-var unset the script no-ops and exits 0 with a clean
+# "skip" marker, so wiring it into a default-CI step is safe.
+#
+# Usage:
+#   bash scripts/check-riscv64-artifacts.sh                  # honor ELIZA_RISCV64_SMOKE
+#   ELIZA_RISCV64_SMOKE=1 bash scripts/check-riscv64-artifacts.sh
+#   ELIZA_RISCV64_SMOKE=1 bash scripts/check-riscv64-artifacts.sh --out build/reports/foo.json
+#   ELIZA_RISCV64_SMOKE=1 bash scripts/check-riscv64-artifacts.sh --no-qemu  # ELF-tag check only
+#
+# Exit code:
+#   0 — every artifact is PASS or a documented SKIP
+#   1 — at least one artifact FAILed
+#   2 — invalid CLI
+
+set -euo pipefail
+
+repo_root="$(cd "$(dirname "$0")/.." && pwd)"
+cd "$repo_root"
+
+OUT="$repo_root/build/reports/riscv64_artifacts.json"
+QEMU_TIMEOUT="${ELIZA_RISCV64_QEMU_TIMEOUT:-60}"
+RUN_QEMU=1
+NODE_BIN="${NODE_BIN:-$(command -v node || true)}"
+RM_PATH_RECURSIVE="$repo_root/packages/scripts/rm-path-recursive.mjs"
+
+while [ $# -gt 0 ]; do
+    case "$1" in
+        --out) OUT="$2"; shift 2;;
+        --no-qemu) RUN_QEMU=0; shift;;
+        --timeout) QEMU_TIMEOUT="$2"; shift 2;;
+        -h|--help)
+            awk '/^# /{print substr($0,3)} /^#$/{print ""} !/^#/{exit}' "$0"
+            exit 0;;
+        *) echo "unknown argument: $1" >&2; exit 2;;
+    esac
+done
+
+mkdir -p "$(dirname "$OUT")"
+
+now_epoch_ms() {
+    # GNU date supports %N; macOS doesn't, but riscv64 cross-builds are
+    # Linux-only so this script targets Linux-only callers.
+    date +%s%3N 2>/dev/null || python3 -c 'import time;print(int(time.time()*1000))'
+}
+
+iso_now() { date -u +'%Y-%m-%dT%H:%M:%SZ'; }
+
+remove_path_recursive() {
+    if [ -z "$NODE_BIN" ]; then
+        echo "[check-riscv64-artifacts] node not on PATH; cannot remove recursively via $RM_PATH_RECURSIVE." >&2
+        return 1
+    fi
+    "$NODE_BIN" "$RM_PATH_RECURSIVE" "$@"
+}
+
+# JSON record buffer. Each record is a single-line JSON object emitted
+# as we go, joined into the final array at the end.
+TMP_JSON="$(mktemp -t riscv64-artifacts-XXXXXX.jsonl)"
+trap 'rm -f "$TMP_JSON"' EXIT
+
+PASS_N=0; FAIL_N=0; SKIP_N=0
+
+emit_record() {
+    # $1 path, $2 kind (executable|shared|static-archive), $3 status, $4 detail, $5 duration_ms
+    local path="$1"; local kind="$2"; local status="$3"; local detail="$4"; local dur="$5"
+    # Properly JSON-escape detail and path.
+    local esc_path esc_detail
+    esc_path=$(printf '%s' "$path" | python3 -c 'import sys,json;sys.stdout.write(json.dumps(sys.stdin.read()))')
+    esc_detail=$(printf '%s' "$detail" | python3 -c 'import sys,json;sys.stdout.write(json.dumps(sys.stdin.read()))')
+    printf '{"path":%s,"kind":"%s","status":"%s","detail":%s,"duration_ms":%s}\n' \
+        "$esc_path" "$kind" "$status" "$esc_detail" "$dur" >> "$TMP_JSON"
+    case "$status" in
+        PASS) PASS_N=$((PASS_N+1));;
+        FAIL) FAIL_N=$((FAIL_N+1));;
+        SKIP) SKIP_N=$((SKIP_N+1));;
+    esac
+    printf '  [%-4s] %-70s %s\n' "$status" "${path#$repo_root/}" "$detail"
+}
+
+write_final_report() {
+    local final_status="$1"; local pre_skip_reason="${2:-}"
+    {
+        printf '{\n'
+        printf '  "generated_at": "%s",\n' "$(iso_now)"
+        printf '  "repo_root": "%s",\n' "$repo_root"
+        printf '  "eliza_riscv64_smoke": "%s",\n' "${ELIZA_RISCV64_SMOKE:-}"
+        printf '  "qemu_bin": "%s",\n' "${QEMU_BIN:-}"
+        printf '  "qemu_run": %s,\n' "$([ "$RUN_QEMU" = "1" ] && echo true || echo false)"
+        printf '  "qemu_timeout_seconds": %s,\n' "$QEMU_TIMEOUT"
+        printf '  "summary": {"pass": %d, "fail": %d, "skip": %d},\n' "$PASS_N" "$FAIL_N" "$SKIP_N"
+        printf '  "final_status": "%s",\n' "$final_status"
+        if [ -n "$pre_skip_reason" ]; then
+            local esc
+            esc=$(printf '%s' "$pre_skip_reason" | python3 -c 'import sys,json;sys.stdout.write(json.dumps(sys.stdin.read()))')
+            printf '  "pre_skip_reason": %s,\n' "$esc"
+        fi
+        printf '  "artifacts": [\n'
+        if [ -s "$TMP_JSON" ]; then
+            awk 'NR>1{printf ",\n"} {printf "    %s", $0} END{printf "\n"}' "$TMP_JSON"
+        fi
+        printf '  ]\n'
+        printf '}\n'
+    } > "$OUT"
+    echo
+    echo "Report: $OUT"
+    echo "PASS=$PASS_N  SKIP=$SKIP_N  FAIL=$FAIL_N"
+}
+
+# ── Gate ─────────────────────────────────────────────────────────────
+if [ "${ELIZA_RISCV64_SMOKE:-0}" != "1" ]; then
+    echo "[check-riscv64-artifacts] ELIZA_RISCV64_SMOKE not set; skipping."
+    echo "[check-riscv64-artifacts] To run: ELIZA_RISCV64_SMOKE=1 bun run check:riscv64-artifacts"
+    write_final_report "SKIP" "ELIZA_RISCV64_SMOKE!=1 (default-CI gate)"
+    exit 0
+fi
+
+# ── QEMU pre-flight ──────────────────────────────────────────────────
+QEMU_BIN="$(command -v qemu-riscv64-static 2>/dev/null || command -v qemu-riscv64 2>/dev/null || true)"
+if [ "$RUN_QEMU" = "1" ] && [ -z "$QEMU_BIN" ]; then
+    cat >&2 <<'EOF'
+[check-riscv64-artifacts] qemu-riscv64-static not found.
+
+Install (Debian/Ubuntu):
+    sudo apt-get install -y qemu-user-static binfmt-support
+
+Install (Fedora/RHEL):
+    sudo dnf install -y qemu-user-static
+
+Re-run, or pass --no-qemu to ELF-tag-only the artifacts.
+
+Treating this run as SKIP (exit 0) so default CI stays green.
+EOF
+    write_final_report "SKIP" "qemu-riscv64-static missing"
+    exit 0
+fi
+
+echo "[check-riscv64-artifacts] qemu_bin=${QEMU_BIN:-<elf-tag only>}  timeout=${QEMU_TIMEOUT}s"
+
+# ── Verifiers ────────────────────────────────────────────────────────
+is_riscv64_elf() {
+    local f="$1"
+    local info
+    # `-L` dereferences symlinks (libfoo.so → libfoo.so.1, etc.).
+    info="$(file -L -b "$f" 2>/dev/null || true)"
+    case "$info" in
+        *"UCB RISC-V"*"double-float ABI"*) return 0;;
+        *) return 1;;
+    esac
+}
+
+ar_members_are_rv64() {
+    local archive="$1"
+    case "$archive" in
+        /*) ;;
+        *) archive="$(cd "$(dirname "$archive")" && pwd)/$(basename "$archive")";;
+    esac
+    local extract_dir="$archive.qemu-extract"
+    remove_path_recursive "$extract_dir"
+    mkdir -p "$extract_dir"
+    if ! ( cd "$extract_dir" && ar x "$archive" >/dev/null 2>&1 ); then
+        remove_path_recursive "$extract_dir"
+        return 1
+    fi
+    local bad=0 saw=0
+    for member in "$extract_dir"/*.o; do
+        [ -f "$member" ] || continue
+        saw=1
+        if ! file -b "$member" | grep -q "UCB RISC-V"; then
+            bad=1; break
+        fi
+    done
+    remove_path_recursive "$extract_dir"
+    [ "$saw" = "1" ] && [ "$bad" = "0" ]
+}
+
+run_executable_under_qemu() {
+    # $1 path. Echoes "status|detail|duration_ms".
+    local exe="$1"
+    if [ ! -x "$exe" ]; then chmod +x "$exe" 2>/dev/null || true; fi
+    if [ "$RUN_QEMU" != "1" ] || [ -z "$QEMU_BIN" ]; then
+        echo "SKIP|qemu disabled (elf-tag only)|0"
+        return
+    fi
+    local log; log="$(mktemp -t riscv64-smoke-XXXXXX.log)"
+    local start end dur ec
+    start="$(now_epoch_ms)"
+    if timeout "$QEMU_TIMEOUT" "$QEMU_BIN" "$exe" >"$log" 2>&1; then
+        end="$(now_epoch_ms)"; dur=$((end - start))
+        rm -f "$log"
+        echo "PASS|qemu exit=0|$dur"
+    else
+        ec=$?
+        end="$(now_epoch_ms)"; dur=$((end - start))
+        local tail; tail="$(tail -c 240 "$log" 2>/dev/null | tr -d '\r' | tr '\n' ' ' | sed 's/ \+/ /g')"
+        rm -f "$log"
+        if [ "$ec" = "124" ]; then
+            echo "FAIL|qemu timeout after ${QEMU_TIMEOUT}s: $tail|$dur"
+        elif [ "$ec" = "77" ]; then
+            # Conventional autotools / GNU "skip" exit code — the test
+            # explicitly punted (usually due to a missing fixture).
+            echo "SKIP|qemu exit=77 (test self-skipped, likely missing fixture): $tail|$dur"
+        elif [ "$ec" = "2" ] && printf '%s' "$tail" | grep -qE "[Uu]sage:|--help|missing|not provided|fixture" ; then
+            # Exit 2 + usage/missing-fixture banner: the test binary
+            # requires an external argument (model GGUF, parity input,
+            # etc.) that the smoke harness doesn't supply. Treat as
+            # SKIP rather than failure — running it correctly is a
+            # fixture-provisioning task, not a riscv64 correctness gap.
+            echo "SKIP|qemu exit=2, usage/fixture message: $tail|$dur"
+        else
+            echo "FAIL|qemu exit=$ec: $tail|$dur"
+        fi
+    fi
+}
+
+# Dlopen-verify a .so under QEMU: compiles a tiny C harness for the
+# host that just dlopen()s the lib by path, then runs THAT host binary
+# only if it dlopen()s a host-arch object — instead, we test the .so by
+# inspecting its NEEDED entries with `readelf`/`file` because dlopen of
+# a riscv64 .so from a qemu-riscv64-static-run binary would need the
+# riscv64 libdl/libc loader chain present at the right sysroot path,
+# which is not assumed available. The harness therefore does:
+#   1. ELF arch check (must be UCB RISC-V).
+#   2. readelf -d sanity (must be DYN, must have at least one NEEDED).
+# This is the same verification ndk-stack and friends use for shipped
+# .so artifacts where running them in isolation is meaningless.
+verify_shared_lib() {
+    local so="$1"
+    if ! is_riscv64_elf "$so"; then
+        echo "FAIL|not a riscv64 ELF (file says: $(file -L -b "$so" 2>/dev/null | head -c 120))|0"
+        return
+    fi
+    if ! command -v readelf >/dev/null 2>&1; then
+        echo "PASS|ELF tag rv64 ok (readelf unavailable, skipped NEEDED check)|0"
+        return
+    fi
+    # readelf needs the real file (follow symlink with realpath if available).
+    local target="$so"
+    if [ -L "$so" ]; then
+        target="$(readlink -f "$so" 2>/dev/null || echo "$so")"
+    fi
+    local needed
+    needed="$(readelf -d "$target" 2>/dev/null | awk '/NEEDED/ {gsub(/[\[\]]/,"",$NF); print $NF}' | tr '\n' ',' | sed 's/,$//')"
+    if [ -z "$needed" ]; then
+        echo "PASS|ELF tag rv64 ok, no NEEDED (leaf .so)|0"
+    else
+        echo "PASS|ELF tag rv64 ok, NEEDED=[$needed]|0"
+    fi
+}
+
+verify_static_archive() {
+    local a="$1"
+    if ar_members_are_rv64 "$a"; then
+        echo "PASS|all .o members are UCB RISC-V|0"
+    else
+        echo "FAIL|one or more .o members are not UCB RISC-V|0"
+    fi
+}
+
+verify_artifact() {
+    # $1 = path. Determines kind by extension / executable bit.
+    local p="$1"
+    if [ ! -e "$p" ]; then
+        emit_record "$p" "missing" "SKIP" "artifact not built yet" "0"
+        return
+    fi
+    case "$p" in
+        *.a)
+            local result; result="$(verify_static_archive "$p")"
+            local status detail dur
+            status="${result%%|*}"; rest="${result#*|}"; detail="${rest%|*}"; dur="${rest##*|}"
+            emit_record "$p" "static-archive" "$status" "$detail" "$dur";;
+        *.so|*.so.*)
+            local result; result="$(verify_shared_lib "$p")"
+            local status detail dur
+            status="${result%%|*}"; rest="${result#*|}"; detail="${rest%|*}"; dur="${rest##*|}"
+            emit_record "$p" "shared-library" "$status" "$detail" "$dur";;
+        *)
+            if [ -f "$p" ] && is_riscv64_elf "$p"; then
+                if [ -x "$p" ] || head -c 4 "$p" | od -An -c 2>/dev/null | grep -q "\\\\177   E   L   F"; then
+                    local result; result="$(run_executable_under_qemu "$p")"
+                    local status detail dur
+                    status="${result%%|*}"; rest="${result#*|}"; detail="${rest%|*}"; dur="${rest##*|}"
+                    emit_record "$p" "executable" "$status" "$detail" "$dur"
+                else
+                    emit_record "$p" "data" "SKIP" "ELF but not executable" "0"
+                fi
+            else
+                emit_record "$p" "unknown" "FAIL" "not a riscv64 ELF (file: $(file -L -b "$p" 2>/dev/null | head -c 120))" "0"
+            fi;;
+    esac
+}
+
+# ── Artifact inventory ───────────────────────────────────────────────
+# Each entry's path is relative to repo root. Missing artifacts are
+# reported as SKIP (with reason) — the build-driver script
+# scripts/build-riscv64-artifacts.sh is responsible for producing them.
+
+NATIVE_PLUGINS=(
+    "qjl-cpu              packages/native/plugins/qjl-cpu/build/riscv64/libqjl.a"
+    "qjl-cpu              packages/native/plugins/qjl-cpu/build/riscv64/qjl_int8_smoke"
+    "qjl-cpu              packages/native/plugins/qjl-cpu/build/riscv64/qjl_avxvnni_smoke"
+    "qjl-cpu              packages/native/plugins/qjl-cpu/build/riscv64/qjl_fork_parity"
+    "qjl-cpu              packages/native/plugins/qjl-cpu/build/riscv64/qjl_bench"
+
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/libpolarquant.a"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_simd_parity_test"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_dot_test"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_preht_dot_test"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_preht_simd_parity_test"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_roundtrip_test"
+    "polarquant-cpu       packages/native/plugins/polarquant-cpu/build/riscv64/polar_bench"
+
+    "turboquant-cpu       packages/native/plugins/turboquant-cpu/build/riscv64/libturboquant.a"
+    "turboquant-cpu       packages/native/plugins/turboquant-cpu/build/riscv64/turboquant_smoke"
+    "turboquant-cpu       packages/native/plugins/turboquant-cpu/build/riscv64/turboquant_simd_parity"
+
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/libsilero_vad.a"
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/libsilero_vad.so"
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/silero_vad_abi_smoke"
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/silero_vad_resample_test"
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/silero_vad_runtime_test"
+    "silero-vad-cpp       packages/native/plugins/silero-vad-cpp/build/riscv64/silero_vad_state_test"
+
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/libvoice_classifier.a"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/libvoice_classifier.so"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_classifier_abi_smoke"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_diarizer_parity_test"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_emotion_classes_test"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_gguf_loader_test"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_mel_features_test"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_speaker_distance_test"
+    "voice-classifier-cpp packages/native/plugins/voice-classifier-cpp/build/riscv64/voice_speaker_parity_test"
+
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/libwakeword.a"
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/libwakeword.so"
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/wakeword_abi_smoke"
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/wakeword_melspec_test"
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/wakeword_runtime_test"
+    "wakeword-cpp         packages/native/plugins/wakeword-cpp/build/riscv64/wakeword_window_test"
+
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/libyolo.a"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/libyolo.so"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/yolo_abi_smoke"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/yolo_classes_test"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/yolo_letterbox_test"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/yolo_nms_test"
+    "yolo-cpp             packages/native/plugins/yolo-cpp/build/riscv64/yolo_runtime_test"
+
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/libface.a"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/libface.so"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_abi_smoke"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_align_test"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_anchor_test"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_distance_test"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_embed_runtime_test"
+    "face-cpp             packages/native/plugins/face-cpp/build/riscv64/face_runtime_test"
+
+    "doctr-cpp            packages/native/plugins/doctr-cpp/build/riscv64/libdoctr.a"
+    "doctr-cpp            packages/native/plugins/doctr-cpp/build/riscv64/doctr_abi_smoke"
+)
+
+# MTP libllama + ggml family stages into either:
+#   build/riscv64-stage/riscv64/                              (linux-riscv64 staging)
+#   packages/app-core/platforms/android/app/src/main/jniLibs/riscv64/
+#   packages/app-core/platforms/android/app/src/main/assets/agent/riscv64/
+# We probe all three locations; the harness reports SKIP only if none
+# contains the .so.
+LLAMA_FAMILY_BASENAMES=(
+    libllama.so
+    libggml.so
+    libggml-base.so
+    libggml-cpu.so
+    libllama-common.so
+    libeliza-llama-shim.so
+)
+
+LLAMA_FAMILY_SEARCH_DIRS=(
+    "build/riscv64-stage/riscv64"
+    "packages/app-core/platforms/android/app/src/main/jniLibs/riscv64"
+    "packages/app-core/platforms/android/app/src/main/assets/agent/riscv64"
+)
+
+SIGSYS_SEARCH=(
+    "${HOME}/.cache/eliza-android-agent/seccomp-shim/riscv64/libsigsys-handler.so"
+)
+
+# ── Walk + verify ────────────────────────────────────────────────────
+echo "── Native plugins ──"
+for entry in "${NATIVE_PLUGINS[@]}"; do
+    # shellcheck disable=SC2086
+    set -- $entry
+    pkg="$1"; path="$2"
+    verify_artifact "$repo_root/$path"
+done
+
+echo
+echo "── libllama / libggml family (MTP) ──"
+for basename in "${LLAMA_FAMILY_BASENAMES[@]}"; do
+    found=""
+    for dir in "${LLAMA_FAMILY_SEARCH_DIRS[@]}"; do
+        candidate="$repo_root/$dir/$basename"
+        if [ -e "$candidate" ]; then
+            found="$candidate"; break
+        fi
+    done
+    if [ -n "$found" ]; then
+        verify_artifact "$found"
+    else
+        # Synthesize a path for the report — caller knows what's missing.
+        emit_record "$repo_root/${LLAMA_FAMILY_SEARCH_DIRS[0]}/$basename" "shared-library" "SKIP" \
+            "not built; run \`bun run build:riscv64-artifacts\` (libllama for linux-riscv64-cpu + android-riscv64-cpu)" "0"
+    fi
+done
+
+echo
+echo "── libsigsys-handler-riscv64 (Bun seccomp shim) ──"
+s_found=""
+for p in "${SIGSYS_SEARCH[@]}"; do
+    if [ -e "$p" ]; then s_found="$p"; break; fi
+done
+if [ -n "$s_found" ]; then
+    verify_artifact "$s_found"
+else
+    emit_record "${SIGSYS_SEARCH[0]}" "shared-library" "SKIP" \
+        "not built; run \`node packages/app-core/scripts/aosp/compile-shim.mjs --abi riscv64\`" "0"
+fi
+
+# ── Verdict ──────────────────────────────────────────────────────────
+if [ "$FAIL_N" -gt 0 ]; then
+    write_final_report "FAIL"
+    exit 1
+fi
+write_final_report "PASS"
+exit 0

scripts/cloud/__tests__/mock-stack-up.test.mjs

@@ -0,0 +1,83 @@
+/**
+ * Smoke test for the cloud mock-stack orchestrator.
+ *
+ * Verifies flag parsing, help/error exit codes, and SIGINT-driven shutdown
+ * without booting the heavy services (cloud-api, frontend, migrations).
+ */
+
+import { describe, expect, test } from "bun:test";
+import { spawn } from "node:child_process";
+import { mkdtempSync, readFileSync } from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+const REPO_ROOT = path.resolve(import.meta.dirname, "../../..");
+const SCRIPT = path.join(REPO_ROOT, "scripts/cloud/mock-stack-up.mjs");
+
+// Bun's test-runner pipe capture loses output on `process.exit(non-zero)`,
+// so we redirect child stdio to files via the shell and read them back.
+function redirectedRun(args, { collectMs = 0, env = {} } = {}) {
+  return new Promise((resolve) => {
+    const tmp = mkdtempSync(path.join(os.tmpdir(), "mock-stack-test-"));
+    const outFile = path.join(tmp, "out.log");
+    const errFile = path.join(tmp, "err.log");
+    const cmd =
+      `exec node ${JSON.stringify(SCRIPT)} ` +
+      `${args.map((a) => JSON.stringify(a)).join(" ")} ` +
+      `>${JSON.stringify(outFile)} 2>${JSON.stringify(errFile)}`;
+    const proc = spawn("sh", ["-c", cmd], { env: { ...process.env, ...env } });
+    if (collectMs > 0) {
+      setTimeout(() => proc.kill("SIGINT"), collectMs);
+    }
+    proc.on("exit", (code, signal) => {
+      let stdout = "";
+      let stderr = "";
+      try {
+        stdout = readFileSync(outFile, "utf8");
+      } catch {}
+      try {
+        stderr = readFileSync(errFile, "utf8");
+      } catch {}
+      resolve({ code, signal, stdout, stderr });
+    });
+  });
+}
+
+describe("mock-stack-up orchestrator", () => {
+  test("--help prints usage and exits 0", async () => {
+    const r = await redirectedRun(["--help"]);
+    expect(r.code).toBe(0);
+    expect(r.stdout).toContain("Usage:");
+    expect(r.stdout).toContain("--no-frontend");
+    expect(r.stdout).toContain("--reset");
+  });
+
+  test("unknown flag exits 1 with usage", async () => {
+    const r = await redirectedRun(["--definitely-not-a-flag"]);
+    expect(r.code).toBe(1);
+    const combined = r.stdout + r.stderr;
+    expect(combined).toContain("Unknown flag");
+    expect(combined).toContain("Usage:");
+  });
+
+  test("skip-everything boot reaches ready banner and SIGINT shuts down cleanly", async () => {
+    const started = Date.now();
+    const r = await redirectedRun(
+      ["--no-frontend", "--no-cp", "--no-hetzner", "--no-migrations"],
+      { collectMs: 4_000 },
+    );
+    const elapsed = Date.now() - started;
+    expect(elapsed).toBeLessThan(15_000);
+    const combined = r.stdout + r.stderr;
+    // Any of: banner printed (success), shutdown logged (signal handled),
+    // or fast non-zero exit (failure handled) prove the orchestrator's
+    // wiring, signal, and failure paths are intact and it didn't hang.
+    const handled =
+      combined.includes("Eliza cloud mock stack") ||
+      combined.includes("shutting down") ||
+      combined.includes("stopped") ||
+      combined.includes("failed to start") ||
+      r.code === 1;
+    expect(handled).toBe(true);
+  }, 20_000);
+});

scripts/cloud/mock-stack-up.mjs

@@ -0,0 +1,492 @@
+#!/usr/bin/env node
+/**
+ * Eliza cloud — one-command local mock stack.
+ *
+ * Boots Hetzner mock, control-plane mock, cloud-api (with MOCK_REDIS + PGlite),
+ * and cloud-frontend, wired together with auto-picked ports and health-check
+ * gating. Streams each subprocess to ./.logs/<service>.log and the console
+ * with a colored prefix. Ctrl+C triggers ordered graceful shutdown.
+ */
+
+import { spawn, spawnSync } from "node:child_process";
+import { createWriteStream, existsSync, mkdirSync, writeSync } from "node:fs";
+import net from "node:net";
+import path from "node:path";
+import process from "node:process";
+
+const REPO_ROOT = path.resolve(import.meta.dirname, "../..");
+const LOG_DIR = path.join(REPO_ROOT, ".logs");
+const PGDATA_DIR = path.join(REPO_ROOT, ".eliza/.pgdata");
+const RM_RECURSIVE_SCRIPT = path.join(
+  REPO_ROOT,
+  "packages",
+  "scripts",
+  "rm-path-recursive.mjs",
+);
+
+const USAGE = `Usage: bun scripts/cloud/mock-stack-up.mjs [flags]
+
+Boots the local Eliza cloud mock stack (Hetzner mock + control-plane mock +
+cloud-api with MOCK_REDIS + PGlite, optionally cloud-frontend) wired together.
+
+Flags:
+  --no-frontend         skip cloud-frontend
+  --no-cp               skip control-plane mock
+  --no-hetzner          skip hetzner mock
+  --no-migrations       skip cloud-shared db:migrate
+  --reset               wipe PGlite data dir before booting
+  --with-daemon         run the autoscale / hot-pool / pool-replenish cron
+                        loops on real intervals against the mock stack, so the
+                        pool is maintained by the daemon (not test-driven ticks).
+                        Interval (ms) overridable via DAEMON_TICK_MS (default 15000).
+  --port-frontend N     override frontend port
+  --port-api N          override cloud-api port
+  --port-cp N           override control-plane port
+  --port-hetzner N      override hetzner port
+  --help                print this usage
+`;
+
+function parseFlags(argv) {
+  const flags = {
+    noFrontend: false,
+    noCp: false,
+    noHetzner: false,
+    noMigrations: false,
+    reset: false,
+    withDaemon: false,
+    help: false,
+    portFrontend: undefined,
+    portApi: undefined,
+    portCp: undefined,
+    portHetzner: undefined,
+  };
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    switch (a) {
+      case "--no-frontend":
+        flags.noFrontend = true;
+        break;
+      case "--no-cp":
+        flags.noCp = true;
+        break;
+      case "--no-hetzner":
+        flags.noHetzner = true;
+        break;
+      case "--no-migrations":
+        flags.noMigrations = true;
+        break;
+      case "--reset":
+        flags.reset = true;
+        break;
+      case "--with-daemon":
+        flags.withDaemon = true;
+        break;
+      case "--help":
+      case "-h":
+        flags.help = true;
+        break;
+      case "--port-frontend":
+        flags.portFrontend = Number.parseInt(argv[++i], 10);
+        break;
+      case "--port-api":
+        flags.portApi = Number.parseInt(argv[++i], 10);
+        break;
+      case "--port-cp":
+        flags.portCp = Number.parseInt(argv[++i], 10);
+        break;
+      case "--port-hetzner":
+        flags.portHetzner = Number.parseInt(argv[++i], 10);
+        break;
+      default:
+        return { error: `Unknown flag: ${a}` };
+    }
+  }
+  return { flags };
+}
+
+const COLOR = {
+  reset: "\x1b[0m",
+  cyan: "\x1b[36m",
+  magenta: "\x1b[35m",
+  yellow: "\x1b[33m",
+  green: "\x1b[32m",
+  red: "\x1b[31m",
+  gray: "\x1b[90m",
+};
+const color = (name, text) => `${COLOR[name] ?? ""}${text}${COLOR.reset}`;
+
+function pickPort() {
+  return new Promise((resolve, reject) => {
+    const srv = net.createServer();
+    srv.unref();
+    srv.on("error", reject);
+    srv.listen(0, "127.0.0.1", () => {
+      const addr = srv.address();
+      const p = typeof addr === "object" && addr ? addr.port : 0;
+      srv.close(() => resolve(p));
+    });
+  });
+}
+
+async function waitForHttp(url, { timeoutMs = 60_000, intervalMs = 500 } = {}) {
+  const start = Date.now();
+  let lastErr;
+  while (Date.now() - start < timeoutMs) {
+    try {
+      const res = await fetch(url, { method: "GET" });
+      if (res.status > 0) return true;
+    } catch (e) {
+      lastErr = e;
+    }
+    await new Promise((r) => setTimeout(r, intervalMs));
+  }
+  throw new Error(
+    `Timeout waiting for ${url}: ${lastErr?.message ?? "no response"}`,
+  );
+}
+
+const services = [];
+/** Active --with-daemon interval timers, cleared on shutdown. */
+const daemonTimers = [];
+
+/**
+ * The cron loops that, in production, a long-running daemon ticks on its own
+ * schedule. `--with-daemon` drives them here so the mock stack maintains its
+ * pool over time without test-enqueued provision jobs (#8921).
+ */
+const DAEMON_CRONS = [
+  { name: "node-autoscale", path: "/v1/cron/node-autoscale" },
+  { name: "agent-hot-pool", path: "/v1/cron/agent-hot-pool" },
+  { name: "pool-replenish", path: "/v1/cron/pool-replenish" },
+];
+
+/**
+ * Start the autoscale/hot-pool daemon loops against the running mock cloud-api.
+ * Each cron is POSTed on `tickMs` intervals with the CRON_SECRET bearer the
+ * routes expect (`verifyCronSecret`). Returns nothing; timers live in
+ * `daemonTimers` and are cleared by `shutdown`.
+ */
+function startDaemons(apiBase, cronSecret) {
+  const tickMs = Number(process.env.DAEMON_TICK_MS) || 15_000;
+  const headers = { authorization: `Bearer ${cronSecret}` };
+  for (const cron of DAEMON_CRONS) {
+    const tick = async () => {
+      try {
+        const res = await fetch(`${apiBase}${cron.path}`, {
+          method: "POST",
+          headers,
+        });
+        process.stdout.write(
+          `${color("gray", "[daemon]")} ${cron.name} → ${res.status}\n`,
+        );
+      } catch (e) {
+        process.stdout.write(
+          `${color("red", "[daemon]")} ${cron.name} tick failed: ${e?.message ?? e}\n`,
+        );
+      }
+    };
+    void tick(); // fire once immediately so the pool warms without waiting a full interval
+    const timer = setInterval(tick, tickMs);
+    timer.unref?.();
+    daemonTimers.push(timer);
+  }
+  process.stdout.write(
+    `${color("gray", "[stack]")} --with-daemon: ticking ${DAEMON_CRONS.length} cron loop(s) every ${tickMs}ms\n`,
+  );
+}
+
+function streamProcess(name, colorName, proc) {
+  const logPath = path.join(LOG_DIR, `${name}.log`);
+  const logStream = createWriteStream(logPath, { flags: "a" });
+  const prefix = color(colorName, `[${name}]`);
+  const tail = [];
+  const pipe = (stream) => {
+    let buf = "";
+    stream.setEncoding("utf8");
+    stream.on("data", (chunk) => {
+      logStream.write(chunk);
+      buf += chunk;
+      let idx;
+      while ((idx = buf.indexOf("\n")) !== -1) {
+        const line = buf.slice(0, idx);
+        buf = buf.slice(idx + 1);
+        tail.push(line);
+        if (tail.length > 100) tail.shift();
+        process.stdout.write(`${prefix} ${line}\n`);
+      }
+    });
+  };
+  pipe(proc.stdout);
+  pipe(proc.stderr);
+  const entry = { name, color: colorName, proc, logPath, tail };
+  services.push(entry);
+  return entry;
+}
+
+async function shutdown(code = 0) {
+  process.stdout.write(`${color("gray", "[stack]")} shutting down...\n`);
+  for (const timer of daemonTimers) clearInterval(timer);
+  daemonTimers.length = 0;
+  for (const svc of [...services].reverse()) {
+    if (svc.proc.exitCode === null && svc.proc.signalCode === null) {
+      try {
+        svc.proc.kill("SIGTERM");
+      } catch {}
+    }
+  }
+  const grace = setTimeout(() => {
+    for (const svc of services) {
+      if (svc.proc.exitCode === null && svc.proc.signalCode === null) {
+        try {
+          svc.proc.kill("SIGKILL");
+        } catch {}
+      }
+    }
+  }, 5_000);
+  await Promise.all(
+    services.map(
+      (svc) =>
+        new Promise((resolve) => {
+          if (svc.proc.exitCode !== null || svc.proc.signalCode !== null)
+            return resolve();
+          svc.proc.once("exit", () => resolve());
+        }),
+    ),
+  );
+  clearTimeout(grace);
+  process.stdout.write(`${color("gray", "[stack]")} stopped\n`);
+  process.exit(code);
+}
+
+function dumpTail(svcName) {
+  const svc = services.find((s) => s.name === svcName);
+  if (!svc) return;
+  process.stderr.write(`\n--- last 30 lines of ${svcName} ---\n`);
+  for (const line of svc.tail.slice(-30)) process.stderr.write(`${line}\n`);
+  process.stderr.write(`--- end ${svcName} ---\n`);
+}
+
+function rmRecursive(targetPath) {
+  const result = spawnSync(
+    process.execPath,
+    [RM_RECURSIVE_SCRIPT, targetPath],
+    {
+      cwd: REPO_ROOT,
+      encoding: "utf8",
+    },
+  );
+  if (result.status !== 0) {
+    const detail = result.stderr || result.stdout || `exit ${result.status}`;
+    throw new Error(`recursive cleanup failed for ${targetPath}: ${detail}`);
+  }
+}
+
+async function main() {
+  const { flags, error } = parseFlags(process.argv.slice(2));
+  if (error) {
+    writeSync(1, `${error}\n\n${USAGE}`);
+    process.exit(1);
+  }
+  if (flags.help) {
+    writeSync(1, USAGE);
+    process.exit(0);
+  }
+
+  mkdirSync(LOG_DIR, { recursive: true });
+  if (flags.reset && existsSync(PGDATA_DIR)) {
+    process.stdout.write(
+      `${color("gray", "[stack]")} --reset: wiping ${PGDATA_DIR}\n`,
+    );
+    rmRecursive(PGDATA_DIR);
+  }
+  mkdirSync(PGDATA_DIR, { recursive: true });
+
+  const pickIfFalsy = async (v) =>
+    v && Number.isFinite(v) ? v : await pickPort();
+  const hetznerPort = await pickIfFalsy(
+    flags.portHetzner ?? Number(process.env.HETZNER_PORT),
+  );
+  const cpPort = await pickIfFalsy(flags.portCp ?? Number(process.env.CP_PORT));
+  const apiPort = await pickIfFalsy(
+    flags.portApi ?? Number(process.env.API_PORT),
+  );
+  const frontendPort = await pickIfFalsy(
+    flags.portFrontend ?? Number(process.env.FRONTEND_PORT),
+  );
+
+  process.on("SIGINT", () => void shutdown(0));
+  process.on("SIGTERM", () => void shutdown(0));
+
+  if (!flags.noMigrations) {
+    process.stdout.write(
+      `${color("gray", "[stack]")} running cloud-shared db:migrate...\n`,
+    );
+    const migrate = spawn(
+      "bun",
+      ["run", "--cwd", "packages/cloud/shared", "db:migrate"],
+      {
+        cwd: REPO_ROOT,
+        env: { ...process.env, DATABASE_URL: `pglite://${PGDATA_DIR}` },
+        stdio: "inherit",
+      },
+    );
+    const code = await new Promise((res) => migrate.on("exit", (c) => res(c)));
+    if (code !== 0) {
+      process.stderr.write(`migrations failed (exit ${code})\n`);
+      process.exit(1);
+    }
+  }
+
+  const tHetzner = `http://127.0.0.1:${hetznerPort}`;
+  const tCp = `http://127.0.0.1:${cpPort}`;
+  const tApi = `http://127.0.0.1:${apiPort}`;
+  const tFrontend = `http://127.0.0.1:${frontendPort}`;
+
+  const baseEnv = {
+    ...process.env,
+    MOCK_REDIS: "1",
+    DATABASE_URL: `pglite://${PGDATA_DIR}`,
+    HCLOUD_API_BASE_URL: `${tHetzner}/v1`,
+    HCLOUD_TOKEN: "local-mock-token",
+    CONTAINER_CONTROL_PLANE_URL: tCp,
+    CONTAINER_CONTROL_PLANE_TOKEN: "local-mock-token",
+    CRON_SECRET: "local-cron-secret",
+    // Fixed dev master key so the mock stack exercises the real secrets
+    // envelope end-to-end. Since #12229 (M4) the LocalKMSProvider fails closed
+    // when SECRETS_MASTER_KEY is unset, so any secrets op during a mock session
+    // (connector-OAuth token storage, provisioning/container-deploy secrets)
+    // would otherwise throw. This is a throwaway local-dev key, never a real one.
+    SECRETS_MASTER_KEY:
+      process.env.SECRETS_MASTER_KEY ?? "0123456789abcdef".repeat(4),
+  };
+
+  try {
+    if (!flags.noHetzner) {
+      const proc = spawn(
+        "bun",
+        [
+          "run",
+          "packages/test/cloud-mocks/bin/hetzner-mock.ts",
+          "--port",
+          String(hetznerPort),
+        ],
+        { cwd: REPO_ROOT, env: baseEnv },
+      );
+      streamProcess("hetzner", "cyan", proc);
+      try {
+        await waitForHttp(`${tHetzner}/`, { timeoutMs: 30_000 });
+      } catch (e) {
+        dumpTail("hetzner");
+        throw new Error(`hetzner mock failed to start: ${e.message}`);
+      }
+    }
+
+    if (!flags.noCp) {
+      const proc = spawn(
+        "bun",
+        ["run", "packages/test/cloud-mocks/bin/control-plane-mock.ts"],
+        {
+          cwd: REPO_ROOT,
+          env: {
+            ...baseEnv,
+            PORT: String(cpPort),
+            HCLOUD_API_BASE_URL: `${tHetzner}/v1`,
+          },
+        },
+      );
+      streamProcess("cp", "magenta", proc);
+      try {
+        await waitForHttp(`${tCp}/`, { timeoutMs: 30_000 });
+      } catch (e) {
+        dumpTail("cp");
+        throw new Error(`control-plane mock failed to start: ${e.message}`);
+      }
+    }
+
+    {
+      const proc = spawn("bun", ["run", "--cwd", "packages/cloud/api", "dev"], {
+        cwd: REPO_ROOT,
+        env: { ...baseEnv, API_DEV_PORT: String(apiPort) },
+      });
+      streamProcess("api", "yellow", proc);
+      try {
+        await waitForHttp(`${tApi}/`, { timeoutMs: 60_000 });
+      } catch (e) {
+        dumpTail("api");
+        throw new Error(`cloud-api failed to start: ${e.message}`);
+      }
+    }
+
+    // The apex frontend lives in packages/app; there is no standalone
+    // packages/cloud-frontend (#9093). Skip its boot gracefully when the dir is
+    // absent so the mock stack still comes up (api + control-plane).
+    const frontendDir = path.join(REPO_ROOT, "packages/cloud-frontend");
+    if (!flags.noFrontend && !existsSync(frontendDir)) {
+      process.stderr.write(
+        `${color("gray", "[stack]")} cloud-frontend removed (#9093); skipping frontend boot. The apex frontend now lives in packages/app.\n`,
+      );
+    } else if (!flags.noFrontend) {
+      const proc = spawn(
+        "bun",
+        [
+          "run",
+          "--cwd",
+          "packages/cloud-frontend",
+          "dev",
+          "--",
+          "--port",
+          String(frontendPort),
+          "--strictPort",
+        ],
+        { cwd: REPO_ROOT, env: { ...baseEnv, VITE_API_URL: tApi } },
+      );
+      streamProcess("fe", "green", proc);
+      try {
+        await waitForHttp(`${tFrontend}/`, { timeoutMs: 60_000 });
+      } catch (e) {
+        dumpTail("fe");
+        throw new Error(`cloud-frontend failed to start: ${e.message}`);
+      }
+    }
+  } catch (e) {
+    process.stderr.write(`\n${color("red", "[stack]")} ${e.message}\n`);
+    await shutdown(1);
+    return;
+  }
+
+  for (const svc of services) {
+    svc.proc.on("exit", (code, signal) => {
+      if (signal === "SIGTERM" || signal === "SIGKILL") return;
+      process.stderr.write(
+        `\n${color("red", "[stack]")} ${svc.name} exited unexpectedly (code=${code}, signal=${signal})\n`,
+      );
+      void shutdown(1);
+    });
+  }
+
+  const pad = (s, n) => (s + " ".repeat(n)).slice(0, n);
+  const lines = [
+    "┌────────────────────────────────────────────────────────────┐",
+    `│ ${pad("Eliza cloud mock stack — ready", 58)} │`,
+    "├────────────────────────────────────────────────────────────┤",
+    `│ Frontend         ${pad(flags.noFrontend ? "(skipped)" : tFrontend, 41)} │`,
+    `│ Cloud API        ${pad(tApi, 41)} │`,
+    `│ Control plane    ${pad(flags.noCp ? "(skipped)" : tCp, 41)} │`,
+    `│ Hetzner mock     ${pad(flags.noHetzner ? "(skipped)" : tHetzner, 41)} │`,
+    `│ DB (PGlite)      ${pad("./.eliza/.pgdata", 41)} │`,
+    `│ Redis            ${pad("in-memory (MOCK_REDIS=1)", 41)} │`,
+    "└────────────────────────────────────────────────────────────┘",
+    `Logs streaming to ${path.relative(REPO_ROOT, LOG_DIR)}/<service>.log`,
+    "Ctrl+C to stop all.",
+  ];
+  process.stdout.write(`\n${lines.join("\n")}\n`);
+
+  if (flags.withDaemon) {
+    startDaemons(tApi, baseEnv.CRON_SECRET);
+  }
+}
+
+main().catch((e) => {
+  process.stderr.write(`fatal: ${e?.stack ?? e}\n`);
+  void shutdown(1);
+});

scripts/cloud/siwe-test-login.mjs

@@ -0,0 +1,85 @@
+#!/usr/bin/env bun
+/**
+ * Headless SIWE ("Sign-In With Ethereum") login — dev + CI smoke.
+ *
+ * Creates/uses a throwaway Ethereum wallet, runs the GENUINE EIP-4361 handshake
+ * against a cloud-api (nonce → sign → verify), and prints a real API key for a
+ * free account. No browser wallet, no human, no mock — the signature, nonce, and
+ * domain are validated by the real server.
+ *
+ * Usage:
+ *   bun run cloud:login:test-wallet                      # against https://api.elizacloud.ai
+ *   bun run cloud:login:test-wallet -- --base http://127.0.0.1:8787
+ *   bun run cloud:login:test-wallet -- --json            # machine-readable
+ *   PRIVATE_KEY=0x... bun run cloud:login:test-wallet    # reuse a fixed wallet
+ *
+ * CI: use as a login gate before driving authenticated flows — exits non-zero if
+ * login fails so a broken auth path turns the job red.
+ */
+
+import { siweTestLogin } from "@elizaos/cloud-shared/lib/auth/siwe-test-login";
+
+function arg(name, fallback) {
+  const i = process.argv.indexOf(`--${name}`);
+  if (i !== -1 && process.argv[i + 1]) return process.argv[i + 1];
+  return fallback;
+}
+
+const baseUrl = arg(
+  "base",
+  process.env.SIWE_BASE ?? "https://api.elizacloud.ai",
+);
+const asJson = process.argv.includes("--json");
+const privateKey = process.env.PRIVATE_KEY?.trim() || undefined;
+
+const startedAt = Date.now();
+let session;
+try {
+  session = await siweTestLogin({ baseUrl, privateKey });
+} catch (err) {
+  console.error(
+    `[siwe-login] FAILED against ${baseUrl}:`,
+    err instanceof Error ? err.message : err,
+  );
+  process.exit(1);
+}
+const loginMs = Date.now() - startedAt;
+
+// Prove the key authorizes a real request.
+let balanceStatus = 0;
+let balance = null;
+try {
+  const res = await fetch(`${baseUrl}/api/v1/credits/balance`, {
+    headers: { authorization: `Bearer ${session.apiKey}` },
+  });
+  balanceStatus = res.status;
+  if (res.ok) balance = (await res.json())?.balance ?? null;
+} catch {
+  // network probe failure is non-fatal to the login itself
+}
+
+if (asJson) {
+  console.log(
+    JSON.stringify({ ...session, baseUrl, loginMs, balanceStatus, balance }),
+  );
+} else {
+  console.log(`[siwe-login] OK  ${baseUrl}`);
+  console.log(`  address      ${session.address}`);
+  console.log(`  userId       ${session.userId}`);
+  console.log(`  orgId        ${session.organizationId}`);
+  console.log(`  isNewAccount ${session.isNewAccount}`);
+  console.log(
+    `  apiKey       ${session.apiKey.slice(0, 12)}…(${session.apiKey.length} chars)`,
+  );
+  console.log(`  loginMs      ${loginMs}`);
+  console.log(
+    `  credits      ${balance} (GET /api/v1/credits/balance -> ${balanceStatus})`,
+  );
+}
+
+if (balanceStatus && balanceStatus !== 200) {
+  console.error(
+    `[siwe-login] WARN: API key did not authorize /api/v1/credits/balance (status ${balanceStatus})`,
+  );
+  process.exit(2);
+}

scripts/cloud/verify-8756-launch-hardening.mjs

@@ -0,0 +1,380 @@
+#!/usr/bin/env node
+/**
+ * Read-only evidence collector for the launch-hardening operator lane.
+ *
+ * Gathers the public GitHub state for #12081, superseding #8756, so reviewers
+ * can decide whether closeout is ready without printing secret values or
+ * mutating live infrastructure.
+ */
+import { spawnSync } from "node:child_process";
+import { mkdirSync, writeFileSync } from "node:fs";
+import path from "node:path";
+
+const arg = (name, fallback = null) => {
+  const index = process.argv.indexOf(name);
+  return index >= 0 ? process.argv[index + 1] : fallback;
+};
+
+const repo = arg("--repo", "elizaOS/eliza");
+const branch = arg("--branch", "develop");
+const environment = arg("--environment", "production");
+const image = arg("--image", "ghcr.io/elizaos/eliza:develop");
+const cloudSecretName = arg("--cloud-secret-name", "ELIZACLOUD_API_KEY");
+const cloudTokenEnvName = "ELIZA_CLOUD_AUTH_TOKEN";
+const outDir = arg("--out-dir", "reports/launch-hardening/8756-status");
+const cloudReport = arg("--cloud-report");
+
+function run(command, args, options = {}) {
+  const result = spawnSync(command, args, {
+    encoding: "utf8",
+    stdio: ["ignore", "pipe", "pipe"],
+    ...options,
+  });
+  return {
+    ok: result.status === 0,
+    status: result.status,
+    stdout: result.stdout.trim(),
+    stderr: result.stderr.trim(),
+  };
+}
+
+function runJson(command, args) {
+  const result = run(command, args);
+  if (!result.ok) return { ok: false, error: result.stderr || result.stdout };
+  try {
+    return { ok: true, value: JSON.parse(result.stdout) };
+  } catch (error) {
+    return {
+      ok: false,
+      error: `failed to parse JSON from ${command}: ${
+        error instanceof Error ? error.message : String(error)
+      }`,
+      raw: result.stdout,
+    };
+  }
+}
+
+function latestRun(workflow) {
+  const result = runJson("gh", [
+    "run",
+    "list",
+    "-R",
+    repo,
+    "--workflow",
+    workflow,
+    "--branch",
+    branch,
+    "--limit",
+    "1",
+    "--json",
+    "databaseId,status,conclusion,createdAt,updatedAt,headSha,displayTitle,event,url",
+  ]);
+  if (!result.ok) return { ok: false, error: result.error };
+  return { ok: true, run: result.value?.[0] ?? null };
+}
+
+function listEnvironmentSecrets() {
+  const result = runJson("gh", [
+    "secret",
+    "list",
+    "-R",
+    repo,
+    "--env",
+    environment,
+    "--json",
+    "name,updatedAt,visibility",
+  ]);
+  if (!result.ok) return { ok: false, error: result.error };
+  const expected = [
+    "SANDBOX_REGISTRY_REDIS_URL",
+    "ELIZA_PROVISIONING_HOST",
+    "ELIZA_PROVISIONING_SSH_KEY",
+  ];
+  const byName = new Map(
+    result.value.map((secret) => [
+      secret.name,
+      {
+        name: secret.name,
+        present: true,
+        updatedAt: secret.updatedAt ?? null,
+        visibility: secret.visibility ?? null,
+      },
+    ]),
+  );
+  return {
+    ok: true,
+    expected: expected.map(
+      (name) =>
+        byName.get(name) ?? {
+          name,
+          present: false,
+          updatedAt: null,
+          visibility: null,
+        },
+    ),
+    count: result.value.length,
+  };
+}
+
+function listRepoSecret(name) {
+  const result = runJson("gh", [
+    "secret",
+    "list",
+    "-R",
+    repo,
+    "--json",
+    "name,updatedAt,visibility",
+  ]);
+  if (!result.ok) {
+    return {
+      ok: false,
+      name,
+      present: Boolean(process.env[cloudTokenEnvName]),
+      updatedAt: null,
+      visibility: null,
+      error: result.error,
+      source: "env-fallback",
+    };
+  }
+  const found = result.value.find((secret) => secret.name === name);
+  return {
+    ok: true,
+    name,
+    present: Boolean(found),
+    updatedAt: found?.updatedAt ?? null,
+    visibility: found?.visibility ?? null,
+    source: "github-repo-secret-list",
+  };
+}
+
+function developHead() {
+  const result = runJson("gh", [
+    "api",
+    `repos/${repo}/git/ref/heads/${branch}`,
+    "--jq",
+    ".object.sha",
+  ]);
+  if (!result.ok) {
+    const fallback = run("gh", [
+      "api",
+      `repos/${repo}/git/ref/heads/${branch}`,
+      "--jq",
+      ".object.sha",
+    ]);
+    return {
+      ok: fallback.ok,
+      sha: fallback.ok ? fallback.stdout : null,
+      error: fallback.ok ? null : fallback.stderr || fallback.stdout,
+    };
+  }
+  return { ok: true, sha: result.value };
+}
+
+function inspectImageDigest() {
+  const result = run("docker", ["buildx", "imagetools", "inspect", image]);
+  if (!result.ok)
+    return { ok: false, image, error: result.stderr || result.stdout };
+  const digest =
+    result.stdout.match(/Digest:\s+(sha256:[a-f0-9]{64})/)?.[1] ?? null;
+  return {
+    ok: true,
+    image,
+    digest,
+    outputExcerpt: result.stdout.split("\n").slice(0, 12).join("\n"),
+  };
+}
+
+function readOptionalCloudReport() {
+  if (!cloudReport) return null;
+  const result = run("node", [
+    "packages/app/scripts/cloud-provisioning-e2e.mjs",
+    "--fresh-agent",
+    "--report",
+    cloudReport,
+  ]);
+  return {
+    attempted: true,
+    ok: result.ok,
+    reportPath: cloudReport,
+    stdout: result.stdout.slice(0, 2000),
+    stderr: result.stderr.slice(0, 2000),
+  };
+}
+
+function runSelfChecks() {
+  return {
+    provisioningWorkerEnvReconcile: run("bun", [
+      "test",
+      "packages/scripts/cloud/admin/daemons/provisioning-worker-env-reconcile.test.ts",
+    ]),
+  };
+}
+
+function verdict(report) {
+  const requiredSecretsPresent = report.environmentSecrets.expected.every(
+    (secret) => secret.present,
+  );
+  const imageReady =
+    report.workflows.buildAgentImage.run?.status === "completed" &&
+    report.workflows.buildAgentImage.run?.conclusion === "success";
+  const deployReady =
+    report.workflows.deployProvisioningWorker.run?.status === "completed" &&
+    report.workflows.deployProvisioningWorker.run?.conclusion === "success";
+  const cloudVerified = report.cloudProvisioning?.ok === true;
+  return {
+    requiredSecretsPresent,
+    imageReady,
+    deployReady,
+    cloudVerified,
+    closeable:
+      requiredSecretsPresent && imageReady && deployReady && cloudVerified,
+  };
+}
+
+function table(rows) {
+  return rows
+    .map(
+      (row) =>
+        `| ${row.map((cell) => String(cell).replaceAll("\n", "<br>")).join(" | ")} |`,
+    )
+    .join("\n");
+}
+
+function renderMarkdown(report) {
+  const v = report.verdict;
+  return `# #12081 / #8756 Launch-Hardening Status
+
+Generated: ${report.generatedAt}
+
+This is a read-only collector. It does not deploy, approve environments, SSH to
+hosts, or print secret values.
+
+## Verdict
+
+${table([
+  ["Check", "Status"],
+  [
+    "Required production environment secrets exist",
+    v.requiredSecretsPresent ? "yes" : "no",
+  ],
+  [
+    "Latest build-agent-image run for develop is green",
+    v.imageReady ? "yes" : "no",
+  ],
+  [
+    "Latest provisioning-worker deploy for develop is green",
+    v.deployReady ? "yes" : "no",
+  ],
+  [
+    "Fresh-agent cloud provisioning probe passed",
+    v.cloudVerified ? "yes" : "no / not run",
+  ],
+  ["Issue closeable from this evidence alone", v.closeable ? "yes" : "no"],
+])}
+
+## Develop Head
+
+- Branch: \`${report.branch}\`
+- SHA: \`${report.developHead.sha ?? "unknown"}\`
+
+## Environment Secrets (${report.environment})
+
+${table([
+  ["Secret", "Present", "Updated At"],
+  ...report.environmentSecrets.expected.map((secret) => [
+    secret.name,
+    secret.present ? "yes" : "no",
+    secret.updatedAt ?? "n/a",
+  ]),
+])}
+
+## Cloud Auth Secret
+
+${table([
+  ["Secret", "Present", "Updated At", "Source"],
+  [
+    report.cloudAuthSecret.name,
+    report.cloudAuthSecret.present ? "yes" : "no",
+    report.cloudAuthSecret.updatedAt ?? "n/a",
+    report.cloudAuthSecret.source,
+  ],
+])}
+
+## Workflows
+
+${table([
+  ["Workflow", "Run", "Status", "Conclusion", "Head SHA", "Updated"],
+  [
+    "Build Agent Image",
+    report.workflows.buildAgentImage.run?.url ?? "missing",
+    report.workflows.buildAgentImage.run?.status ?? "missing",
+    report.workflows.buildAgentImage.run?.conclusion ?? "",
+    report.workflows.buildAgentImage.run?.headSha ?? "",
+    report.workflows.buildAgentImage.run?.updatedAt ?? "",
+  ],
+  [
+    "Deploy Eliza Provisioning Worker",
+    report.workflows.deployProvisioningWorker.run?.url ?? "missing",
+    report.workflows.deployProvisioningWorker.run?.status ?? "missing",
+    report.workflows.deployProvisioningWorker.run?.conclusion ?? "",
+    report.workflows.deployProvisioningWorker.run?.headSha ?? "",
+    report.workflows.deployProvisioningWorker.run?.updatedAt ?? "",
+  ],
+])}
+
+## Image
+
+- Image: \`${report.image.image}\`
+- Digest: \`${report.image.digest ?? "unavailable"}\`
+- Inspect succeeded: ${report.image.ok ? "yes" : "no"}
+
+## Local Regression Check
+
+- \`provisioning-worker-env-reconcile.test.ts\`: ${
+    report.selfChecks.provisioningWorkerEnvReconcile.ok ? "passed" : "failed"
+  }
+
+## Fresh-Agent Probe
+
+${
+  report.cloudProvisioning
+    ? `- Attempted: yes
+- Passed: ${report.cloudProvisioning.ok ? "yes" : "no"}
+- Report path: \`${report.cloudProvisioning.reportPath}\``
+    : "- Attempted: no. Pass `--cloud-report <path>` with `ELIZA_CLOUD_AUTH_TOKEN` available to run the real fresh-agent probe."
+}
+
+## Remaining
+
+This evidence can close the #12081 operator lane only when all verdict rows are
+green. If the workflow rows are still pending, cancelled, or missing, the live
+operator lane has not been proven complete. If the fresh-agent probe is not run,
+persistence, lean-plugin loading, and runtime reachability remain unverified.
+`;
+}
+
+mkdirSync(outDir, { recursive: true });
+const report = {
+  generatedAt: new Date().toISOString(),
+  repo,
+  branch,
+  environment,
+  developHead: developHead(),
+  environmentSecrets: listEnvironmentSecrets(),
+  cloudAuthSecret: listRepoSecret(cloudSecretName),
+  workflows: {
+    buildAgentImage: latestRun("build-agent-image.yml"),
+    deployProvisioningWorker: latestRun("deploy-eliza-provisioning-worker.yml"),
+  },
+  image: inspectImageDigest(),
+  selfChecks: runSelfChecks(),
+  cloudProvisioning: readOptionalCloudReport(),
+};
+report.verdict = verdict(report);
+
+writeFileSync(
+  path.join(outDir, "status.json"),
+  `${JSON.stringify(report, null, 2)}\n`,
+);
+writeFileSync(path.join(outDir, "README.md"), renderMarkdown(report));
+console.log(JSON.stringify(report.verdict, null, 2));

scripts/e2e-recordings/audit-ui-coverage.mjs

@@ -0,0 +1,99 @@
+#!/usr/bin/env node
+/**
+ * Audits the unified UI E2E recording manifest.
+ *
+ * This does not run the browser suites. It verifies that every manifest entry
+ * points at a real package script and that known standalone UI Playwright
+ * packages are either runnable in the manifest or explicitly accounted for.
+ */
+
+import fs from "node:fs";
+import path from "node:path";
+import {
+  REPO_ROOT,
+  SKIPPED_EXTERNAL_UI_E2E_SUITES,
+  UI_E2E_COVERED_BY_APP,
+  UI_E2E_SUITES,
+} from "./suites.mjs";
+
+const REQUIRED_STANDALONE_UI_DIRS = [
+  "packages/app",
+  "packages/test/cloud-e2e",
+  "packages/homepage",
+  "packages/os/homepage",
+  "packages/os/usb-installer",
+  "packages/ui",
+  "packages/feed/tools/dag-visualizer",
+];
+
+const coveredDirs = new Set([
+  ...UI_E2E_SUITES.map((suite) => suite.configDir),
+  ...UI_E2E_COVERED_BY_APP.map((suite) => suite.configDir),
+  ...SKIPPED_EXTERNAL_UI_E2E_SUITES.map((suite) => suite.configDir),
+]);
+
+function readPackageJson(configDir) {
+  const packageJsonPath = path.join(REPO_ROOT, configDir, "package.json");
+  if (!fs.existsSync(packageJsonPath)) {
+    throw new Error(`${configDir}: missing package.json`);
+  }
+  return JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));
+}
+
+function fail(message) {
+  console.error(`[ui-e2e-audit] ERROR ${message}`);
+  process.exitCode = 1;
+}
+
+for (const suite of UI_E2E_SUITES) {
+  try {
+    if (suite.command) {
+      const commandScript = suite.command[1];
+      if (
+        !commandScript ||
+        !fs.existsSync(path.join(REPO_ROOT, commandScript))
+      ) {
+        fail(`${suite.name}: command script is missing (${commandScript})`);
+      }
+      if (suite.checkCommand) {
+        const checkScript = suite.checkCommand[1];
+        if (!checkScript || !fs.existsSync(path.join(REPO_ROOT, checkScript))) {
+          fail(`${suite.name}: check script is missing (${checkScript})`);
+        }
+      }
+    } else {
+      const packageJson = readPackageJson(suite.configDir);
+      if (!packageJson.scripts?.[suite.script]) {
+        fail(
+          `${suite.name}: ${suite.configDir} has no "${suite.script}" script`,
+        );
+      }
+    }
+    if (!suite.coverage || suite.coverage.length < 40) {
+      fail(`${suite.name}: coverage description is missing or too terse`);
+    }
+  } catch (error) {
+    fail(error instanceof Error ? error.message : String(error));
+  }
+}
+
+for (const configDir of REQUIRED_STANDALONE_UI_DIRS) {
+  if (!coveredDirs.has(configDir)) {
+    fail(`${configDir}: standalone UI package is not in the E2E manifest`);
+  }
+}
+
+const duplicateNames = UI_E2E_SUITES.map((suite) => suite.name).filter(
+  (name, index, all) => all.indexOf(name) !== index,
+);
+if (duplicateNames.length > 0) {
+  fail(`duplicate suite names: ${[...new Set(duplicateNames)].join(", ")}`);
+}
+
+if (process.exitCode) {
+  process.exit(process.exitCode);
+}
+
+console.log(
+  `[ui-e2e-audit] ${UI_E2E_SUITES.length} runnable UI E2E suites audited; ${UI_E2E_COVERED_BY_APP.length} covered-through-app entries; ${SKIPPED_EXTERNAL_UI_E2E_SUITES.length} external suites documented.`,
+);

scripts/e2e-recordings/capture-android-emu.mjs

@@ -0,0 +1,261 @@
+#!/usr/bin/env node
+/**
+ * Captures Android emulator e2e evidence: boots (or reuses) an AVD, installs the
+ * app APK, drives it against a host agent, and collects a screenshot plus logcat
+ * into the generated native-capture output. Exits with SKIP_EXIT_CODE (77) when
+ * no emulator is available. Shares arg parsing and manifest writing with the
+ * other native capture scripts via native-capture-common.mjs.
+ */
+import fs from "node:fs";
+import path from "node:path";
+import {
+  captureAndroidLogcat,
+  captureAndroidScreenshot,
+} from "../../packages/app/scripts/lib/android-capture.mjs";
+import {
+  ensureEmulatorBooted,
+  ensureEmulatorPermissive,
+  installApk,
+  listAvds,
+  listDevices,
+  resolveAdb,
+  resolveApk,
+  resolveEmulator,
+} from "../../packages/app/scripts/lib/android-device.mjs";
+import {
+  argValue,
+  copyArtifact,
+  createCaptureLog,
+  hasArg,
+  resolveCapturePaths,
+  runCommandWithLog,
+  SKIP_EXIT_CODE,
+  startDeviceE2EHostAgent,
+  writeCaptureManifest,
+} from "./native-capture-common.mjs";
+
+const REPO_ROOT = path.resolve(import.meta.dirname, "..", "..");
+const APP_DIR = path.join(REPO_ROOT, "packages", "app");
+const args = process.argv.slice(2);
+
+function androidAvailabilityReason() {
+  let adb;
+  try {
+    adb = resolveAdb();
+  } catch (error) {
+    return error instanceof Error ? error.message : String(error);
+  }
+
+  const requestedSerial = argValue(
+    args,
+    "--serial",
+    process.env.ANDROID_SERIAL,
+  );
+  if (requestedSerial && !requestedSerial.startsWith("emulator-")) {
+    return `requested Android serial ${requestedSerial} is not an emulator`;
+  }
+
+  const devices = listDevices(adb);
+  if (
+    requestedSerial
+      ? devices.includes(requestedSerial)
+      : devices.some((serial) => serial.startsWith("emulator-"))
+  ) {
+    return null;
+  }
+
+  const emulator = resolveEmulator();
+  if (!emulator) {
+    return "No Android emulator is attached and the emulator binary was not found.";
+  }
+  const avds = listAvds(emulator);
+  if (avds.length === 0) {
+    return "No Android AVD is configured. Create one with Android Studio Device Manager.";
+  }
+  return null;
+}
+
+async function selectOrBootEmulator(adb, log) {
+  const requestedSerial = argValue(
+    args,
+    "--serial",
+    process.env.ANDROID_SERIAL,
+  );
+  if (requestedSerial) return requestedSerial;
+  const existing = listDevices(adb).find((serial) =>
+    serial.startsWith("emulator-"),
+  );
+  if (existing) {
+    log(`reusing attached Android emulator ${existing}`);
+    return existing;
+  }
+  return ensureEmulatorBooted({
+    adb,
+    avd: argValue(args, "--avd"),
+    log,
+  });
+}
+
+async function ensureDebugApk(logSink) {
+  try {
+    const apk = resolveApk(process.env.ELIZA_ANDROID_APK);
+    logSink.log(`using debug APK ${apk}`);
+    return apk;
+  } catch (error) {
+    if (
+      hasArg(args, "--skip-build") ||
+      process.env.ANDROID_CAPTURE_SKIP_BUILD === "1"
+    ) {
+      throw error;
+    }
+  }
+
+  await runCommandWithLog(
+    "bun",
+    ["run", "--cwd", "packages/app", "build:android"],
+    {
+      cwd: REPO_ROOT,
+      env: {
+        ELIZA_MOBILE_REPO_ROOT: REPO_ROOT,
+        ELIZA_WEBVIEW_DEBUG: "1",
+        ELIZA_BUN_RISCV64_OPTIONAL: "1",
+      },
+      logSink,
+      label: "bun run --cwd packages/app build:android",
+    },
+  );
+  const apk = resolveApk(process.env.ELIZA_ANDROID_APK);
+  logSink.log(`using debug APK ${apk}`);
+  return apk;
+}
+
+async function main() {
+  const reason = androidAvailabilityReason();
+  if (hasArg(args, "--check")) {
+    if (reason) {
+      console.log(reason);
+      process.exit(SKIP_EXIT_CODE);
+    }
+    console.log("Android emulator capture prerequisites are available.");
+    return;
+  }
+  if (reason) {
+    console.log(reason);
+    process.exit(SKIP_EXIT_CODE);
+  }
+
+  const { prefix, evidenceDir, recordingResultDir } = resolveCapturePaths({
+    repoRoot: REPO_ROOT,
+    platform: "android-emu",
+    slug: "android-emu-capture",
+    args,
+  });
+  fs.rmSync(recordingResultDir, { recursive: true, force: true });
+  fs.mkdirSync(recordingResultDir, { recursive: true });
+
+  const logSink = createCaptureLog(
+    path.join(evidenceDir, `${prefix}-capture.log`),
+    "android-emu-capture",
+  );
+  let hostAgent = null;
+  try {
+    const adb = resolveAdb();
+    const serial = await selectOrBootEmulator(adb, logSink.log);
+    process.env.ANDROID_SERIAL = serial;
+    await ensureEmulatorPermissive(adb, serial, { log: logSink.log });
+    const apk = await ensureDebugApk(logSink);
+    logSink.log(`installing debug APK on ${serial}: ${apk}`);
+    installApk(adb, serial, apk);
+
+    hostAgent = await startDeviceE2EHostAgent({
+      repoRoot: REPO_ROOT,
+      logSink,
+    });
+
+    await runCommandWithLog(
+      "bun",
+      ["run", "--cwd", "packages/app", "test:e2e:android:onboarding"],
+      {
+        cwd: REPO_ROOT,
+        env: {
+          ANDROID_SERIAL: serial,
+          ELIZA_ANDROID_BACKEND: "host",
+          ELIZA_ANDROID_REQUIRE_AGENT: "1",
+        },
+        logSink,
+        label: "bun run --cwd packages/app test:e2e:android:onboarding",
+      },
+    );
+
+    const sourceDir = path.join(
+      APP_DIR,
+      "test-results",
+      "android-onboarding-to-home",
+    );
+    const homeScreenshot = copyArtifact(
+      path.join(sourceDir, "home-landing.png"),
+      evidenceDir,
+      `${prefix}-home-landing.png`,
+    );
+    const walkthrough = copyArtifact(
+      path.join(sourceDir, "onboarding-to-home.mp4"),
+      evidenceDir,
+      `${prefix}-onboarding-to-home.mp4`,
+    );
+    const deviceScreenshot = captureAndroidScreenshot({
+      adb,
+      serial,
+      artifactDir: evidenceDir,
+      filename: `${prefix}-device-final.png`,
+      log: logSink.log,
+    });
+    const logcat = captureAndroidLogcat({
+      adb,
+      serial,
+      artifactDir: evidenceDir,
+      filename: `${prefix}-logcat.txt`,
+      log: logSink.log,
+    });
+
+    copyArtifact(homeScreenshot, recordingResultDir, "home-landing.png");
+    copyArtifact(deviceScreenshot, recordingResultDir, "device-final.png");
+    copyArtifact(walkthrough, recordingResultDir, "onboarding-to-home.mp4");
+    copyArtifact(logcat, recordingResultDir, "logcat.txt", {
+      required: false,
+    });
+    copyArtifact(logSink.logPath, recordingResultDir, "capture.log", {
+      required: false,
+    });
+    const manifest = {
+      platform: "android-emu",
+      serial,
+      evidenceDir,
+      artifacts: {
+        homeScreenshot,
+        deviceScreenshot,
+        walkthrough,
+        logcat,
+        captureLog: logSink.logPath,
+      },
+    };
+    writeCaptureManifest(
+      path.join(recordingResultDir, "manifest.json"),
+      manifest,
+    );
+    writeCaptureManifest(path.join(evidenceDir, "manifest.json"), manifest);
+    logSink.log(`capture artifacts written to ${evidenceDir}`);
+  } catch (error) {
+    logSink.log(
+      `FAILED: ${error instanceof Error ? error.message : String(error)}`,
+    );
+    throw error;
+  } finally {
+    await hostAgent?.stop();
+    logSink?.close();
+  }
+}
+
+main().catch((error) => {
+  console.error(error instanceof Error ? error.stack || error.message : error);
+  process.exit(1);
+});

scripts/e2e-recordings/capture-ios-sim.mjs

@@ -0,0 +1,277 @@
+#!/usr/bin/env node
+/**
+ * Captures iOS Simulator e2e evidence: boots a simulator, builds and installs
+ * the app (unless --skip-build), drives it against a host agent, and collects a
+ * screenshot plus video into the generated native-capture output. Exits with
+ * SKIP_EXIT_CODE (77) when no simulator is available. Shares arg parsing and
+ * manifest writing with the other native capture scripts via
+ * native-capture-common.mjs.
+ */
+import fs from "node:fs";
+import path from "node:path";
+import {
+  captureIosSimulatorScreenshot,
+  ensureBootedIosSimulator,
+  iosSimulatorAvailabilityReason,
+  startIosSimulatorVideo,
+} from "../../packages/app/scripts/lib/ios-simulator-capture.mjs";
+import {
+  argValue,
+  copyArtifact,
+  createCaptureLog,
+  hasArg,
+  resolveCapturePaths,
+  runCommandWithLog,
+  SKIP_EXIT_CODE,
+  startDeviceE2EHostAgent,
+  writeCaptureManifest,
+} from "./native-capture-common.mjs";
+
+const REPO_ROOT = path.resolve(import.meta.dirname, "..", "..");
+const APP_DIR = path.join(REPO_ROOT, "packages", "app");
+const args = process.argv.slice(2);
+
+async function maybeBuildIosSimulatorApp(logSink) {
+  if (
+    hasArg(args, "--skip-build") ||
+    process.env.IOS_CAPTURE_SKIP_BUILD === "1"
+  ) {
+    logSink.log("skipping iOS simulator build");
+    return;
+  }
+  await runCommandWithLog(
+    "bun",
+    ["run", "--cwd", "packages/app", "build:ios:local:sim"],
+    {
+      cwd: REPO_ROOT,
+      logSink,
+      label: "bun run --cwd packages/app build:ios:local:sim",
+    },
+  );
+}
+
+async function main() {
+  const deviceName = argValue(args, "--device", process.env.IOS_SIMULATOR_NAME);
+  const reason = iosSimulatorAvailabilityReason({ deviceName });
+  if (hasArg(args, "--check")) {
+    if (reason) {
+      console.log(reason);
+      process.exit(SKIP_EXIT_CODE);
+    }
+    console.log("iOS simulator capture prerequisites are available.");
+    return;
+  }
+  if (reason) {
+    console.log(reason);
+    process.exit(SKIP_EXIT_CODE);
+  }
+
+  const { prefix, evidenceDir, recordingResultDir } = resolveCapturePaths({
+    repoRoot: REPO_ROOT,
+    platform: "ios-sim",
+    slug: "ios-sim-capture",
+    args,
+  });
+  fs.rmSync(recordingResultDir, { recursive: true, force: true });
+  fs.mkdirSync(recordingResultDir, { recursive: true });
+
+  const logSink = createCaptureLog(
+    path.join(evidenceDir, `${prefix}-capture.log`),
+    "ios-sim-capture",
+  );
+  const sourceDir = path.join(
+    APP_DIR,
+    "test-results",
+    "ios-onboarding-to-home",
+  );
+  let hostAgent = null;
+  let recording = null;
+  let udid = null;
+  try {
+    udid = ensureBootedIosSimulator({ deviceName, log: logSink.log });
+    await maybeBuildIosSimulatorApp(logSink);
+    hostAgent = await startDeviceE2EHostAgent({
+      repoRoot: REPO_ROOT,
+      logSink,
+    });
+
+    recording = startIosSimulatorVideo({
+      target: "booted",
+      artifactDir: evidenceDir,
+      filename: `${prefix}-onboarding-to-home.mov`,
+      log: logSink.log,
+    });
+
+    await runCommandWithLog(
+      process.execPath,
+      [
+        "packages/app/scripts/ios-onboarding-smoke.mjs",
+        "--no-video",
+        "--api-base",
+        "http://127.0.0.1:31337",
+      ],
+      {
+        cwd: REPO_ROOT,
+        logSink,
+        label:
+          "node packages/app/scripts/ios-onboarding-smoke.mjs --no-video --api-base http://127.0.0.1:31337",
+      },
+    );
+
+    const video = await recording.stop();
+    recording = null;
+    if (!video) throw new Error("iOS simulator recording was not written.");
+
+    const finalScreenshot = captureIosSimulatorScreenshot({
+      target: "booted",
+      artifactDir: evidenceDir,
+      filename: `${prefix}-home-landing.png`,
+      log: logSink.log,
+    });
+
+    const freshScreenshot = copyArtifact(
+      path.join(sourceDir, "fresh-onboarding.png"),
+      evidenceDir,
+      `${prefix}-fresh-onboarding.png`,
+    );
+    const smokeHomeScreenshot = copyArtifact(
+      path.join(sourceDir, "home-landing.png"),
+      evidenceDir,
+      `${prefix}-smoke-home-landing.png`,
+    );
+    const resultJson = copyArtifact(
+      path.join(sourceDir, "result.json"),
+      evidenceDir,
+      `${prefix}-result.json`,
+      { required: false },
+    );
+
+    copyArtifact(freshScreenshot, recordingResultDir, "fresh-onboarding.png");
+    copyArtifact(
+      smokeHomeScreenshot,
+      recordingResultDir,
+      "smoke-home-landing.png",
+    );
+    copyArtifact(finalScreenshot, recordingResultDir, "home-landing.png");
+    copyArtifact(video, recordingResultDir, "onboarding-to-home.mov");
+    copyArtifact(resultJson, recordingResultDir, "result.json", {
+      required: false,
+    });
+    copyArtifact(logSink.logPath, recordingResultDir, "capture.log", {
+      required: false,
+    });
+    const manifest = {
+      platform: "ios-sim",
+      udid,
+      evidenceDir,
+      artifacts: {
+        freshScreenshot,
+        smokeHomeScreenshot,
+        finalScreenshot,
+        walkthrough: video,
+        resultJson,
+        captureLog: logSink.logPath,
+      },
+    };
+    writeCaptureManifest(
+      path.join(recordingResultDir, "manifest.json"),
+      manifest,
+    );
+    writeCaptureManifest(path.join(evidenceDir, "manifest.json"), manifest);
+    logSink.log(`capture artifacts written to ${evidenceDir}`);
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : String(error);
+    logSink.log(`FAILED: ${errorMessage}`);
+    const video = recording ? await recording.stop() : null;
+    recording = null;
+    let failureDeviceScreenshot = null;
+    try {
+      failureDeviceScreenshot = captureIosSimulatorScreenshot({
+        target: "booted",
+        artifactDir: evidenceDir,
+        filename: `${prefix}-failure-final.png`,
+        log: logSink.log,
+      });
+    } catch (screenshotError) {
+      logSink.log(
+        `failed to capture final failure screenshot: ${
+          screenshotError instanceof Error
+            ? screenshotError.message
+            : String(screenshotError)
+        }`,
+      );
+    }
+    const freshScreenshot = copyArtifact(
+      path.join(sourceDir, "fresh-onboarding.png"),
+      evidenceDir,
+      `${prefix}-fresh-onboarding.png`,
+      { required: false },
+    );
+    const smokeFailureScreenshot = copyArtifact(
+      path.join(sourceDir, "failure.png"),
+      evidenceDir,
+      `${prefix}-smoke-failure.png`,
+      { required: false },
+    );
+    const resultJson = copyArtifact(
+      path.join(sourceDir, "result.json"),
+      evidenceDir,
+      `${prefix}-result.json`,
+      { required: false },
+    );
+    copyArtifact(freshScreenshot, recordingResultDir, "fresh-onboarding.png", {
+      required: false,
+    });
+    copyArtifact(
+      smokeFailureScreenshot,
+      recordingResultDir,
+      "smoke-failure.png",
+      { required: false },
+    );
+    copyArtifact(
+      failureDeviceScreenshot,
+      recordingResultDir,
+      "failure-final.png",
+      { required: false },
+    );
+    copyArtifact(video, recordingResultDir, "onboarding-to-home.mov", {
+      required: false,
+    });
+    copyArtifact(resultJson, recordingResultDir, "result.json", {
+      required: false,
+    });
+    copyArtifact(logSink.logPath, recordingResultDir, "capture.log", {
+      required: false,
+    });
+    const manifest = {
+      platform: "ios-sim",
+      status: "failed",
+      error: errorMessage,
+      udid,
+      evidenceDir,
+      artifacts: {
+        freshScreenshot,
+        smokeFailureScreenshot,
+        failureDeviceScreenshot,
+        walkthrough: video,
+        resultJson,
+        captureLog: logSink.logPath,
+      },
+    };
+    writeCaptureManifest(
+      path.join(recordingResultDir, "manifest.json"),
+      manifest,
+    );
+    writeCaptureManifest(path.join(evidenceDir, "manifest.json"), manifest);
+    throw error;
+  } finally {
+    if (recording) await recording.stop();
+    await hostAgent?.stop();
+    logSink?.close();
+  }
+}
+
+main().catch((error) => {
+  console.error(error instanceof Error ? error.stack || error.message : error);
+  process.exit(1);
+});

scripts/e2e-recordings/generate-contact-sheets.mjs

@@ -0,0 +1,374 @@
+#!/usr/bin/env node
+
+/**
+ * generate-contact-sheets.mjs
+ *
+ * Scans e2e-recordings/*\/test-results/ for Playwright test output,
+ * extracts trace.zip files, copies non-blank frames to
+ * e2e-recordings/contact-sheets/<package>/<test-slug>/frames/, and
+ * writes e2e-recordings/manifest.json.
+ *
+ * Blank frames (uniform white/black — JPEG < 5 KB) are skipped.
+ * No per-test HTML is generated; use generate-viewer.mjs for the index.
+ *
+ * Usage:
+ *   node scripts/e2e-recordings/generate-contact-sheets.mjs
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const RECORDINGS_DIR = path.join(REPO_ROOT, "e2e-recordings");
+const CONTACT_SHEETS_DIR = path.join(RECORDINGS_DIR, "contact-sheets");
+const MANIFEST_PATH = path.join(RECORDINGS_DIR, "manifest.json");
+const RM_RECURSIVE_SCRIPT = path.join(
+  REPO_ROOT,
+  "packages",
+  "scripts",
+  "rm-path-recursive.mjs",
+);
+
+// 500 = only filter corrupted/empty files; dark/sparse real frames can be 2-4 KB
+const MIN_FRAME_BYTES = 500;
+
+function toSlug(name) {
+  return name
+    .toLowerCase()
+    .replace(/[^a-z0-9]+/g, "-")
+    .replace(/^-+|-+$/g, "");
+}
+
+const INTERESTING_METHODS = new Set([
+  "goto",
+  "click",
+  "fill",
+  "hover",
+  "press",
+  "check",
+  "uncheck",
+  "selectOption",
+  "selectText",
+  "waitForSelector",
+  "waitForURL",
+  "waitForLoadState",
+  "screenshot",
+  "tap",
+  "dblclick",
+  "dragTo",
+  "dispatchEvent",
+  "setInputFiles",
+  "type",
+]);
+
+/**
+ * Extract frames from a trace.zip.
+ * Returns [{ screenshotSrc: string|null }, ...] with _tmpDir attached.
+ */
+function extractTraceFrames(zipPath) {
+  const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "pw-trace-"));
+  try {
+    const result = spawnSync("unzip", ["-q", "-o", zipPath, "-d", tmpDir], {
+      encoding: "utf8",
+    });
+    if (result.status !== 0) {
+      console.warn(`  [warn] unzip failed for ${zipPath}: ${result.stderr}`);
+      return [];
+    }
+
+    const traceFiles = fs.existsSync(tmpDir)
+      ? fs
+          .readdirSync(tmpDir)
+          .filter((f) => /^\d+-trace\.trace$/.test(f))
+          .sort()
+      : [];
+
+    if (traceFiles.length === 0) {
+      console.warn(`  [warn] no *-trace.trace found in ${zipPath}`);
+      return [];
+    }
+
+    const allEntries = [];
+    for (const tf of traceFiles) {
+      const lines = fs.readFileSync(path.join(tmpDir, tf), "utf8").split("\n");
+      for (const line of lines) {
+        const trimmed = line.trim();
+        if (!trimmed) continue;
+        try {
+          allEntries.push(JSON.parse(trimmed));
+        } catch {
+          /* skip */
+        }
+      }
+    }
+
+    const screencasts = allEntries
+      .filter((e) => e.type === "screencast-frame" && e.sha1)
+      .sort((a, b) => (a.timestamp ?? 0) - (b.timestamp ?? 0));
+
+    const afterMap = new Map();
+    for (const e of allEntries) {
+      if (e.type === "after" && e.callId) afterMap.set(e.callId, e);
+    }
+
+    const actions = allEntries.filter(
+      (e) =>
+        e.type === "before" && e.method && INTERESTING_METHODS.has(e.method),
+    );
+
+    const frames = [];
+    for (const action of actions) {
+      const afterEntry = afterMap.get(action.callId);
+      const startTime = action.startTime ?? 0;
+      const endTime = afterEntry?.endTime ?? startTime + 10000;
+      // For goto actions, allow frames up to 2s after endTime for async renders
+      const extendedEndTime =
+        action.method === "goto" ? endTime + 2000 : endTime;
+
+      let bestFrame = null;
+      for (let i = screencasts.length - 1; i >= 0; i--) {
+        const f = screencasts[i];
+        if (
+          (f.timestamp ?? 0) <= extendedEndTime &&
+          (f.timestamp ?? 0) >= startTime
+        ) {
+          bestFrame = f;
+          break;
+        }
+      }
+      if (!bestFrame) {
+        bestFrame =
+          screencasts.find((f) => (f.timestamp ?? 0) >= startTime) ?? null;
+      }
+
+      let screenshotSrc = null;
+      if (bestFrame) {
+        const candidate = path.join(tmpDir, "resources", bestFrame.sha1);
+        if (fs.existsSync(candidate)) screenshotSrc = candidate;
+      }
+
+      frames.push({ screenshotSrc });
+    }
+
+    // If no actions matched, fall back to first and last screencast frames
+    if (frames.length === 0 && screencasts.length > 0) {
+      const addFrame = (sc) => {
+        const candidate = path.join(tmpDir, "resources", sc.sha1);
+        if (fs.existsSync(candidate)) frames.push({ screenshotSrc: candidate });
+      };
+      addFrame(screencasts[0]);
+      if (screencasts.length > 1) addFrame(screencasts[screencasts.length - 1]);
+    }
+
+    frames._tmpDir = tmpDir;
+    return frames;
+  } catch (err) {
+    console.warn(`  [warn] error extracting ${zipPath}: ${err.message}`);
+    return [];
+  }
+}
+
+function rmRecursive(targetPath) {
+  const result = spawnSync(
+    process.execPath,
+    [RM_RECURSIVE_SCRIPT, targetPath],
+    {
+      cwd: REPO_ROOT,
+      encoding: "utf8",
+    },
+  );
+  if (result.status !== 0) {
+    const detail = result.stderr || result.stdout || `exit ${result.status}`;
+    throw new Error(`recursive cleanup failed for ${targetPath}: ${detail}`);
+  }
+}
+
+function processTestDir(testResultDir, packageName) {
+  const testDirName = path.basename(testResultDir);
+  const slug = toSlug(testDirName);
+  const outDir = path.join(CONTACT_SHEETS_DIR, packageName, slug);
+  const framesDir = path.join(outDir, "frames");
+
+  const zipNames = fs
+    .readdirSync(testResultDir)
+    .filter((f) => /^trace(-\d+)?\.zip$/.test(f))
+    .sort();
+
+  const videoFiles = fs
+    .readdirSync(testResultDir)
+    .filter((f) => /\.(webm|mp4|mov)$/i.test(f));
+  const videoFile = videoFiles[0] ?? null;
+
+  const screenshotFiles = fs
+    .readdirSync(testResultDir)
+    .filter((f) => /\.(png|jpe?g)$/i.test(f))
+    .sort();
+
+  if (zipNames.length === 0 && !videoFile && screenshotFiles.length === 0) {
+    return null;
+  }
+
+  fs.mkdirSync(framesDir, { recursive: true });
+
+  const copiedRelPaths = [];
+  const tmpDirs = [];
+  let frameIdx = 0;
+
+  for (const zipName of zipNames) {
+    const zipPath = path.join(testResultDir, zipName);
+    const frames = extractTraceFrames(zipPath);
+    if (frames._tmpDir) tmpDirs.push(frames._tmpDir);
+
+    for (const frame of frames) {
+      if (!frame.screenshotSrc) continue;
+
+      // Skip uniform white/black flash frames
+      try {
+        const size = fs.statSync(frame.screenshotSrc).size;
+        if (size < MIN_FRAME_BYTES) continue;
+      } catch {
+        continue;
+      }
+
+      const ext = path.extname(frame.screenshotSrc) || ".jpeg";
+      const destName = `${String(frameIdx).padStart(4, "0")}${ext}`;
+      const destPath = path.join(framesDir, destName);
+      try {
+        fs.copyFileSync(frame.screenshotSrc, destPath);
+        const relOutDir = path.relative(RECORDINGS_DIR, outDir);
+        copiedRelPaths.push(path.join(relOutDir, "frames", destName));
+        frameIdx++;
+      } catch (err) {
+        console.warn(`  [warn] could not copy frame: ${err.message}`);
+      }
+    }
+  }
+
+  for (const tmpDir of tmpDirs) {
+    try {
+      rmRecursive(tmpDir);
+    } catch {}
+  }
+
+  for (const screenshotName of screenshotFiles) {
+    const screenshotPath = path.join(testResultDir, screenshotName);
+    try {
+      if (fs.statSync(screenshotPath).size < MIN_FRAME_BYTES) continue;
+      const ext = path.extname(screenshotName) || ".png";
+      const destName = `${String(frameIdx).padStart(4, "0")}${ext}`;
+      const destPath = path.join(framesDir, destName);
+      fs.copyFileSync(screenshotPath, destPath);
+      const relOutDir = path.relative(RECORDINGS_DIR, outDir);
+      copiedRelPaths.push(path.join(relOutDir, "frames", destName));
+      frameIdx++;
+    } catch (err) {
+      console.warn(`  [warn] could not copy screenshot: ${err.message}`);
+    }
+  }
+
+  const relVideo = videoFile
+    ? path.relative(RECORDINGS_DIR, path.join(testResultDir, videoFile))
+    : null;
+
+  return {
+    name: testDirName,
+    slug,
+    package: packageName,
+    resultDir: path.relative(RECORDINGS_DIR, testResultDir),
+    video: relVideo,
+    frameCount: copiedRelPaths.length,
+    frames: copiedRelPaths,
+  };
+}
+
+function findPackageDirs() {
+  if (!fs.existsSync(RECORDINGS_DIR)) return [];
+  return fs
+    .readdirSync(RECORDINGS_DIR)
+    .filter((name) => {
+      if (
+        name === "contact-sheets" ||
+        name === "manifest.json" ||
+        name === "index.html"
+      )
+        return false;
+      const full = path.join(RECORDINGS_DIR, name);
+      return fs.statSync(full).isDirectory();
+    })
+    .map((name) => ({ name, full: path.join(RECORDINGS_DIR, name) }));
+}
+
+function findTestResultDirs(packageRecordingDir) {
+  const testResultsDir = path.join(packageRecordingDir, "test-results");
+  if (!fs.existsSync(testResultsDir)) return [];
+  return fs
+    .readdirSync(testResultsDir)
+    .map((name) => path.join(testResultsDir, name))
+    .filter((full) => fs.statSync(full).isDirectory());
+}
+
+async function main() {
+  console.log("Scanning e2e-recordings for Playwright test output…");
+
+  const packageDirs = findPackageDirs();
+  if (packageDirs.length === 0) {
+    console.log("No package recording directories found under e2e-recordings/");
+    console.log(
+      "Expected structure: e2e-recordings/<package>/test-results/<test-dir>/",
+    );
+    return;
+  }
+
+  const manifest = { generated: new Date().toISOString(), packages: {} };
+
+  for (const { name: packageName, full: packageDir } of packageDirs) {
+    console.log(`\nPackage: ${packageName}`);
+    const testDirs = findTestResultDirs(packageDir);
+    if (testDirs.length === 0) {
+      console.log("  No test-results/ directory found.");
+      continue;
+    }
+
+    const tests = [];
+    for (const testDir of testDirs) {
+      console.log(`  Processing: ${path.basename(testDir)}`);
+      try {
+        const meta = processTestDir(testDir, packageName);
+        if (meta) {
+          tests.push(meta);
+          console.log(
+            `    → ${meta.frameCount} real frames (blank frames filtered)`,
+          );
+        } else {
+          console.log("    → skipped (no trace or video)");
+        }
+      } catch (err) {
+        console.warn(`  [error] ${testDir}: ${err.message}`);
+      }
+    }
+
+    if (tests.length > 0) manifest.packages[packageName] = { tests };
+  }
+
+  fs.writeFileSync(MANIFEST_PATH, JSON.stringify(manifest, null, 2), "utf8");
+  console.log(`\nManifest written: ${MANIFEST_PATH}`);
+
+  const totalTests = Object.values(manifest.packages).reduce(
+    (sum, pkg) => sum + pkg.tests.length,
+    0,
+  );
+  console.log(
+    `Done. ${totalTests} test(s) processed across ${Object.keys(manifest.packages).length} package(s).`,
+  );
+}
+
+main().catch((err) => {
+  console.error("Fatal error:", err);
+  process.exit(1);
+});

scripts/e2e-recordings/generate-viewer.mjs

@@ -0,0 +1,526 @@
+#!/usr/bin/env node
+/**
+ * generate-viewer.mjs
+ *
+ * Reads e2e-recordings/manifest.json and writes e2e-recordings/index.html —
+ * ONE self-contained dark-themed page showing every test as a card with:
+ *   - a horizontal filmstrip of all real frames (lazy-loaded)
+ *   - a full-screen lightbox on frame click (← → Esc navigation)
+ *   - a video link when a recording exists
+ *   - package filter tabs + search
+ *
+ * No per-test contact-sheet.html files are linked or generated.
+ *
+ * Usage:
+ *   node scripts/e2e-recordings/generate-viewer.mjs
+ */
+
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const RECORDINGS_DIR = path.join(REPO_ROOT, "e2e-recordings");
+const MANIFEST_PATH = path.join(RECORDINGS_DIR, "manifest.json");
+const OUTPUT_PATH = path.join(RECORDINGS_DIR, "index.html");
+
+function esc(str) {
+  return String(str ?? "")
+    .replace(/&/g, "&")
+    .replace(/</g, "&lt;")
+    .replace(/>/g, "&gt;")
+    .replace(/"/g, "&quot;");
+}
+
+function jsonStr(obj) {
+  return JSON.stringify(obj).replace(/<\//g, "<\\/");
+}
+
+function buildHtml(manifest) {
+  const packages = manifest.packages ?? {};
+  const packageNames = Object.keys(packages).sort();
+
+  const allTests = [];
+  for (const pkgName of packageNames) {
+    for (const test of packages[pkgName]?.tests ?? []) {
+      allTests.push({ ...test, package: test.package ?? pkgName });
+    }
+  }
+
+  const testsJson = jsonStr(allTests);
+  const packageNamesJson = jsonStr(packageNames);
+
+  return `<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>E2E Recordings</title>
+<style>
+  *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
+
+  :root {
+    --bg: #0d0d0d;
+    --surface: #161616;
+    --surface2: #1e1e1e;
+    --border: #2a2a2a;
+    --text: #e0e0e0;
+    --muted: #888;
+    --accent: #ff6600;
+    --accent-dk: #cc5200;
+    --radius: 8px;
+  }
+
+  html, body {
+    background: var(--bg);
+    color: var(--text);
+    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
+    min-height: 100vh;
+  }
+
+  /* ── Header ── */
+  .page-header {
+    padding: 24px 32px 0;
+    border-bottom: 1px solid var(--border);
+  }
+  .page-header h1 {
+    font-size: 1.35rem;
+    font-weight: 700;
+    color: #fff;
+    display: flex;
+    align-items: center;
+    gap: 10px;
+  }
+  .page-header h1 .logo {
+    width: 16px; height: 16px;
+    background: var(--accent);
+    border-radius: 3px;
+    flex-shrink: 0;
+  }
+  .page-header .meta {
+    font-size: 0.75rem;
+    color: var(--muted);
+    margin: 6px 0 16px;
+  }
+
+  /* ── Toolbar ── */
+  .toolbar {
+    display: flex;
+    align-items: center;
+    gap: 12px;
+    flex-wrap: wrap;
+    padding: 14px 32px;
+    border-bottom: 1px solid var(--border);
+    background: var(--surface);
+    position: sticky;
+    top: 0;
+    z-index: 10;
+  }
+  .filter-tabs { display: flex; gap: 4px; flex-wrap: wrap; }
+  .tab-btn {
+    background: none;
+    border: 1px solid var(--border);
+    color: var(--muted);
+    padding: 4px 13px;
+    border-radius: 20px;
+    font-size: 0.75rem;
+    cursor: pointer;
+    transition: all 0.12s;
+    white-space: nowrap;
+  }
+  .tab-btn:hover { border-color: var(--accent); color: var(--text); }
+  .tab-btn.active { background: var(--accent); border-color: var(--accent); color: #fff; font-weight: 600; }
+  .search-wrap { flex: 1; min-width: 150px; max-width: 340px; }
+  .search-input {
+    width: 100%;
+    background: var(--surface2);
+    border: 1px solid var(--border);
+    color: var(--text);
+    padding: 6px 11px;
+    border-radius: 6px;
+    font-size: 0.8rem;
+    outline: none;
+  }
+  .search-input:focus { border-color: var(--accent); }
+  .search-input::placeholder { color: var(--muted); }
+  .result-count { font-size: 0.75rem; color: var(--muted); margin-left: auto; white-space: nowrap; }
+
+  /* ── Grid ── */
+  .grid-wrapper { padding: 20px 32px 40px; }
+  .tests-grid {
+    display: grid;
+    grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
+    gap: 14px;
+  }
+  .empty-state {
+    grid-column: 1 / -1;
+    text-align: center;
+    padding: 80px 0;
+    color: var(--muted);
+    font-size: 0.9rem;
+  }
+
+  /* ── Card ── */
+  .test-card {
+    background: var(--surface);
+    border: 1px solid var(--border);
+    border-radius: var(--radius);
+    overflow: hidden;
+    display: flex;
+    flex-direction: column;
+    transition: border-color 0.12s;
+  }
+  .test-card:hover { border-color: #3a3a3a; }
+
+  /* ── Filmstrip ── */
+  .filmstrip-wrap {
+    width: 100%;
+    overflow-x: auto;
+    background: #111;
+    scrollbar-width: thin;
+    scrollbar-color: #333 #111;
+  }
+  .filmstrip-wrap::-webkit-scrollbar { height: 4px; }
+  .filmstrip-wrap::-webkit-scrollbar-thumb { background: #333; border-radius: 2px; }
+  .filmstrip {
+    display: flex;
+    gap: 3px;
+    padding: 6px;
+    min-height: 90px;
+    align-items: center;
+  }
+  .filmstrip-empty {
+    width: 100%;
+    min-height: 90px;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    font-size: 0.7rem;
+    color: #333;
+    background: #111;
+  }
+  .frame-thumb {
+    flex-shrink: 0;
+    height: 78px;
+    width: auto;
+    border-radius: 3px;
+    cursor: pointer;
+    border: 1px solid transparent;
+    transition: border-color 0.1s, transform 0.1s;
+    display: block;
+    background: #0a0a0a;
+  }
+  .frame-thumb:hover { border-color: var(--accent); transform: scale(1.04); }
+  .frame-thumb:first-child { border-color: #2ecc71; }
+  .frame-thumb:last-child:not(:first-child) { border-color: #e74c3c; }
+
+  /* ── Card body ── */
+  .card-body { padding: 10px 12px; flex: 1; }
+  .card-top { display: flex; align-items: flex-start; gap: 7px; margin-bottom: 6px; }
+  .pkg-badge {
+    flex-shrink: 0;
+    background: var(--accent);
+    color: #fff;
+    font-size: 0.62rem;
+    font-weight: 700;
+    padding: 2px 6px;
+    border-radius: 4px;
+    text-transform: uppercase;
+    letter-spacing: 0.04em;
+    margin-top: 1px;
+  }
+  .card-name { font-size: 0.8rem; font-weight: 500; color: #ddd; line-height: 1.35; word-break: break-word; }
+  .card-meta { font-size: 0.7rem; color: var(--muted); margin-bottom: 8px; }
+  .card-actions { display: flex; gap: 6px; flex-wrap: wrap; }
+  .card-link {
+    font-size: 0.7rem;
+    padding: 3px 9px;
+    border-radius: 4px;
+    text-decoration: none;
+    font-weight: 500;
+    border: 1px solid var(--border);
+    background: var(--surface2);
+    color: var(--text);
+    transition: border-color 0.1s, color 0.1s;
+  }
+  .card-link:hover { border-color: var(--accent); color: var(--accent); }
+  .card-link.na { color: #444; border-color: #222; pointer-events: none; }
+
+  /* ── Lightbox ── */
+  #lb {
+    display: none;
+    position: fixed;
+    inset: 0;
+    background: rgba(0,0,0,0.92);
+    z-index: 1000;
+    align-items: center;
+    justify-content: center;
+    flex-direction: column;
+  }
+  #lb.open { display: flex; }
+  #lb-img {
+    max-width: 92vw;
+    max-height: 82vh;
+    border-radius: 6px;
+    box-shadow: 0 8px 40px rgba(0,0,0,0.8);
+    display: block;
+  }
+  #lb-caption {
+    margin-top: 10px;
+    font-size: 0.75rem;
+    color: #888;
+  }
+  #lb-close {
+    position: fixed;
+    top: 18px; right: 22px;
+    background: none;
+    border: none;
+    color: #888;
+    font-size: 1.6rem;
+    cursor: pointer;
+    line-height: 1;
+    padding: 4px;
+  }
+  #lb-close:hover { color: #fff; }
+  .lb-arrow {
+    position: fixed;
+    top: 50%;
+    transform: translateY(-50%);
+    background: rgba(255,255,255,0.06);
+    border: 1px solid rgba(255,255,255,0.12);
+    color: #ccc;
+    font-size: 1.4rem;
+    padding: 12px 16px;
+    cursor: pointer;
+    border-radius: 6px;
+    user-select: none;
+    transition: background 0.1s;
+  }
+  .lb-arrow:hover { background: rgba(255,255,255,0.12); color: #fff; }
+  #lb-prev { left: 16px; }
+  #lb-next { right: 16px; }
+
+  @media (max-width: 640px) {
+    .page-header, .toolbar, .grid-wrapper { padding-left: 14px; padding-right: 14px; }
+    .tests-grid { grid-template-columns: 1fr; }
+    #lb-prev { left: 6px; }
+    #lb-next { right: 6px; }
+  }
+</style>
+</head>
+<body>
+
+<div class="page-header">
+  <h1><span class="logo"></span> E2E Recordings</h1>
+  <div class="meta">Generated: ${esc(manifest.generated ?? "")} &nbsp;·&nbsp; ${allTests.length} test${allTests.length !== 1 ? "s" : ""} across ${packageNames.length} package${packageNames.length !== 1 ? "s" : ""}</div>
+</div>
+
+<div class="toolbar">
+  <div class="filter-tabs" id="filterTabs">
+    <button class="tab-btn active" data-pkg="__all__">All</button>
+    ${packageNames.map((p) => `<button class="tab-btn" data-pkg="${esc(p)}">${esc(p)}</button>`).join("\n    ")}
+  </div>
+  <div class="search-wrap">
+    <input class="search-input" id="searchInput" type="search" placeholder="Search tests…">
+  </div>
+  <div class="result-count" id="resultCount"></div>
+</div>
+
+<div class="grid-wrapper">
+  <div class="tests-grid" id="testsGrid"></div>
+</div>
+
+<!-- Lightbox -->
+<div id="lb" role="dialog" aria-modal="true">
+  <button id="lb-close" aria-label="Close">&times;</button>
+  <button class="lb-arrow" id="lb-prev" aria-label="Previous">&#8592;</button>
+  <img id="lb-img" src="" alt="">
+  <div id="lb-caption"></div>
+  <button class="lb-arrow" id="lb-next" aria-label="Next">&#8594;</button>
+</div>
+
+<script>
+(function () {
+  const ALL_TESTS = ${testsJson};
+
+  // ── Lightbox state ─────────────────────────────────────────
+  let lbFrames = [];
+  let lbIdx = 0;
+
+  const lb = document.getElementById('lb');
+  const lbImg = document.getElementById('lb-img');
+  const lbCap = document.getElementById('lb-caption');
+
+  function openLightbox(frames, idx) {
+    lbFrames = frames;
+    lbIdx = idx;
+    showLbFrame();
+    lb.classList.add('open');
+    document.body.style.overflow = 'hidden';
+  }
+  function closeLightbox() {
+    lb.classList.remove('open');
+    document.body.style.overflow = '';
+  }
+  function showLbFrame() {
+    lbImg.src = lbFrames[lbIdx] ?? '';
+    lbCap.textContent = 'Frame ' + (lbIdx + 1) + ' / ' + lbFrames.length;
+  }
+
+  document.getElementById('lb-close').addEventListener('click', closeLightbox);
+  document.getElementById('lb-prev').addEventListener('click', () => {
+    if (lbFrames.length === 0) return;
+    lbIdx = (lbIdx - 1 + lbFrames.length) % lbFrames.length;
+    showLbFrame();
+  });
+  document.getElementById('lb-next').addEventListener('click', () => {
+    if (lbFrames.length === 0) return;
+    lbIdx = (lbIdx + 1) % lbFrames.length;
+    showLbFrame();
+  });
+  lb.addEventListener('click', (e) => { if (e.target === lb) closeLightbox(); });
+  document.addEventListener('keydown', (e) => {
+    if (!lb.classList.contains('open')) return;
+    if (e.key === 'Escape') closeLightbox();
+    if (e.key === 'ArrowLeft') { lbIdx = (lbIdx - 1 + lbFrames.length) % lbFrames.length; showLbFrame(); }
+    if (e.key === 'ArrowRight') { lbIdx = (lbIdx + 1) % lbFrames.length; showLbFrame(); }
+  });
+
+  // ── Filtering ───────────────────────────────────────────────
+  let activePackage = '__all__';
+  let searchQuery = '';
+
+  function getFiltered() {
+    return ALL_TESTS.filter((t) => {
+      if (activePackage !== '__all__' && t.package !== activePackage) return false;
+      if (searchQuery) {
+        const q = searchQuery.toLowerCase();
+        if (!t.name.toLowerCase().includes(q) && !t.package.toLowerCase().includes(q)) return false;
+      }
+      return true;
+    });
+  }
+
+  function renderGrid() {
+    const grid = document.getElementById('testsGrid');
+    const countEl = document.getElementById('resultCount');
+    const tests = getFiltered();
+    countEl.textContent = tests.length + ' test' + (tests.length !== 1 ? 's' : '');
+    if (tests.length === 0) {
+      grid.innerHTML = '<div class="empty-state">No tests match the current filter.</div>';
+      return;
+    }
+    grid.innerHTML = tests.map(buildCard).join('');
+    // Wire up filmstrip click handlers
+    grid.querySelectorAll('.frame-thumb').forEach((img) => {
+      img.addEventListener('click', () => {
+        const frames = JSON.parse(img.closest('.test-card').dataset.frames);
+        const idx = parseInt(img.dataset.idx, 10);
+        openLightbox(frames, idx);
+      });
+    });
+  }
+
+  function buildCard(t) {
+    const frames = t.frames ?? [];
+    let filmstrip;
+    if (frames.length === 0) {
+      filmstrip = '<div class="filmstrip-empty">no frames</div>';
+    } else {
+      const thumbs = frames.map((src, i) =>
+        '<img class="frame-thumb" src="' + esc(src) + '" loading="lazy" data-idx="' + i + '" alt="frame ' + (i+1) + '">'
+      ).join('');
+      filmstrip = '<div class="filmstrip-wrap"><div class="filmstrip">' + thumbs + '</div></div>';
+    }
+
+    const videoLink = t.video
+      ? '<a class="card-link" href="' + esc(t.video) + '" target="_blank">Video</a>'
+      : '<span class="card-link na">no video</span>';
+
+    const framesMeta = t.frameCount != null
+      ? t.frameCount + ' frame' + (t.frameCount !== 1 ? 's' : '')
+      : '';
+
+    return (
+      '<div class="test-card" data-pkg="' + esc(t.package) + '" data-frames='' + escAttr(JSON.stringify(frames)) + ''>' +
+      filmstrip +
+      '<div class="card-body">' +
+        '<div class="card-top">' +
+          '<span class="pkg-badge">' + esc(t.package) + '</span>' +
+          '<span class="card-name">' + esc(t.name) + '</span>' +
+        '</div>' +
+        '<div class="card-meta">' + esc(framesMeta) + '</div>' +
+        '<div class="card-actions">' + videoLink + '</div>' +
+      '</div>' +
+      '</div>'
+    );
+  }
+
+  function esc(str) {
+    return String(str || '')
+      .replace(/&/g, '&amp;')
+      .replace(/</g, '&lt;')
+      .replace(/>/g, '&gt;')
+      .replace(/"/g, '&quot;');
+  }
+  function escAttr(str) {
+    return String(str || '')
+      .replace(/&/g, '&amp;')
+      .replace(/'/g, '&#39;')
+      .replace(/</g, '&lt;')
+      .replace(/>/g, '&gt;');
+  }
+
+  document.getElementById('filterTabs').addEventListener('click', (e) => {
+    const btn = e.target.closest('.tab-btn');
+    if (!btn) return;
+    activePackage = btn.dataset.pkg;
+    document.querySelectorAll('.tab-btn').forEach((b) => b.classList.remove('active'));
+    btn.classList.add('active');
+    renderGrid();
+  });
+
+  document.getElementById('searchInput').addEventListener('input', (e) => {
+    searchQuery = e.target.value.trim();
+    renderGrid();
+  });
+
+  renderGrid();
+})();
+</script>
+</body>
+</html>`;
+}
+
+function main() {
+  if (!fs.existsSync(MANIFEST_PATH)) {
+    console.error(`manifest.json not found at ${MANIFEST_PATH}`);
+    console.error("Run generate-contact-sheets.mjs first.");
+    process.exit(1);
+  }
+
+  let manifest;
+  try {
+    manifest = JSON.parse(fs.readFileSync(MANIFEST_PATH, "utf8"));
+  } catch (err) {
+    console.error(`Failed to parse manifest.json: ${err.message}`);
+    process.exit(1);
+  }
+
+  const html = buildHtml(manifest);
+  fs.writeFileSync(OUTPUT_PATH, html, "utf8");
+  console.log(`Viewer written: ${OUTPUT_PATH}`);
+
+  const totalTests = Object.values(manifest.packages ?? {}).reduce(
+    (sum, pkg) => sum + (pkg.tests?.length ?? 0),
+    0,
+  );
+  console.log(
+    `Indexed ${totalTests} test(s) across ${Object.keys(manifest.packages ?? {}).length} package(s).`,
+  );
+}
+
+main();

scripts/e2e-recordings/native-capture-common.mjs

@@ -0,0 +1,237 @@
+/**
+ * Shared helpers for the native capture scripts (capture-android-emu,
+ * capture-ios-sim): CLI/env arg parsing, capture-path resolution, capture
+ * logging, starting the device-facing host agent, artifact copying, and writing
+ * the capture manifest. SKIP_EXIT_CODE (77) is the agreed "device unavailable,
+ * skip cleanly" exit code across those scripts.
+ */
+import { spawn } from "node:child_process";
+import fs from "node:fs";
+import path from "node:path";
+
+export const SKIP_EXIT_CODE = 77;
+
+export function hasArg(args, flag) {
+  return args.includes(flag);
+}
+
+export function argValue(args, flag, fallback = null) {
+  const index = args.indexOf(flag);
+  return index >= 0 && args[index + 1] ? args[index + 1] : fallback;
+}
+
+export const DEFAULT_API_PORT = 31337;
+
+/**
+ * Resolve the backend API port a capture run should target. Precedence:
+ *   1. explicit `--api-port <n>` CLI arg,
+ *   2. `ELIZA_API_PORT` env (the orchestrator advertises the auto-shifted port
+ *      here so parallel worktree stacks don't collide on 31337),
+ *   3. the built-in default (31337).
+ * A bare hardcoded 31337 silently probes the wrong backend on a port-shifted
+ * stack; this resolver keeps the same default while honoring an override
+ * (#13624). Only a bare positive integer in range is accepted; anything else
+ * falls through to the next source.
+ *
+ * @param {string[]} [args] argv slice (e.g. process.argv.slice(2))
+ * @param {Record<string, string | undefined>} [env]
+ * @returns {number}
+ */
+export function resolveApiPort(args = [], env = process.env) {
+  const candidates = [argValue(args, "--api-port"), env?.ELIZA_API_PORT];
+  for (const raw of candidates) {
+    if (typeof raw !== "string") continue;
+    const trimmed = raw.trim();
+    if (!/^\d+$/.test(trimmed)) continue;
+    const parsed = Number.parseInt(trimmed, 10);
+    if (Number.isInteger(parsed) && parsed > 0 && parsed <= 65535) {
+      return parsed;
+    }
+  }
+  return DEFAULT_API_PORT;
+}
+
+export function resolveCapturePaths({ repoRoot, platform, slug, args }) {
+  const issue = argValue(args, "--issue", process.env.EVIDENCE_ISSUE);
+  const prefix =
+    argValue(args, "--evidence-prefix", process.env.EVIDENCE_PREFIX) ??
+    (issue ? `${issue}-${slug}` : `local-${slug}`);
+  const evidenceDir = path.resolve(
+    repoRoot,
+    argValue(
+      args,
+      "--out",
+      process.env.EVIDENCE_ARTIFACT_DIR ??
+        path.join(
+          "e2e-recordings",
+          platform,
+          "test-results",
+          "native-capture",
+          prefix,
+        ),
+    ),
+  );
+  const recordingResultDir = path.join(
+    repoRoot,
+    "e2e-recordings",
+    platform,
+    "test-results",
+    "native-capture",
+  );
+  fs.mkdirSync(evidenceDir, { recursive: true });
+  fs.mkdirSync(recordingResultDir, { recursive: true });
+  return { prefix, evidenceDir, recordingResultDir };
+}
+
+export function createCaptureLog(logPath, prefix) {
+  fs.mkdirSync(path.dirname(logPath), { recursive: true });
+  const stream = fs.createWriteStream(logPath, { flags: "a" });
+  return {
+    logPath,
+    writeRaw(chunk) {
+      stream.write(chunk);
+    },
+    log(message) {
+      const line = `[${prefix}] ${message}`;
+      console.log(line);
+      stream.write(`${line}\n`);
+    },
+    close() {
+      stream.end();
+    },
+  };
+}
+
+export async function runCommandWithLog(command, args, options = {}) {
+  const {
+    cwd,
+    env = {},
+    logSink,
+    label = `${command} ${args.join(" ")}`,
+  } = options;
+  logSink?.log(`running ${label}`);
+
+  await new Promise((resolve, reject) => {
+    const child = spawn(command, args, {
+      cwd,
+      env: { ...process.env, ...env },
+      stdio: ["ignore", "pipe", "pipe"],
+    });
+    child.stdout.on("data", (chunk) => {
+      process.stdout.write(chunk);
+      logSink?.writeRaw(chunk);
+    });
+    child.stderr.on("data", (chunk) => {
+      process.stderr.write(chunk);
+      logSink?.writeRaw(chunk);
+    });
+    child.on("error", reject);
+    child.on("close", (code) => {
+      if (code === 0) {
+        resolve();
+        return;
+      }
+      reject(new Error(`${label} exited with ${code}`));
+    });
+  });
+}
+
+async function healthReady(url) {
+  try {
+    const response = await fetch(url, {
+      headers: { "X-ElizaOS-Client-Id": "native-capture" },
+    });
+    return response.ok;
+  } catch {
+    return false;
+  }
+}
+
+async function waitForHealth(url, timeoutMs) {
+  const deadline = Date.now() + timeoutMs;
+  while (Date.now() < deadline) {
+    if (await healthReady(url)) return true;
+    await new Promise((resolve) => setTimeout(resolve, 1_000));
+  }
+  return false;
+}
+
+export async function startDeviceE2EHostAgent({
+  repoRoot,
+  port = resolveApiPort(),
+  timeoutMs = 180_000,
+  logSink,
+}) {
+  const healthUrl = `http://127.0.0.1:${port}/api/health`;
+  if (await waitForHealth(healthUrl, 2_000)) {
+    logSink?.log(`reusing host agent at ${healthUrl}`);
+    return { started: false, async stop() {} };
+  }
+
+  logSink?.log(`starting deterministic host agent at ${healthUrl}`);
+  const child = spawn(
+    process.execPath,
+    [
+      "packages/app-core/scripts/run-node-tsx.mjs",
+      "packages/app-core/scripts/serve-real-local-agent.ts",
+    ],
+    {
+      cwd: repoRoot,
+      env: {
+        ...process.env,
+        ELIZA_API_PORT: String(port),
+        ELIZA_PAIRING_DISABLED: "1",
+      },
+      stdio: ["ignore", "pipe", "pipe"],
+    },
+  );
+  child.stdout.on("data", (chunk) => {
+    process.stdout.write(chunk);
+    logSink?.writeRaw(chunk);
+  });
+  child.stderr.on("data", (chunk) => {
+    process.stderr.write(chunk);
+    logSink?.writeRaw(chunk);
+  });
+
+  if (!(await waitForHealth(healthUrl, timeoutMs))) {
+    child.kill("SIGTERM");
+    throw new Error(`host agent did not become healthy at ${healthUrl}`);
+  }
+
+  return {
+    started: true,
+    async stop() {
+      if (child.exitCode !== null) return;
+      logSink?.log("stopping deterministic host agent");
+      child.kill("SIGTERM");
+      await Promise.race([
+        new Promise((resolve) => child.once("close", resolve)),
+        new Promise((resolve) => setTimeout(resolve, 5_000)),
+      ]);
+      if (child.exitCode === null) child.kill("SIGKILL");
+    },
+  };
+}
+
+export function copyArtifact(src, destDir, destName, { required = true } = {}) {
+  if (!src) {
+    if (required) throw new Error("copyArtifact requires a source path");
+    return null;
+  }
+  try {
+    if (fs.statSync(src).size <= 0) throw new Error("empty file");
+  } catch (error) {
+    if (!required) return null;
+    throw new Error(`artifact missing or empty: ${src} (${error.message})`);
+  }
+  fs.mkdirSync(destDir, { recursive: true });
+  const dest = path.join(destDir, destName ?? path.basename(src));
+  fs.copyFileSync(src, dest);
+  return dest;
+}
+
+export function writeCaptureManifest(filePath, data) {
+  fs.mkdirSync(path.dirname(filePath), { recursive: true });
+  fs.writeFileSync(filePath, `${JSON.stringify(data, null, 2)}\n`);
+}

scripts/e2e-recordings/run-all-require-evidence.test.mjs

@@ -0,0 +1,81 @@
+// #13624: the e2e-recordings sweep must not go green having recorded nothing.
+// classifyRunResults folds a `skipped` suite into `failed` under
+// --require-evidence (auto-on under CI) so a headless runner that captured zero
+// artifacts fails the run, while preserving the soft-skip behavior otherwise.
+import { describe, expect, it } from "vitest";
+import { classifyRunResults, parseRunAllArgs } from "./run-all.mjs";
+
+const passedSuite = { name: "app", passed: true, skipped: false, exitCode: 0 };
+const failedSuite = {
+  name: "cloud",
+  passed: false,
+  skipped: false,
+  exitCode: 1,
+};
+const skippedSuite = {
+  name: "ios-sim",
+  passed: false,
+  skipped: true,
+  exitCode: 77,
+  reason: "no booted simulator",
+};
+
+describe("classifyRunResults require-evidence contract (#13624)", () => {
+  it("requires OCR for generated evidence review dashboards", () => {
+    expect(parseRunAllArgs(["--review"]).reviewOcr).toBe("on");
+    expect(parseRunAllArgs(["--review", "--review-ocr=on"]).reviewOcr).toBe(
+      "on",
+    );
+    expect(() => parseRunAllArgs(["--review", "--review-ocr=off"])).toThrow(
+      /--review-ocr must be on/,
+    );
+    expect(() => parseRunAllArgs(["--review", "--review-ocr=auto"])).toThrow(
+      /--review-ocr must be on/,
+    );
+  });
+
+  it("without require-evidence: a skip stays a soft skip and does not fail the run", () => {
+    const c = classifyRunResults([passedSuite, skippedSuite], false);
+    expect(c.passed.map((r) => r.name)).toEqual(["app"]);
+    expect(c.softSkipped.map((r) => r.name)).toEqual(["ios-sim"]);
+    expect(c.failed).toEqual([]);
+    expect(c.shouldFail).toBe(false);
+  });
+
+  it("REGRESSION: WITH require-evidence a skipped suite becomes a failure (green-with-nothing closed)", () => {
+    const c = classifyRunResults([passedSuite, skippedSuite], true);
+    // The skip is no longer counted as a benign skip...
+    expect(c.softSkipped).toEqual([]);
+    // ...it is folded into failed and the run must fail.
+    expect(c.failed.map((r) => r.name)).toEqual(["ios-sim"]);
+    expect(c.skippedButRequired.map((r) => r.name)).toEqual(["ios-sim"]);
+    expect(c.shouldFail).toBe(true);
+  });
+
+  it("a real non-zero failure always fails the run, require-evidence or not", () => {
+    expect(classifyRunResults([failedSuite], false).shouldFail).toBe(true);
+    expect(classifyRunResults([failedSuite], true).shouldFail).toBe(true);
+  });
+
+  it("an all-passing run stays green under require-evidence", () => {
+    const c = classifyRunResults([passedSuite], true);
+    expect(c.shouldFail).toBe(false);
+    expect(c.failed).toEqual([]);
+    expect(c.skippedButRequired).toEqual([]);
+  });
+
+  it("mixed: pass + real-fail + skip under require-evidence folds skip AND fail together", () => {
+    const c = classifyRunResults(
+      [passedSuite, failedSuite, skippedSuite],
+      true,
+    );
+    expect(c.passed.map((r) => r.name)).toEqual(["app"]);
+    expect(c.failed.map((r) => r.name).sort()).toEqual(["cloud", "ios-sim"]);
+    expect(c.shouldFail).toBe(true);
+  });
+
+  it("preserves the skip reason so the failure line can explain what was missing", () => {
+    const c = classifyRunResults([skippedSuite], true);
+    expect(c.failed[0].reason).toBe("no booted simulator");
+  });
+});

scripts/e2e-recordings/run-all.mjs

@@ -0,0 +1,478 @@
+#!/usr/bin/env node
+
+/**
+ * run-all.mjs
+ *
+ * Orchestrates running all E2E suites with recording enabled, then generates
+ * contact sheets, the viewer index, and an optional local evidence dashboard.
+ *
+ * Usage:
+ *   node scripts/e2e-recordings/run-all.mjs
+ *
+ * Options:
+ *   --packages=<comma-list>   Run only the named packages (e.g. --packages=homepage,app-core)
+ *   --skip-tests              Skip running tests; only regenerate sheets + viewer
+ *   --skip-sheets             Skip generating contact sheets
+ *   --skip-viewer             Skip generating the viewer index
+ *   --review                  Generate evidence/index.html for manual review
+ *   --open-review             Open the generated evidence review dashboard
+ *   --review-ocr=on           OCR mode for --review. Packaged OCR is required.
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { resolveRequireEvidence } from "../../packages/app/scripts/lib/capture-output.mjs";
+import { RECORDINGS_DIR, REPO_ROOT, UI_E2E_SUITES } from "./suites.mjs";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const SKIP_EXIT_CODE = 77;
+
+const SCRIPTS_DIR = __dirname;
+const PACKAGES = UI_E2E_SUITES;
+
+// ─── CLI argument parsing ────────────────────────────────────────────────────
+export function parseRunAllArgs(argv) {
+  const flagMap = new Map();
+  for (const arg of argv) {
+    const [key, val] = arg.replace(/^--/, "").split("=");
+    flagMap.set(key, val ?? true);
+  }
+
+  const reviewOcr = flagMap.has("review-ocr")
+    ? String(flagMap.get("review-ocr"))
+    : "on";
+  if (reviewOcr !== "on") {
+    throw new Error(
+      "--review-ocr must be on; OCR is required for evidence review and uses the packaged tesseract.js dependency",
+    );
+  }
+
+  return {
+    onlyPackages: flagMap.has("packages")
+      ? String(flagMap.get("packages"))
+          .split(",")
+          .map((s) => s.trim())
+      : null,
+    skipTests:
+      flagMap.get("skip-tests") === true ||
+      flagMap.get("skip-tests") === "true",
+    skipSheets:
+      flagMap.get("skip-sheets") === true ||
+      flagMap.get("skip-sheets") === "true",
+    skipViewer:
+      flagMap.get("skip-viewer") === true ||
+      flagMap.get("skip-viewer") === "true",
+    review:
+      flagMap.get("review") === true ||
+      flagMap.get("review") === "true" ||
+      flagMap.get("open-review") === true ||
+      flagMap.get("open-review") === "true",
+    openReview:
+      flagMap.get("open-review") === true ||
+      flagMap.get("open-review") === "true",
+    reviewOcr,
+    requireEvidence: resolveRequireEvidence(argv),
+  };
+}
+
+const args = process.argv.slice(2);
+const {
+  onlyPackages,
+  skipTests,
+  skipSheets,
+  skipViewer,
+  review,
+  openReview,
+  reviewOcr,
+  requireEvidence,
+} = parseRunAllArgs(args);
+
+// When evidence is required (explicit --require-evidence, or auto-on under CI),
+// a suite that soft-skips (SKIP_EXIT_CODE=77, missing dir/script/package.json,
+// or an unavailable availability-check) produced ZERO artifacts and must count
+// as a FAILURE, not a benign skip. Otherwise the recordings sweep goes green on
+// a headless runner with nothing captured (#13624 "green-with-nothing"). We
+// hand argv explicitly (stripping the `=value` join key parsing above only
+// affects flagMap; resolveRequireEvidence re-scans raw argv).
+// ─── Helpers ─────────────────────────────────────────────────────────────────
+
+function banner(text) {
+  const line = "─".repeat(60);
+  console.log(`\n${line}`);
+  console.log(`  ${text}`);
+  console.log(`${line}`);
+}
+
+function runScript(scriptFile, scriptArgs = []) {
+  const result = spawnSync(
+    process.execPath, // node
+    [scriptFile, ...scriptArgs],
+    {
+      cwd: REPO_ROOT,
+      stdio: "inherit",
+      env: { ...process.env },
+    },
+  );
+  return result.status ?? 1;
+}
+
+function formatCommand(command) {
+  return command.join(" ");
+}
+
+function resolveCommand(command) {
+  const [bin, ...args] = command;
+  return [bin === "node" ? process.execPath : bin, args];
+}
+
+function runCommandSuite(pkg) {
+  if (pkg.checkCommand) {
+    const [checkBin, checkArgs] = resolveCommand(pkg.checkCommand);
+    const check = spawnSync(checkBin, checkArgs, {
+      cwd: REPO_ROOT,
+      encoding: "utf8",
+      env: { ...process.env },
+    });
+    const checkOutput = [check.stdout, check.stderr]
+      .filter(Boolean)
+      .join("\n")
+      .trim();
+    if (check.status === SKIP_EXIT_CODE) {
+      const reason = checkOutput || "platform/tooling unavailable";
+      console.warn(`  [skip] ${pkg.name}: ${reason}`);
+      return {
+        name: pkg.name,
+        passed: false,
+        skipped: true,
+        exitCode: SKIP_EXIT_CODE,
+        reason,
+      };
+    }
+    if (check.status !== 0) {
+      if (checkOutput) console.warn(checkOutput);
+      console.warn(
+        `  ✗ ${pkg.name} availability check failed (exit ${check.status})`,
+      );
+      return {
+        name: pkg.name,
+        passed: false,
+        skipped: false,
+        exitCode: check.status ?? 1,
+      };
+    }
+  }
+
+  const outputDir = path.join(RECORDINGS_DIR, pkg.name, "test-results");
+  fs.mkdirSync(outputDir, { recursive: true });
+  console.log(`  Running: ${formatCommand(pkg.command)}`);
+  console.log(`  Output:  e2e-recordings/${pkg.name}/test-results/`);
+
+  const [bin, args] = resolveCommand(pkg.command);
+  const result = spawnSync(bin, args, {
+    cwd: REPO_ROOT,
+    stdio: "inherit",
+    env: {
+      ...process.env,
+      E2E_RECORD: "1",
+      ...(pkg.recordEnv ?? {}),
+    },
+  });
+
+  const exitCode = result.status ?? 1;
+  const passed = exitCode === 0;
+  if (passed) {
+    console.log(`  ✓ ${pkg.name} passed`);
+  } else if (exitCode === SKIP_EXIT_CODE) {
+    console.warn(`  [skip] ${pkg.name}: platform/tooling unavailable`);
+    return {
+      name: pkg.name,
+      passed: false,
+      skipped: true,
+      exitCode,
+      reason: "platform/tooling unavailable",
+    };
+  } else {
+    console.warn(`  ✗ ${pkg.name} failed (exit ${exitCode})`);
+  }
+
+  return { name: pkg.name, passed, skipped: false, exitCode };
+}
+
+/**
+ * Run a single package's E2E test suite with recording enabled.
+ * Returns { name, passed: boolean, skipped: boolean, exitCode: number }.
+ */
+function runPackageTests(pkg) {
+  if (pkg.command) return runCommandSuite(pkg);
+
+  const configDirAbs = path.join(REPO_ROOT, pkg.configDir);
+
+  // Skip if the package directory doesn't exist
+  if (!fs.existsSync(configDirAbs)) {
+    console.warn(
+      `  [skip] ${pkg.name}: directory not found (${pkg.configDir})`,
+    );
+    return {
+      name: pkg.name,
+      passed: false,
+      skipped: true,
+      exitCode: -1,
+      reason: `directory not found (${pkg.configDir})`,
+    };
+  }
+
+  // Check the script exists in package.json
+  let pkgJson;
+  try {
+    pkgJson = JSON.parse(
+      fs.readFileSync(path.join(configDirAbs, "package.json"), "utf8"),
+    );
+  } catch {
+    console.warn(`  [skip] ${pkg.name}: could not read package.json`);
+    return {
+      name: pkg.name,
+      passed: false,
+      skipped: true,
+      exitCode: -1,
+      reason: "could not read package.json",
+    };
+  }
+
+  if (!pkgJson.scripts?.[pkg.script]) {
+    console.warn(
+      `  [skip] ${pkg.name}: script "${pkg.script}" not defined in package.json`,
+    );
+    return {
+      name: pkg.name,
+      passed: false,
+      skipped: true,
+      exitCode: -1,
+      reason: `script "${pkg.script}" not defined in package.json`,
+    };
+  }
+
+  // Ensure the recording output directory exists so Playwright has somewhere to write
+  const recordingOut = path.join(RECORDINGS_DIR, pkg.name, "test-results");
+  fs.mkdirSync(recordingOut, { recursive: true });
+
+  console.log(`  Running: bun run --cwd ${pkg.configDir} ${pkg.script}`);
+  console.log(`  Output:  e2e-recordings/${pkg.name}/test-results/`);
+
+  const result = spawnSync("bun", ["run", "--cwd", pkg.configDir, pkg.script], {
+    cwd: REPO_ROOT,
+    stdio: "inherit",
+    env: {
+      ...process.env,
+      // Signal to Playwright config that we want full recording.
+      // The config itself computes outputDir from import.meta.dirname + E2E_RECORD.
+      E2E_RECORD: "1",
+      // Per-package extra env (e.g. ELIZA_UI_SMOKE_FORCE_STUB for the app package).
+      ...(pkg.recordEnv ?? {}),
+    },
+  });
+
+  const exitCode = result.status ?? 1;
+  const passed = exitCode === 0;
+
+  if (passed) {
+    console.log(`  ✓ ${pkg.name} passed`);
+  } else {
+    console.warn(`  ✗ ${pkg.name} failed (exit ${exitCode})`);
+  }
+
+  return { name: pkg.name, passed, skipped: false, exitCode };
+}
+
+// ─── Main ─────────────────────────────────────────────────────────────────────
+
+async function main() {
+  const startTime = Date.now();
+
+  // Filter packages if --packages flag was supplied
+  const packagesToRun = onlyPackages
+    ? PACKAGES.filter((p) => onlyPackages.includes(p.name))
+    : PACKAGES;
+
+  if (onlyPackages && packagesToRun.length === 0) {
+    console.error(`No packages matched: ${onlyPackages.join(", ")}`);
+    console.error(
+      `Available packages: ${PACKAGES.map((p) => p.name).join(", ")}`,
+    );
+    process.exit(1);
+  }
+
+  fs.mkdirSync(RECORDINGS_DIR, { recursive: true });
+
+  // ─── Step 1: Run tests ─────────────────────────────────────
+  const results = [];
+
+  if (skipTests && requireEvidence) {
+    console.error(
+      "[require-evidence] --skip-tests cannot be combined with --require-evidence: " +
+        "a run that skips every suite captures no evidence.",
+    );
+    process.exit(1);
+  }
+
+  if (skipTests) {
+    console.log("Skipping test runs (--skip-tests).");
+    for (const pkg of packagesToRun) {
+      results.push({
+        name: pkg.name,
+        passed: true,
+        skipped: true,
+        exitCode: 0,
+      });
+    }
+  } else {
+    banner("Running E2E test suites");
+    for (const pkg of packagesToRun) {
+      console.log(`\n▶ ${pkg.name}`);
+      const r = runPackageTests(pkg);
+      results.push(r);
+    }
+  }
+
+  // ─── Step 2: Generate contact sheets ──────────────────────
+  if (!skipSheets) {
+    banner("Generating contact sheets");
+    const sheetsScript = path.join(SCRIPTS_DIR, "generate-contact-sheets.mjs");
+    if (fs.existsSync(sheetsScript)) {
+      const code = runScript(sheetsScript);
+      if (code !== 0) {
+        console.warn(
+          `[warn] generate-contact-sheets.mjs exited with code ${code}`,
+        );
+      }
+    } else {
+      console.warn("[warn] generate-contact-sheets.mjs not found — skipping");
+    }
+  } else {
+    console.log("Skipping contact sheet generation (--skip-sheets).");
+  }
+
+  // ─── Step 3: Generate viewer ───────────────────────────────
+  if (!skipViewer) {
+    banner("Generating viewer index");
+    const viewerScript = path.join(SCRIPTS_DIR, "generate-viewer.mjs");
+    if (fs.existsSync(viewerScript)) {
+      const code = runScript(viewerScript);
+      if (code !== 0) {
+        console.warn(`[warn] generate-viewer.mjs exited with code ${code}`);
+      }
+    } else {
+      console.warn("[warn] generate-viewer.mjs not found — skipping");
+    }
+  } else {
+    console.log("Skipping viewer generation (--skip-viewer).");
+  }
+
+  // ─── Step 4: Generate evidence review dashboard ───────────
+  if (review) {
+    banner("Generating evidence review dashboard");
+    const reviewScript = path.join(
+      REPO_ROOT,
+      "scripts",
+      "evidence-review",
+      "generate.mjs",
+    );
+    if (fs.existsSync(reviewScript)) {
+      const reviewArgs = [
+        "--source=e2e-recordings",
+        `--ocr=${reviewOcr}`,
+        openReview ? "--open" : "--no-open",
+      ];
+      const code = runScript(reviewScript, reviewArgs);
+      if (code !== 0) {
+        console.warn(`[warn] evidence review exited with code ${code}`);
+      }
+    } else {
+      console.warn("[warn] evidence-review/generate.mjs not found — skipping");
+    }
+  }
+
+  // ─── Summary ───────────────────────────────────────────────
+  const elapsed = ((Date.now() - startTime) / 1000).toFixed(1);
+  banner("Summary");
+
+  const { passed, failed, softSkipped, skippedButRequired } =
+    classifyRunResults(results, requireEvidence);
+
+  if (passed.length > 0) {
+    console.log(`\nPassed (${passed.length}):`);
+    for (const r of passed) console.log(`  ✓ ${r.name}`);
+  }
+  if (failed.length > 0) {
+    console.log(`\nFailed (${failed.length}):`);
+    for (const r of failed) {
+      const why = r.skipped
+        ? `skipped but evidence required${r.reason ? `: ${r.reason}` : ""}`
+        : `exit ${r.exitCode}`;
+      console.log(`  ✗ ${r.name}  (${why})`);
+    }
+  }
+  if (softSkipped.length > 0) {
+    console.log(`\nSkipped (${softSkipped.length}):`);
+    for (const r of softSkipped) {
+      console.log(`  - ${r.name}${r.reason ? `: ${r.reason}` : ""}`);
+    }
+  }
+  if (requireEvidence && skippedButRequired.length > 0) {
+    console.warn(
+      `\n[require-evidence] ${skippedButRequired.length} suite(s) skipped with no artifacts — failing the run.`,
+    );
+  }
+
+  const indexPath = path.join(RECORDINGS_DIR, "index.html");
+  if (fs.existsSync(indexPath)) {
+    console.log(`\nViewer: ${indexPath}`);
+    console.log(`        file://${indexPath}`);
+  }
+  const reviewPath = path.join(REPO_ROOT, "evidence", "index.html");
+  if (review && fs.existsSync(reviewPath)) {
+    console.log(`\nEvidence review: ${reviewPath}`);
+    console.log(`                 file://${reviewPath}`);
+  }
+
+  console.log(`\nTotal time: ${elapsed}s`);
+
+  // Exit non-zero if any suite failed
+  if (failed.length > 0) {
+    process.exit(1);
+  }
+}
+
+// Exported so the require-evidence exit contract is unit-testable without
+// spawning the full sweep. Under --require-evidence a `skipped` suite captured
+// ZERO artifacts, so it must count as a FAILURE (not a benign skip) — otherwise
+// the recordings sweep goes green on a headless runner having recorded nothing
+// (#13624 "green-with-nothing"). Without the flag, behavior is preserved: a
+// skip stays a soft skip and only real non-zero exits fail the run.
+export function classifyRunResults(results, requireEvidence) {
+  const passed = results.filter((r) => r.passed && !r.skipped);
+  const skipped = results.filter((r) => r.skipped);
+  const failedRuns = results.filter((r) => !r.passed && !r.skipped);
+  const skippedButRequired = requireEvidence ? skipped : [];
+  const softSkipped = requireEvidence ? [] : skipped;
+  const failed = [...failedRuns, ...skippedButRequired];
+  return {
+    passed,
+    failed,
+    softSkipped,
+    skippedButRequired,
+    shouldFail: failed.length > 0,
+  };
+}
+
+// The orchestrator self-executes only when run as a script, so the pure helper
+// above can be imported by tests without kicking off the whole sweep.
+const isMain = process.argv[1] && path.resolve(process.argv[1]) === __filename;
+if (isMain) {
+  main().catch((err) => {
+    console.error("Fatal error:", err);
+    process.exit(1);
+  });
+}

scripts/e2e-recordings/suites.mjs

@@ -0,0 +1,149 @@
+/**
+ * Declarative catalog of the UI end-to-end recording suites (name, working
+ * directory, run script, coverage blurb, and any record-mode env). The
+ * run-all.mjs orchestrator iterates this list to execute each suite and collect
+ * its screenshots, traces, and videos into RECORDINGS_DIR.
+ */
+import path from "node:path";
+
+export const REPO_ROOT = path.resolve(import.meta.dirname, "..", "..");
+export const RECORDINGS_DIR = path.join(REPO_ROOT, "e2e-recordings");
+
+export const UI_E2E_SUITES = [
+  {
+    name: "app",
+    displayName: "Main app shell",
+    configDir: "packages/app",
+    script: "test:e2e",
+    coverage:
+      "Runs the installed app shell, login/session startup, chat, all registered plugin views, settings, mobile viewport, inputs, screenshots, traces, and videos.",
+    recordEnv: { ELIZA_UI_SMOKE_FORCE_STUB: "1" },
+  },
+  {
+    name: "cloud-e2e",
+    displayName: "Cloud full-stack mock e2e",
+    configDir: "packages/test/cloud-e2e",
+    script: "test",
+    coverage:
+      "Boots the local cloud API, cloud frontend, auth cookie login, provisioning flows, screenshots, traces, and videos.",
+  },
+  {
+    name: "homepage",
+    displayName: "Homepage",
+    configDir: "packages/homepage",
+    script: "test:e2e",
+    coverage:
+      "Runs marketing routes, navigation, onboarding controls, contact capture, route coverage, screenshots, traces, and videos.",
+  },
+  {
+    name: "os-homepage",
+    displayName: "OS homepage",
+    configDir: "packages/os/homepage",
+    script: "test:e2e",
+    coverage:
+      "Runs OS homepage routes, checkout/preorder flows, link resilience, mobile/desktop screenshots, traces, and videos.",
+  },
+  {
+    name: "os-usb-installer",
+    displayName: "OS USB installer",
+    configDir: "packages/os/usb-installer",
+    script: "test:e2e",
+    coverage:
+      "Runs the installer wizard UI, visual pages, mobile/desktop screenshots, traces, and videos against the mocked installer API.",
+  },
+  {
+    name: "ui-agent-surface",
+    displayName: "Shared UI agent surface",
+    configDir: "packages/ui",
+    script: "test:agent-surface-e2e",
+    coverage:
+      "Runs the shared agent-surface fixture in Chromium, drives fill/click/focus capability bridge interactions, and records screenshots.",
+  },
+  {
+    name: "ui-launcher",
+    displayName: "Launcher view launcher",
+    configDir: "packages/ui",
+    script: "test:launcher-e2e",
+    coverage:
+      "Runs the Launcher launcher fixture in Chromium, asserts tiles + image tiles render, captures desktop/mobile rest + edit screenshots, drives tap-launch/long-press-edit/favorite/page-nav with a recorded video, and asserts the view-interaction telemetry stream fired.",
+  },
+  {
+    name: "feed-dag-visualizer",
+    displayName: "Feed DAG visualizer",
+    configDir: "packages/feed/tools/dag-visualizer",
+    script: "test",
+    coverage:
+      "Runs the standalone Feed DAG visualizer browser e2e with screenshots, traces, and videos.",
+  },
+  {
+    name: "android-emu",
+    displayName: "Android emulator app capture",
+    configDir: "packages/app",
+    command: ["node", "scripts/e2e-recordings/capture-android-emu.mjs"],
+    checkCommand: [
+      "node",
+      "scripts/e2e-recordings/capture-android-emu.mjs",
+      "--check",
+    ],
+    coverage:
+      "Boots or reuses an Android emulator, starts the deterministic host agent, drives the real Capacitor onboarding flow, and writes emulator screenshot, screenrecord, logcat, and capture logs to generated native-capture output.",
+  },
+  {
+    name: "ios-sim",
+    displayName: "iOS simulator app capture",
+    configDir: "packages/app",
+    command: ["node", "scripts/e2e-recordings/capture-ios-sim.mjs"],
+    checkCommand: [
+      "node",
+      "scripts/e2e-recordings/capture-ios-sim.mjs",
+      "--check",
+    ],
+    coverage:
+      "Boots or reuses an iOS Simulator, starts the deterministic host agent, drives first-run onboarding, and writes simulator screenshots, recordVideo output, smoke result JSON, and capture logs to generated native-capture output.",
+  },
+];
+
+export const UI_E2E_COVERED_BY_APP = [
+  {
+    name: "app-core",
+    configDir: "packages/app-core",
+    coveredBy: "app",
+    reason:
+      "App-core owns the app API/dev stack used by packages/app Playwright; its standalone Playwright config is not runnable because the package has no storybook script/e2e dir.",
+  },
+  {
+    name: "plugin-views",
+    configDir: "plugins/* with build:views",
+    coveredBy: "app",
+    reason:
+      "Plugin view packages are registered and clicked through inside packages/app/test/ui-smoke plugin and app interaction suites.",
+  },
+  {
+    name: "feed-web",
+    configDir: "packages/feed",
+    coveredBy: "feed-dag-visualizer",
+    reason:
+      "The full Feed web e2e lanes require an external app stack, wallet extension, and optional DB/services; deterministic standalone Feed UI recording is covered by the DAG visualizer suite.",
+  },
+];
+
+export const SKIPPED_EXTERNAL_UI_E2E_SUITES = [
+  {
+    name: "feed-browser-wallet",
+    configDir: "packages/feed/tools/e2e",
+    script: "test",
+    reason:
+      "Requires RUN_FEED_E2E=1, a live Feed app on :3000, and the MetaMask extension.",
+  },
+  {
+    name: "feed-chroma-wallet",
+    configDir: "packages/feed/tools/chroma",
+    script: "test",
+    reason:
+      "Requires RUN_FEED_E2E=1, a Feed app on :3100, Anvil, deployed contracts, and wallet state.",
+  },
+];
+
+export function suiteByName(name) {
+  return UI_E2E_SUITES.find((suite) => suite.name === name) ?? null;
+}

scripts/evidence-doctor.mjs

@@ -0,0 +1,232 @@
+#!/usr/bin/env node
+/**
+ * Preflight health check for the evidence-capture toolchain. An agent runs this
+ * before capturing PR evidence to learn which tools are present and — for each
+ * missing one — the exact command to install or start it, so a missing binary
+ * turns into a fixable instruction instead of a silent `skipped` record deep in
+ * a capture run.
+ *
+ * Every probe reports `ok` / `missing` / `optional-missing` with a concrete
+ * `fix` string; nothing here fabricates availability. Default exit is 0 (a
+ * report, never a blocker); `--strict` exits non-zero when a REQUIRED tool is
+ * missing so CI or a capture wrapper can gate on it. `--json` prints the same
+ * findings machine-readably for the evidence harness to fold into a bundle.
+ *
+ * The required set is the floor every environment needs for the baseline
+ * screenshot + OCR + video evidence the PR gate demands (playwright browsers,
+ * ffmpeg, tesseract). The optional set unlocks richer verification (GPU/Baidu
+ * OCR, Apple Vision, API- or CLI-driven VLM review) and degrades to an honest
+ * skip when absent — so it is reported, never failed on.
+ */
+
+import { execFile } from "node:child_process";
+import { existsSync } from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { pathToFileURL } from "node:url";
+import { promisify } from "node:util";
+
+const execFileAsync = promisify(execFile);
+
+/** Run `bin args…`; resolve the trimmed stdout, or null when the probe fails. */
+async function probeCommand(bin, args, timeoutMs = 10_000) {
+  try {
+    const { stdout, stderr } = await execFileAsync(bin, args, {
+      timeout: timeoutMs,
+    });
+    return `${stdout}${stderr}`.trim();
+  } catch {
+    return null;
+  }
+}
+
+function firstLine(text) {
+  return (text ?? "").split("\n")[0]?.trim() ?? "";
+}
+
+/** Where `scripts/gpu-vision/serve.mjs` records a ready OCR server. */
+function gpuVisionServeStatePath() {
+  const override = process.env.ELIZA_GPU_VISION_CACHE?.trim();
+  const root = override
+    ? path.resolve(override)
+    : path.join(os.homedir(), ".cache", "eliza", "gpu-vision");
+  return path.join(root, "serve.json");
+}
+
+/**
+ * One probe result. `required` findings that are `missing` fail `--strict`;
+ * optional ones only inform. `fix` is the literal command a reader can paste.
+ */
+export async function runProbes(env) {
+  const probes = [];
+
+  // ---- Required: the baseline screenshot + OCR + video evidence floor. ----
+
+  const tesseract = await probeCommand(
+    env.ELIZA_TESSERACT_BIN || "tesseract",
+    ["--version"],
+  );
+  probes.push({
+    id: "tesseract",
+    required: true,
+    ok: tesseract !== null,
+    detail: tesseract ? firstLine(tesseract) : "tesseract not on PATH",
+    fix: "macOS: brew install tesseract · Debian/Ubuntu: sudo apt-get install -y tesseract-ocr · (the packaged tesseract.js fallback needs no system binary: ELIZA_MVP_OCR_ENGINE=packaged)",
+  });
+
+  const ffmpeg = await probeCommand("ffmpeg", ["-version"]);
+  probes.push({
+    id: "ffmpeg",
+    required: true,
+    ok: ffmpeg !== null,
+    detail: ffmpeg
+      ? firstLine(ffmpeg)
+      : "ffmpeg not on PATH (ffmpeg-static ships with @elizaos/evidence for keyframe/walkthrough work)",
+    fix: "macOS: brew install ffmpeg · Debian/Ubuntu: sudo apt-get install -y ffmpeg · or rely on the bundled ffmpeg-static in packages/evidence",
+  });
+
+  const playwrightChromium = existsSyncPlaywrightBrowsers();
+  probes.push({
+    id: "playwright-browsers",
+    required: true,
+    ok: playwrightChromium,
+    detail: playwrightChromium
+      ? "Playwright browser cache present"
+      : "no Playwright browser cache found",
+    fix: "bunx playwright install chromium  (add --with-deps on Linux CI)",
+  });
+
+  // ---- Optional: richer verification; each degrades to an honest skip. ----
+
+  const gpuServe = gpuVisionServeStatePath();
+  const gpuUrl = env.ELIZA_GPU_VISION_URL?.trim();
+  probes.push({
+    id: "gpu-vision-ocr",
+    required: false,
+    ok: Boolean(gpuUrl) || existsSync(gpuServe),
+    detail: gpuUrl
+      ? `ELIZA_GPU_VISION_URL=${gpuUrl}`
+      : existsSync(gpuServe)
+        ? `serve.json present at ${gpuServe}`
+        : "GPU/Baidu Unlimited-OCR server not running (falls back to tesseract)",
+    fix: "node scripts/gpu-vision/setup.mjs && node scripts/gpu-vision/serve.mjs  (GPU host; sets ELIZA_GPU_VISION_URL / serve.json)",
+  });
+
+  const appleVision =
+    process.platform === "darwin"
+      ? await probeCommand("swift", ["--version"])
+      : null;
+  probes.push({
+    id: "apple-vision-ocr",
+    required: false,
+    ok: appleVision !== null,
+    detail:
+      process.platform !== "darwin"
+        ? "apple-vision OCR is macOS-only"
+        : appleVision
+          ? firstLine(appleVision)
+          : "swift toolchain not installed",
+    fix: "macOS only: xcode-select --install  (enables the on-device Vision OCR helper)",
+  });
+
+  const visionApi =
+    Boolean(env.ANTHROPIC_API_KEY?.trim()) ||
+    Boolean(env.OPENAI_API_KEY?.trim()) ||
+    Boolean(env.ELIZA_VISION_QA_BASE_URL?.trim());
+  probes.push({
+    id: "vlm-vision-qa-api",
+    required: false,
+    ok: visionApi,
+    detail: visionApi
+      ? "an API/base-url backend is configured for vision-qa"
+      : "no ANTHROPIC_API_KEY / OPENAI_API_KEY / ELIZA_VISION_QA_BASE_URL set",
+    fix: "export ANTHROPIC_API_KEY=… (or OPENAI_API_KEY, or ELIZA_VISION_QA_BASE_URL for a local llama-server)",
+  });
+
+  const claudeCli = await probeCommand("claude", ["--version"]);
+  const codexCli = await probeCommand("codex", ["--version"]);
+  probes.push({
+    id: "coding-agent-cli",
+    required: false,
+    ok: claudeCli !== null || codexCli !== null,
+    detail: [
+      claudeCli ? `claude ${firstLine(claudeCli)}` : null,
+      codexCli ? `codex ${firstLine(codexCli)}` : null,
+    ]
+      .filter(Boolean)
+      .join(" · ") || "neither claude nor codex CLI on PATH",
+    fix: "install the Claude Code CLI or the Codex CLI to let vision-qa review screenshots via an already-authed coding agent (ELIZA_VISION_QA_BACKEND=cli)",
+  });
+
+  return probes;
+}
+
+/** True when a Playwright browser cache exists in any of the usual locations. */
+function existsSyncPlaywrightBrowsers() {
+  const override = process.env.PLAYWRIGHT_BROWSERS_PATH?.trim();
+  const candidates = [
+    override,
+    path.join(os.homedir(), "Library", "Caches", "ms-playwright"),
+    path.join(os.homedir(), ".cache", "ms-playwright"),
+    path.join(
+      process.env.LOCALAPPDATA || os.homedir(),
+      "ms-playwright",
+    ),
+  ].filter(Boolean);
+  return candidates.some((dir) => existsSync(dir));
+}
+
+export function summarize(probes) {
+  const requiredMissing = probes.filter((p) => p.required && !p.ok);
+  const optionalMissing = probes.filter((p) => !p.required && !p.ok);
+  return { requiredMissing, optionalMissing, ok: requiredMissing.length === 0 };
+}
+
+function printHuman(probes) {
+  console.log("Evidence toolchain doctor\n");
+  for (const probe of probes) {
+    const tag = probe.ok ? "ok  " : probe.required ? "MISS" : "opt ";
+    console.log(`  [${tag}] ${probe.id}: ${probe.detail}`);
+    if (!probe.ok) console.log(`         fix: ${probe.fix}`);
+  }
+  const { requiredMissing, optionalMissing, ok } = summarize(probes);
+  console.log("");
+  if (ok) {
+    console.log(
+      `Required tools present. ${optionalMissing.length} optional capability(ies) unavailable (evidence degrades to honest skips).`,
+    );
+  } else {
+    console.log(
+      `${requiredMissing.length} REQUIRED tool(s) missing: ${requiredMissing
+        .map((p) => p.id)
+        .join(", ")}. Install them before capturing evidence (see fix lines above).`,
+    );
+  }
+}
+
+async function main() {
+  const args = process.argv.slice(2);
+  if (args.includes("--help") || args.includes("-h")) {
+    console.log(
+      "Usage: node scripts/evidence-doctor.mjs [--json] [--strict]\n\n" +
+        "  --json    Print findings as JSON.\n" +
+        "  --strict  Exit non-zero when a REQUIRED tool is missing.\n",
+    );
+    return;
+  }
+  const probes = await runProbes(process.env);
+  const { ok } = summarize(probes);
+  if (args.includes("--json")) {
+    console.log(JSON.stringify({ ok, probes }, null, 2));
+  } else {
+    printHuman(probes);
+  }
+  if (args.includes("--strict") && !ok) process.exit(1);
+}
+
+if (
+  process.argv[1] &&
+  import.meta.url === pathToFileURL(process.argv[1]).href
+) {
+  await main();
+}

scripts/evidence-doctor.test.mjs

@@ -0,0 +1,54 @@
+/**
+ * Tests for the evidence-toolchain doctor. `runProbes` shells out to real
+ * binaries, so these assert the shape and classification invariants (every
+ * probe carries a fix, required-missing drives strict failure) rather than the
+ * presence of any given tool, which varies by host.
+ */
+
+import assert from "node:assert/strict";
+import { describe, it } from "node:test";
+import { runProbes, summarize } from "./evidence-doctor.mjs";
+
+describe("evidence-doctor", () => {
+  it("returns a probe for every capability with a pasteable fix", async () => {
+    const probes = await runProbes({});
+    const ids = probes.map((p) => p.id);
+    for (const id of [
+      "tesseract",
+      "ffmpeg",
+      "playwright-browsers",
+      "gpu-vision-ocr",
+      "apple-vision-ocr",
+      "vlm-vision-qa-api",
+      "coding-agent-cli",
+    ]) {
+      assert.ok(ids.includes(id), `missing probe: ${id}`);
+    }
+    for (const probe of probes) {
+      assert.equal(typeof probe.ok, "boolean");
+      assert.equal(typeof probe.required, "boolean");
+      assert.ok(probe.fix.length > 0, `${probe.id} lacks a fix`);
+    }
+  });
+
+  it("keys strict failure on required-missing only", () => {
+    const optionalOnly = summarize([
+      { id: "a", required: true, ok: true, fix: "x" },
+      { id: "b", required: false, ok: false, fix: "x" },
+    ]);
+    assert.equal(optionalOnly.ok, true);
+    assert.equal(optionalOnly.optionalMissing.length, 1);
+
+    const requiredGap = summarize([
+      { id: "a", required: true, ok: false, fix: "x" },
+    ]);
+    assert.equal(requiredGap.ok, false);
+    assert.equal(requiredGap.requiredMissing.length, 1);
+  });
+
+  it("detects an API vision backend from env", async () => {
+    const probes = await runProbes({ ANTHROPIC_API_KEY: "sk-test" });
+    const api = probes.find((p) => p.id === "vlm-vision-qa-api");
+    assert.equal(api.ok, true);
+  });
+});

scripts/evidence-review/evidence-review.test.mjs

@@ -0,0 +1,204 @@
+/**
+ * Unit tests for the evidence-review classifier and screenshot heuristics.
+ */
+
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import test from "node:test";
+import { fileURLToPath } from "node:url";
+import { analyzeImageFile, classifyArtifactPath, inferSource } from "./lib.mjs";
+
+const WHITE_PIXEL_PNG = Buffer.from(
+  "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+/p9sAAAAASUVORK5CYII=",
+  "base64",
+);
+
+const REPO_ROOT = path.resolve(
+  fileURLToPath(import.meta.url),
+  "..",
+  "..",
+  "..",
+);
+const GENERATE = "scripts/evidence-review/generate.mjs";
+
+/** Write a minimal schema-1 evidence bundle with a screenshot and a log. */
+async function writeBundle(dir, { runId = "bundle-run-001" } = {}) {
+  await mkdir(path.join(dir, "screens"), { recursive: true });
+  await writeFile(path.join(dir, "screens", "a.png"), WHITE_PIXEL_PNG);
+  await writeFile(
+    path.join(dir, "notes.log"),
+    "hello from the bundle\nsecond\n",
+  );
+  const now = new Date().toISOString();
+  const entry = (p, kind, bytes) => ({
+    path: p,
+    sha256: "0".repeat(64),
+    bytes,
+    kind,
+    source: "unit-audit",
+    producedBy: "evidence-review.test",
+    createdAt: now,
+  });
+  await writeFile(
+    path.join(dir, "manifest.json"),
+    JSON.stringify(
+      {
+        schema: 1,
+        runId,
+        createdAt: now,
+        metaSha256: "0".repeat(64),
+        artifacts: [
+          entry("screens/a.png", "screenshot", WHITE_PIXEL_PNG.length),
+          entry("notes.log", "log", 28),
+        ],
+      },
+      null,
+      2,
+    ),
+  );
+}
+
+/** Run the reviewer CLI under node from the repo root. */
+function runGenerate(args) {
+  return spawnSync("node", [GENERATE, ...args], {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+  });
+}
+
+test("classifies supported evidence artifact types", () => {
+  assert.equal(classifyArtifactPath("shot.png"), "image");
+  assert.equal(classifyArtifactPath("walkthrough.mp4"), "video");
+  assert.equal(classifyArtifactPath("server.log"), "log");
+  assert.equal(classifyArtifactPath("trajectory.jsonl"), "trajectory");
+  assert.equal(classifyArtifactPath("report.json"), "report");
+  assert.equal(classifyArtifactPath("index.html"), "viewer");
+  assert.equal(classifyArtifactPath("trace.zip"), "archive");
+  assert.equal(classifyArtifactPath("archive.bin"), null);
+});
+
+test("infers the standard evidence source directories", () => {
+  const root = "/repo";
+  assert.equal(
+    inferSource(root, "/repo/packages/app/aesthetic-audit-output/report.json"),
+    "app-audit",
+  );
+  assert.equal(
+    inferSource(root, "/repo/e2e-recordings/app/test-results/x/trace.zip"),
+    "e2e-recordings",
+  );
+  assert.equal(
+    inferSource(root, "/repo/reports/live-test-runs/run/trajectory.jsonl"),
+    "live-test-runs",
+  );
+  assert.equal(
+    inferSource(root, "/repo/device-e2e-output/android/run.json"),
+    "device-e2e",
+  );
+  assert.equal(
+    inferSource(root, "/repo/packages/app/reports/walkthrough/run/steps.json"),
+    "walkthrough",
+  );
+  assert.equal(
+    inferSource(root, "/repo/packages/scenario-runner/reports/run.jsonl"),
+    "scenario-runner",
+  );
+  assert.equal(inferSource(root, "/repo/evidence/matrix-run.json"), "evidence");
+});
+
+test("flags one-color screenshots and summarizes dominant colors", async () => {
+  const tmpDir = await mkdtemp(path.join(os.tmpdir(), "evidence-review-"));
+  try {
+    const imagePath = path.join(tmpDir, "solid.png");
+    await writeFile(imagePath, WHITE_PIXEL_PNG);
+
+    const analysis = await analyzeImageFile(imagePath);
+    assert.equal(analysis.width, 1);
+    assert.equal(analysis.height, 1);
+    assert.equal(analysis.colorBuckets, 1);
+    assert.match(analysis.issues.join(" "), /one color/);
+    assert.match(analysis.issues.join(" "), /near-solid/);
+    assert.equal(analysis.dominantColors[0].hex, "#ffffff");
+  } finally {
+    await rm(tmpDir, { recursive: true, force: true });
+  }
+});
+
+test("--bundle reviews an evidence bundle's manifest without silo scanning", async () => {
+  const tmpDir = await mkdtemp(path.join(os.tmpdir(), "evidence-bundle-"));
+  try {
+    const bundleDir = path.join(tmpDir, "bundle");
+    const outDir = path.join(tmpDir, "out");
+    await writeBundle(bundleDir);
+
+    const result = runGenerate([
+      `--bundle=${bundleDir}`,
+      `--out=${outDir}`,
+      "--ocr=off",
+      "--no-open",
+    ]);
+    assert.equal(result.status, 0, `generate failed: ${result.stderr}`);
+
+    const manifest = JSON.parse(
+      await readFile(path.join(outDir, "manifest.json"), "utf8"),
+    );
+    // Bare --bundle reviews only the bundle, so no silo dirs were scanned.
+    assert.deepEqual(manifest.scanDirs, []);
+    assert.equal(manifest.artifacts.length, 2);
+
+    const shot = manifest.artifacts.find((a) => a.type === "image");
+    assert.ok(shot, "screenshot artifact present");
+    assert.equal(shot.source, "unit-audit");
+    assert.equal(shot.bundleRunId, "bundle-run-001");
+    // The bundle screenshot ran through the shared image heuristics.
+    assert.ok(shot.image && shot.image.dominantColors.length > 0);
+
+    const log = manifest.artifacts.find((a) => a.type === "log");
+    assert.ok(log, "log artifact present");
+    assert.match(log.preview, /hello from the bundle/);
+  } finally {
+    await rm(tmpDir, { recursive: true, force: true });
+  }
+});
+
+test("--bundle fails fast when a manifest lists a missing file", async () => {
+  const tmpDir = await mkdtemp(path.join(os.tmpdir(), "evidence-bundle-"));
+  try {
+    const bundleDir = path.join(tmpDir, "bundle");
+    await mkdir(bundleDir, { recursive: true });
+    await writeFile(
+      path.join(bundleDir, "manifest.json"),
+      JSON.stringify({
+        schema: 1,
+        runId: "broken",
+        createdAt: new Date().toISOString(),
+        metaSha256: "0".repeat(64),
+        artifacts: [
+          {
+            path: "screens/missing.png",
+            sha256: "0".repeat(64),
+            bytes: 1,
+            kind: "screenshot",
+            source: "unit-audit",
+            producedBy: "test",
+            createdAt: new Date().toISOString(),
+          },
+        ],
+      }),
+    );
+
+    const result = runGenerate([
+      `--bundle=${bundleDir}`,
+      `--out=${path.join(tmpDir, "out")}`,
+      "--ocr=off",
+      "--no-open",
+    ]);
+    assert.notEqual(result.status, 0);
+    assert.match(result.stderr, /missing from the bundle/);
+  } finally {
+    await rm(tmpDir, { recursive: true, force: true });
+  }
+});

scripts/evidence-review/generate.mjs

@@ -0,0 +1,707 @@
+#!/usr/bin/env node
+/**
+ * Local evidence reviewer for screenshots, videos, logs, trajectories, and
+ * reports produced by the repo's existing verification lanes. It scans the
+ * evidence silos, computes deterministic image heuristics, runs packaged OCR,
+ * writes `evidence/manifest.json`, and generates a single browser dashboard for
+ * the manual "capturing is not reviewing" pass.
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import {
+  closeOcrEngines,
+  ocrImage,
+  resolveOcrEngine,
+} from "../../packages/app/scripts/mvp-visual-verify/ocr.mjs";
+import {
+  analyzeImageFile,
+  classifyArtifactPath,
+  htmlEscape,
+  inferSource,
+  summarizeTextPreview,
+  toPosixPath,
+} from "./lib.mjs";
+
+// sharp is not imported here: all pixel work runs inside
+// @elizaos/evidence/visual-primitives (reached via lib.mjs and ocr.mjs), which
+// owns the only `sharp` this repo resolves from a root-level script. Importing
+// it here fails module resolution (sharp is nested under the evidence package),
+// which is why analyzeImageFile no longer takes a sharp handle.
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const DEFAULT_OUTPUT_DIR = path.join(REPO_ROOT, "evidence");
+const MAX_TEXT_BYTES = 256 * 1024;
+const DEFAULT_SCAN_DIRS = [
+  "evidence",
+  "e2e-recordings",
+  "device-e2e-output",
+  "packages/app/aesthetic-audit-output",
+  "packages/app/device-e2e-output",
+  "packages/app/ios/build/boot-capture",
+  "packages/app/ios/build/device-logs",
+  "packages/app/test-results",
+  "packages/app/reports/walkthrough",
+  "packages/scenario-runner/reports",
+  "reports/live-test-runs",
+  "reports/walkthrough",
+];
+const SKIP_DIR_NAMES = new Set([
+  ".git",
+  "node_modules",
+  ".turbo",
+  "dist",
+  "build",
+  ".codex-pr-worktrees",
+]);
+
+function parseArgs(argv) {
+  const options = {
+    outputDir: DEFAULT_OUTPUT_DIR,
+    open: false,
+    ocr: "on",
+    maxArtifacts: 900,
+    maxImages: 240,
+    maxFilesPerDir: 6000,
+    scanDirs: [],
+    bundleDir: null,
+  };
+  for (const arg of argv) {
+    if (arg === "--open") options.open = true;
+    else if (arg === "--no-open") options.open = false;
+    else if (arg.startsWith("--out=")) {
+      options.outputDir = path.resolve(REPO_ROOT, arg.slice("--out=".length));
+    } else if (arg.startsWith("--ocr=")) {
+      options.ocr = arg.slice("--ocr=".length);
+    } else if (arg.startsWith("--max-images=")) {
+      options.maxImages = Number.parseInt(
+        arg.slice("--max-images=".length),
+        10,
+      );
+    } else if (arg.startsWith("--max-artifacts=")) {
+      options.maxArtifacts = Number.parseInt(
+        arg.slice("--max-artifacts=".length),
+        10,
+      );
+    } else if (arg.startsWith("--max-files-per-dir=")) {
+      options.maxFilesPerDir = Number.parseInt(
+        arg.slice("--max-files-per-dir=".length),
+        10,
+      );
+    } else if (arg.startsWith("--source=")) {
+      options.scanDirs.push(arg.slice("--source=".length));
+    } else if (arg.startsWith("--bundle=")) {
+      options.bundleDir = path.resolve(
+        REPO_ROOT,
+        arg.slice("--bundle=".length),
+      );
+    } else if (arg === "--help" || arg === "-h") {
+      printHelp();
+      process.exit(0);
+    } else {
+      throw new Error(`unknown argument: ${arg}`);
+    }
+  }
+  if (!Number.isFinite(options.maxImages) || options.maxImages < 0) {
+    throw new Error("--max-images must be a non-negative number");
+  }
+  if (!Number.isFinite(options.maxArtifacts) || options.maxArtifacts < 100) {
+    throw new Error("--max-artifacts must be at least 100");
+  }
+  if (!["auto", "on", "off"].includes(options.ocr)) {
+    throw new Error("--ocr must be auto, on, or off");
+  }
+  if (
+    !Number.isFinite(options.maxFilesPerDir) ||
+    options.maxFilesPerDir < 100
+  ) {
+    throw new Error("--max-files-per-dir must be at least 100");
+  }
+  return options;
+}
+
+function printHelp() {
+  console.log(`Usage: node scripts/evidence-review/generate.mjs [options]
+
+Options:
+  --open                  Open the generated dashboard in the browser.
+  --no-open               Do not open the dashboard.
+  --out=<dir>             Output directory. Default: evidence/
+  --source=<dir>          Scan a specific directory. Repeatable.
+  --bundle=<dir>          Read an evidence bundle's manifest.json (the @elizaos/evidence
+                          BundleManifest inventory) and review its artifacts. Used alone
+                          it reviews only that bundle; add --source to also scan silos.
+  --ocr=on|auto|off       Run OCR with the packaged tesseract.js engine. Default: on.
+  --max-artifacts=<n>     Limit total artifacts in the dashboard. Default: 900.
+  --max-images=<n>        Limit image heuristic work. Default: 240.
+  --max-files-per-dir=<n> Bound each scan root. Default: 6000.`);
+}
+
+function dirExists(dirPath) {
+  try {
+    return fs.statSync(dirPath).isDirectory();
+  } catch {
+    return false;
+  }
+}
+
+function resolveScanDirs(options) {
+  const dirs =
+    options.scanDirs.length > 0 ? options.scanDirs : DEFAULT_SCAN_DIRS;
+  return dirs
+    .map((dir) => path.resolve(REPO_ROOT, dir))
+    .filter((dir) => dirExists(dir));
+}
+
+async function runOcr(filePath, options) {
+  if (options.ocr === "off") {
+    return { status: "disabled", text: "" };
+  }
+  const result = await ocrImage(filePath);
+  if (!result.available) {
+    return {
+      status: options.ocr === "on" ? "failed" : "unavailable",
+      text: "",
+      error: result.reason,
+    };
+  }
+  return {
+    status: "ok",
+    engine: result.engine,
+    text: summarizeTextPreview(result.text, 1800).trim(),
+  };
+}
+
+function readTextPreview(filePath) {
+  try {
+    const stat = fs.statSync(filePath);
+    if (stat.size > MAX_TEXT_BYTES) {
+      const fd = fs.openSync(filePath, "r");
+      try {
+        const buffer = Buffer.alloc(MAX_TEXT_BYTES);
+        const bytesRead = fs.readSync(fd, buffer, 0, MAX_TEXT_BYTES, 0);
+        return `${summarizeTextPreview(buffer.subarray(0, bytesRead).toString("utf8"))}\n...[truncated ${stat.size - bytesRead} bytes]`;
+      } finally {
+        fs.closeSync(fd);
+      }
+    }
+    return summarizeTextPreview(fs.readFileSync(filePath, "utf8"));
+  } catch (error) {
+    return `Could not read preview: ${error?.message || error}`;
+  }
+}
+
+function walkFiles(scanRoot, maxFiles) {
+  const files = [];
+  const stack = [scanRoot];
+  while (stack.length > 0 && files.length < maxFiles) {
+    const dir = stack.pop();
+    let entries;
+    try {
+      entries = fs.readdirSync(dir, { withFileTypes: true });
+    } catch {
+      continue;
+    }
+    for (const entry of entries) {
+      if (files.length >= maxFiles) break;
+      const full = path.join(dir, entry.name);
+      if (entry.isDirectory()) {
+        if (!SKIP_DIR_NAMES.has(entry.name)) stack.push(full);
+      } else if (entry.isFile()) {
+        const type = classifyArtifactPath(full);
+        if (type) files.push({ full, type });
+      }
+    }
+  }
+  return files.sort((a, b) => a.full.localeCompare(b.full));
+}
+
+/**
+ * Build one reviewer artifact record: stat plus the OCR/image heuristics for
+ * screenshots and a text preview for logs/reports. Shared by the silo scan and
+ * the `--bundle` manifest reader so both render through exactly the same
+ * pipeline; `counters` carries the running id, image-analysis budget, and OCR
+ * tallies across both sources.
+ */
+async function buildArtifactRecord(full, meta, options, counters) {
+  const stat = fs.statSync(full);
+  const artifact = {
+    id: `${counters.nextId++}`,
+    type: meta.type,
+    source: meta.source,
+    path: toPosixPath(path.relative(REPO_ROOT, full)),
+    href: toPosixPath(path.relative(options.outputDir, full)),
+    bytes: stat.size,
+    mtime: stat.mtime.toISOString(),
+  };
+  if (meta.bundleRunId) artifact.bundleRunId = meta.bundleRunId;
+  if (meta.type === "image") {
+    if (options.ocr !== "off") {
+      artifact.ocr = await runOcr(full, options);
+      if (artifact.ocr.status === "ok") counters.ocrOk += 1;
+      else if (options.ocr === "on") counters.ocrFail += 1;
+    }
+    if (counters.imageAnalysis < options.maxImages) {
+      counters.imageAnalysis += 1;
+      try {
+        artifact.image = await analyzeImageFile(full);
+      } catch (error) {
+        artifact.imageError = error?.message || String(error);
+      }
+    } else {
+      artifact.imageSkipped = "max image analysis limit reached";
+    }
+  } else if (
+    meta.type === "log" ||
+    meta.type === "report" ||
+    meta.type === "trajectory" ||
+    meta.type === "viewer"
+  ) {
+    artifact.preview = readTextPreview(full);
+  }
+  return artifact;
+}
+
+/**
+ * The @elizaos/evidence BundleManifest artifact kinds mapped onto the reviewer's
+ * coarser artifact types. `screenshot`/`keyframe` are pixels the image
+ * heuristics and OCR run over; `analysis`/`qa`/`report` render as inspectable
+ * text. An unlisted kind falls back to extension-based classification.
+ */
+const BUNDLE_KIND_TO_TYPE = {
+  screenshot: "image",
+  keyframe: "image",
+  video: "video",
+  log: "log",
+  trajectory: "trajectory",
+  report: "report",
+  analysis: "report",
+  qa: "report",
+  "html-tree": "viewer",
+};
+
+/**
+ * Read and structurally validate an evidence bundle's `manifest.json` — the
+ * signed artifact inventory @elizaos/evidence writes (schema 1). This is
+ * untrusted disk input (error-policy:J3): a missing or malformed manifest throws
+ * an explicit error rather than silently reviewing a partial/forged inventory.
+ */
+function readBundleManifest(bundleDir) {
+  const manifestPath = path.join(bundleDir, "manifest.json");
+  if (!fs.existsSync(manifestPath)) {
+    throw new Error(
+      `--bundle: no manifest.json in ${toPosixPath(path.relative(REPO_ROOT, bundleDir))} (expected an @elizaos/evidence bundle directory)`,
+    );
+  }
+  let manifest;
+  try {
+    manifest = JSON.parse(fs.readFileSync(manifestPath, "utf8"));
+  } catch (error) {
+    throw new Error(
+      `--bundle: ${manifestPath} is not valid JSON: ${error?.message || error}`,
+    );
+  }
+  if (manifest?.schema !== 1 || !Array.isArray(manifest.artifacts)) {
+    throw new Error(
+      `--bundle: ${manifestPath} is not a schema-1 bundle manifest (need { schema: 1, artifacts: [] })`,
+    );
+  }
+  return manifest;
+}
+
+/**
+ * Append reviewer artifacts read from a bundle manifest. Each listed file's
+ * absolute path is recorded in `seen` so the subsequent silo scan does not
+ * re-add the same bytes when the bundle lives under a scan root. A manifest
+ * entry whose path escapes the bundle, or names a file missing from disk, throws
+ * — a bundle's signed inventory must match its contents.
+ */
+async function collectBundleArtifacts(bundleDir, options, counters, seen, out) {
+  const manifest = readBundleManifest(bundleDir);
+  const runId = typeof manifest.runId === "string" ? manifest.runId : null;
+  for (const entry of manifest.artifacts) {
+    if (out.length >= options.maxArtifacts) break;
+    const rel = typeof entry?.path === "string" ? entry.path : "";
+    const full = path.resolve(bundleDir, rel);
+    if (full !== bundleDir && !full.startsWith(bundleDir + path.sep)) {
+      throw new Error(
+        `--bundle: artifact path ${JSON.stringify(rel)} escapes the bundle directory`,
+      );
+    }
+    if (!fs.existsSync(full)) {
+      throw new Error(
+        `--bundle: manifest lists ${rel || "(empty path)"} but it is missing from the bundle`,
+      );
+    }
+    seen.add(full);
+    const type = BUNDLE_KIND_TO_TYPE[entry.kind] ?? classifyArtifactPath(full);
+    if (!type) continue;
+    const source =
+      typeof entry.source === "string" && entry.source
+        ? entry.source
+        : inferSource(REPO_ROOT, full);
+    out.push(
+      await buildArtifactRecord(
+        full,
+        { type, source, bundleRunId: runId },
+        options,
+        counters,
+      ),
+    );
+  }
+}
+
+async function collectArtifacts(options) {
+  // A bare --bundle reviews just that bundle; the default silos are scanned only
+  // when no bundle is given, or alongside a bundle when --source is explicit.
+  const scanDirs =
+    options.bundleDir && options.scanDirs.length === 0
+      ? []
+      : resolveScanDirs(options);
+  const ocrEngine =
+    options.ocr === "off"
+      ? { available: false, kind: "disabled", label: null, reason: null }
+      : await resolveOcrEngine();
+  if (options.ocr === "on" && !ocrEngine.available) {
+    throw new Error(
+      `OCR is required but unavailable: ${ocrEngine.reason}. Run \`bun install\` so the packaged tesseract.js dependency is available, or set ELIZA_TESSERACT_BIN to a system tesseract binary.`,
+    );
+  }
+  const counters = { nextId: 1, imageAnalysis: 0, ocrOk: 0, ocrFail: 0 };
+  const artifacts = [];
+  // Absolute paths already emitted, so a bundle under a scan root is not listed
+  // twice: the bundle read wins, the silo scan skips those files.
+  const seen = new Set();
+
+  if (options.bundleDir) {
+    await collectBundleArtifacts(
+      options.bundleDir,
+      options,
+      counters,
+      seen,
+      artifacts,
+    );
+  }
+
+  for (const scanRoot of scanDirs) {
+    if (artifacts.length >= options.maxArtifacts) break;
+    const files = walkFiles(scanRoot, options.maxFilesPerDir);
+    for (const { full, type } of files) {
+      if (artifacts.length >= options.maxArtifacts) break;
+      if (seen.has(full)) continue;
+      seen.add(full);
+      artifacts.push(
+        await buildArtifactRecord(
+          full,
+          { type, source: inferSource(REPO_ROOT, full) },
+          options,
+          counters,
+        ),
+      );
+    }
+  }
+
+  return {
+    generatedAt: new Date().toISOString(),
+    repoRoot: REPO_ROOT,
+    outputDir: options.outputDir,
+    scanDirs: scanDirs.map((dir) => toPosixPath(path.relative(REPO_ROOT, dir))),
+    bundleDir: options.bundleDir
+      ? toPosixPath(path.relative(REPO_ROOT, options.bundleDir))
+      : null,
+    ocr: {
+      mode: options.ocr,
+      engine: ocrEngine.available ? ocrEngine.label : null,
+      available: options.ocr === "off" ? false : Boolean(ocrEngine.available),
+      ok: counters.ocrOk,
+      failures: counters.ocrFail,
+    },
+    artifacts,
+  };
+}
+
+function formatBytes(bytes) {
+  if (!Number.isFinite(bytes)) return "";
+  const units = ["B", "KB", "MB", "GB"];
+  let value = bytes;
+  let unit = 0;
+  while (value >= 1024 && unit < units.length - 1) {
+    value /= 1024;
+    unit += 1;
+  }
+  return `${value.toFixed(unit === 0 ? 0 : 1)} ${units[unit]}`;
+}
+
+function percent(value) {
+  if (!Number.isFinite(value)) return "0.0%";
+  return `${(value * 100).toFixed(1)}%`;
+}
+
+function artifactStatus(artifact) {
+  if (artifact.imageError) return { label: "analysis error", cls: "bad" };
+  if (artifact.image?.issues?.length) return { label: "review", cls: "warn" };
+  if (artifact.type === "viewer") return { label: "viewer", cls: "info" };
+  return { label: artifact.type, cls: "ok" };
+}
+
+function buildCard(artifact) {
+  const status = artifactStatus(artifact);
+  const dominant = artifact.image?.dominantColors ?? [];
+  const textSnippet =
+    artifact.ocr?.text ||
+    (artifact.preview ? artifact.preview.slice(0, 900) : "") ||
+    "";
+  const issueText = [
+    ...(artifact.image?.issues ?? []),
+    artifact.imageError,
+    artifact.ocr?.error,
+  ]
+    .filter(Boolean)
+    .join("; ");
+
+  let media = "";
+  if (artifact.type === "image") {
+    media = `<button class="thumb-button" data-img="${htmlEscape(artifact.href)}" data-caption="${htmlEscape(artifact.path)}"><img loading="lazy" src="${htmlEscape(artifact.href)}" alt="${htmlEscape(artifact.path)}"></button>`;
+  } else if (artifact.type === "video") {
+    media = `<video controls preload="metadata" src="${htmlEscape(artifact.href)}"></video>`;
+  } else {
+    media = `<pre>${htmlEscape(textSnippet || artifact.path)}</pre>`;
+  }
+
+  return `<article class="card" data-type="${htmlEscape(artifact.type)}" data-source="${htmlEscape(artifact.source)}" data-search="${htmlEscape(`${artifact.path} ${artifact.source} ${textSnippet} ${issueText}`.toLowerCase())}">
+    <div class="media">${media}</div>
+    <div class="body">
+      <div class="row">
+        <span class="pill ${status.cls}">${htmlEscape(status.label)}</span>
+        <span class="source">${htmlEscape(artifact.source)}</span>
+      </div>
+      <h2>${htmlEscape(path.basename(artifact.path))}</h2>
+      <p class="path">${htmlEscape(artifact.path)}</p>
+      <p class="meta">${htmlEscape(formatBytes(artifact.bytes))} | ${htmlEscape(artifact.mtime)}</p>
+      ${
+        artifact.image
+          ? `<dl class="metrics">
+              <div><dt>Size</dt><dd>${artifact.image.width}x${artifact.image.height}</dd></div>
+              <div><dt>Colors</dt><dd>${artifact.image.colorBuckets}</dd></div>
+              <div><dt>Dominant</dt><dd>${percent(artifact.image.dominantRatio)}</dd></div>
+              <div><dt>Blue</dt><dd>${percent(artifact.image.blueRatio)}</dd></div>
+              <div><dt>Orange</dt><dd>${percent(artifact.image.orangeRatio)}</dd></div>
+            </dl>`
+          : ""
+      }
+      ${
+        dominant.length
+          ? `<div class="swatches">${dominant
+              .map(
+                (color) =>
+                  `<span title="${htmlEscape(`${color.hex} ${percent(color.ratio)}`)}" style="background:${htmlEscape(color.hex)}"></span>`,
+              )
+              .join("")}</div>`
+          : ""
+      }
+      ${
+        issueText
+          ? `<p class="issues"><strong>Heuristics:</strong> ${htmlEscape(issueText)}</p>`
+          : ""
+      }
+      ${
+        artifact.ocr
+          ? `<p class="ocr"><strong>OCR:</strong> ${htmlEscape(artifact.ocr.status)}${artifact.ocr.text ? ` - ${htmlEscape(artifact.ocr.text)}` : ""}</p>`
+          : ""
+      }
+      <a class="open-link" href="${htmlEscape(artifact.href)}">Open artifact</a>
+    </div>
+  </article>`;
+}
+
+function buildHtml(manifest) {
+  const counts = manifest.artifacts.reduce(
+    (acc, artifact) => {
+      acc[artifact.type] = (acc[artifact.type] ?? 0) + 1;
+      acc.total += 1;
+      if (artifact.image?.issues?.length || artifact.imageError)
+        acc.review += 1;
+      return acc;
+    },
+    { total: 0, review: 0 },
+  );
+  const sources = [
+    ...new Set(manifest.artifacts.map((artifact) => artifact.source)),
+  ].sort();
+  const types = [
+    ...new Set(manifest.artifacts.map((artifact) => artifact.type)),
+  ].sort();
+  const ocrLabel =
+    manifest.ocr.mode === "off"
+      ? "off"
+      : `${manifest.ocr.mode}${manifest.ocr.available ? ` available (${manifest.ocr.engine})` : " unavailable"}`;
+
+  return `<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Evidence Review</title>
+<style>
+  :root { color-scheme: dark; --bg:#101010; --panel:#171717; --panel2:#202020; --line:#313131; --txt:#eeeeee; --muted:#a1a1a1; --accent:#ff6a00; --warn:#f59e0b; --bad:#ef4444; --ok:#22c55e; }
+  * { box-sizing: border-box; }
+  body { margin:0; background:var(--bg); color:var(--txt); font:13px/1.45 ui-sans-serif, system-ui, -apple-system, Segoe UI, sans-serif; }
+  header { padding:24px 28px 14px; border-bottom:1px solid var(--line); background:#0d0d0d; position:sticky; top:0; z-index:10; }
+  h1 { margin:0 0 6px; font-size:22px; letter-spacing:0; }
+  .subtitle { color:var(--muted); margin:0 0 14px; }
+  .stats { display:flex; flex-wrap:wrap; gap:8px; margin-bottom:14px; }
+  .stat { border:1px solid var(--line); background:var(--panel); border-radius:6px; padding:6px 10px; }
+  .toolbar { display:flex; flex-wrap:wrap; gap:8px; align-items:center; }
+  input, select { background:var(--panel2); color:var(--txt); border:1px solid var(--line); border-radius:6px; padding:8px 10px; min-height:36px; }
+  input { min-width:260px; flex:1; }
+  main { padding:20px 28px 48px; }
+  .grid { display:grid; grid-template-columns:repeat(auto-fill, minmax(340px, 1fr)); gap:14px; }
+  .card { background:var(--panel); border:1px solid var(--line); border-radius:8px; overflow:hidden; min-width:0; }
+  .media { background:#090909; min-height:150px; display:flex; align-items:center; justify-content:center; border-bottom:1px solid var(--line); }
+  .media img { width:100%; height:230px; object-fit:contain; display:block; }
+  .media video { width:100%; max-height:280px; display:block; }
+  .media pre { width:100%; max-height:230px; overflow:auto; margin:0; padding:12px; white-space:pre-wrap; color:#d7d7d7; }
+  .thumb-button { all:unset; display:block; width:100%; cursor:zoom-in; }
+  .body { padding:12px; }
+  .row { display:flex; align-items:center; gap:8px; margin-bottom:8px; }
+  .pill { border-radius:999px; padding:2px 8px; font-size:11px; font-weight:700; text-transform:uppercase; color:#111; }
+  .pill.ok { background:var(--ok); }
+  .pill.warn { background:var(--warn); }
+  .pill.bad { background:var(--bad); color:#fff; }
+  .pill.info { background:#b4b4b4; }
+  .source { color:var(--muted); font-size:12px; }
+  h2 { font-size:15px; line-height:1.25; margin:0 0 6px; overflow-wrap:anywhere; }
+  .path, .meta { color:var(--muted); margin:0 0 8px; overflow-wrap:anywhere; }
+  .metrics { display:grid; grid-template-columns:repeat(5, minmax(0,1fr)); gap:6px; margin:10px 0; }
+  .metrics div { background:var(--panel2); border:1px solid var(--line); border-radius:6px; padding:6px; min-width:0; }
+  dt { color:var(--muted); font-size:10px; text-transform:uppercase; }
+  dd { margin:0; font-weight:700; }
+  .swatches { display:flex; gap:5px; margin:8px 0; }
+  .swatches span { width:22px; height:22px; border-radius:4px; border:1px solid rgba(255,255,255,.18); }
+  .issues { color:#ffd08a; }
+  .ocr { color:#d5d5d5; max-height:90px; overflow:auto; }
+  .open-link { color:var(--accent); text-decoration:none; font-weight:700; }
+  #lightbox { display:none; position:fixed; inset:0; background:rgba(0,0,0,.92); z-index:99; align-items:center; justify-content:center; padding:34px; }
+  #lightbox.open { display:flex; }
+  #lightbox img { max-width:94vw; max-height:86vh; border-radius:8px; box-shadow:0 14px 70px rgba(0,0,0,.75); }
+  #lightbox button { position:fixed; top:18px; right:22px; background:var(--panel2); border:1px solid var(--line); color:var(--txt); border-radius:6px; padding:7px 11px; cursor:pointer; }
+  #caption { position:fixed; bottom:18px; left:24px; right:24px; text-align:center; color:var(--muted); overflow-wrap:anywhere; }
+  @media (max-width: 720px) { header, main { padding-left:14px; padding-right:14px; } .grid { grid-template-columns:1fr; } .metrics { grid-template-columns:repeat(2, minmax(0,1fr)); } }
+</style>
+</head>
+<body>
+<header>
+  <h1>Evidence Review</h1>
+  <p class="subtitle">Generated ${htmlEscape(manifest.generatedAt)} from ${htmlEscape([manifest.bundleDir ? `bundle ${manifest.bundleDir}` : null, ...manifest.scanDirs].filter(Boolean).join(", "))}. OCR: ${htmlEscape(ocrLabel)}.</p>
+  <div class="stats">
+    <span class="stat">${counts.total} artifacts</span>
+    <span class="stat">${counts.image ?? 0} screenshots</span>
+    <span class="stat">${counts.video ?? 0} videos</span>
+    <span class="stat">${counts.log ?? 0} logs</span>
+    <span class="stat">${counts.trajectory ?? 0} trajectories</span>
+    <span class="stat">${counts.review} need review</span>
+  </div>
+  <div class="toolbar">
+    <input id="search" type="search" placeholder="Search path, source, OCR, preview text">
+    <select id="type"><option value="">All types</option>${types.map((type) => `<option>${htmlEscape(type)}</option>`).join("")}</select>
+    <select id="source"><option value="">All sources</option>${sources.map((source) => `<option>${htmlEscape(source)}</option>`).join("")}</select>
+  </div>
+</header>
+<main><div id="grid" class="grid">${manifest.artifacts.map(buildCard).join("\n")}</div></main>
+<div id="lightbox"><button id="close">Close</button><img id="large" alt=""><div id="caption"></div></div>
+<script>
+  const search = document.getElementById("search");
+  const type = document.getElementById("type");
+  const source = document.getElementById("source");
+  const cards = [...document.querySelectorAll(".card")];
+  function applyFilters() {
+    const q = search.value.trim().toLowerCase();
+    const t = type.value;
+    const s = source.value;
+    for (const card of cards) {
+      const ok = (!q || card.dataset.search.includes(q)) && (!t || card.dataset.type === t) && (!s || card.dataset.source === s);
+      card.style.display = ok ? "" : "none";
+    }
+  }
+  search.addEventListener("input", applyFilters);
+  type.addEventListener("change", applyFilters);
+  source.addEventListener("change", applyFilters);
+  const lb = document.getElementById("lightbox");
+  const large = document.getElementById("large");
+  const caption = document.getElementById("caption");
+  document.addEventListener("click", (event) => {
+    const button = event.target.closest("[data-img]");
+    if (!button) return;
+    large.src = button.dataset.img;
+    caption.textContent = button.dataset.caption || "";
+    lb.classList.add("open");
+  });
+  document.getElementById("close").addEventListener("click", () => lb.classList.remove("open"));
+  lb.addEventListener("click", (event) => { if (event.target === lb) lb.classList.remove("open"); });
+  document.addEventListener("keydown", (event) => { if (event.key === "Escape") lb.classList.remove("open"); });
+</script>
+</body>
+</html>`;
+}
+
+function openFile(filePath) {
+  const opener =
+    process.platform === "darwin"
+      ? "open"
+      : process.platform === "win32"
+        ? "cmd"
+        : "xdg-open";
+  const args =
+    process.platform === "win32" ? ["/c", "start", "", filePath] : [filePath];
+  spawnSync(opener, args, { stdio: "ignore", detached: true });
+}
+
+async function main() {
+  const options = parseArgs(process.argv.slice(2));
+  fs.mkdirSync(options.outputDir, { recursive: true });
+
+  const manifest = await collectArtifacts(options);
+  const manifestPath = path.join(options.outputDir, "manifest.json");
+  const indexPath = path.join(options.outputDir, "index.html");
+  fs.writeFileSync(manifestPath, JSON.stringify(manifest, null, 2), "utf8");
+  fs.writeFileSync(indexPath, buildHtml(manifest), "utf8");
+
+  const counts = manifest.artifacts.reduce(
+    (acc, artifact) => {
+      acc[artifact.type] = (acc[artifact.type] ?? 0) + 1;
+      acc.total += 1;
+      return acc;
+    },
+    { total: 0 },
+  );
+
+  console.log(`Evidence manifest: ${manifestPath}`);
+  console.log(`Evidence dashboard: ${indexPath}`);
+  console.log(
+    `Artifacts: ${counts.total} total, ${counts.image ?? 0} screenshots, ${counts.video ?? 0} videos, ${counts.trajectory ?? 0} trajectories, ${counts.log ?? 0} logs.`,
+  );
+  if (!manifest.ocr.available && options.ocr !== "off") {
+    console.log(
+      "OCR: unavailable. Run `bun install` so packaged tesseract.js is installed, or set ELIZA_TESSERACT_BIN.",
+    );
+  }
+  if (options.ocr === "on" && manifest.ocr.failures > 0) {
+    console.error(`OCR: ${manifest.ocr.failures} screenshot(s) failed OCR.`);
+    process.exitCode = 1;
+  }
+  if (options.open) openFile(indexPath);
+}
+
+main()
+  .catch((error) => {
+    console.error(error?.stack || error?.message || String(error));
+    process.exitCode = 1;
+  })
+  .finally(async () => {
+    await closeOcrEngines();
+    if (process.exitCode) process.exit(process.exitCode);
+  });

scripts/evidence-review/lib.mjs

@@ -0,0 +1,93 @@
+/**
+ * Evidence artifact discovery and screenshot heuristics for the local reviewer.
+ * The generator keeps filesystem and HTML output in `generate.mjs`; this module
+ * holds the small pure pieces so the dashboard's classification rules have
+ * focused tests.
+ */
+
+import path from "node:path";
+import { analyzeImageFile as analyzeSharedImageFile } from "@elizaos/evidence/visual-primitives";
+
+export const IMAGE_EXTENSIONS = new Set([".png", ".jpg", ".jpeg", ".webp"]);
+export const VIDEO_EXTENSIONS = new Set([".mp4", ".mov", ".webm", ".m4v"]);
+export const LOG_EXTENSIONS = new Set([".log", ".txt", ".out", ".err"]);
+export const REPORT_EXTENSIONS = new Set([
+  ".json",
+  ".jsonl",
+  ".xml",
+  ".html",
+  ".md",
+]);
+export const ARCHIVE_EXTENSIONS = new Set([".zip"]);
+const ANSI_ESCAPE_PATTERN = new RegExp(
+  `${String.fromCharCode(27)}\\[[0-9;]*m`,
+  "g",
+);
+
+export function htmlEscape(value) {
+  return String(value ?? "")
+    .replace(/&/g, "&")
+    .replace(/</g, "&lt;")
+    .replace(/>/g, "&gt;")
+    .replace(/"/g, "&quot;");
+}
+
+export function toPosixPath(value) {
+  return String(value).split(path.sep).join("/");
+}
+
+export function classifyArtifactPath(filePath) {
+  const ext = path.extname(filePath).toLowerCase();
+  if (IMAGE_EXTENSIONS.has(ext)) return "image";
+  if (VIDEO_EXTENSIONS.has(ext)) return "video";
+  if (LOG_EXTENSIONS.has(ext)) return "log";
+  if (ARCHIVE_EXTENSIONS.has(ext)) return "archive";
+  if (REPORT_EXTENSIONS.has(ext)) {
+    if (ext === ".jsonl" || /trajectory|llm-calls/i.test(filePath)) {
+      return "trajectory";
+    }
+    if (ext === ".html") return "viewer";
+    return "report";
+  }
+  return null;
+}
+
+export function inferSource(root, filePath) {
+  const rel = toPosixPath(path.relative(root, filePath));
+  if (rel.startsWith("evidence/")) return "evidence";
+  if (rel.startsWith("e2e-recordings/")) return "e2e-recordings";
+  if (rel.startsWith("device-e2e-output/")) return "device-e2e";
+  if (rel.startsWith("packages/app/aesthetic-audit-output/")) {
+    return "app-audit";
+  }
+  if (rel.startsWith("packages/app/device-e2e-output/")) return "device-e2e";
+  if (rel.startsWith("packages/app/ios/build/boot-capture/")) {
+    return "ios-boot-capture";
+  }
+  if (rel.startsWith("packages/app/ios/build/device-logs/")) {
+    return "ios-device-logs";
+  }
+  if (rel.startsWith("packages/app/test-results/")) return "app-test-results";
+  if (rel.startsWith("packages/app/reports/walkthrough/")) {
+    return "walkthrough";
+  }
+  if (rel.startsWith("packages/scenario-runner/reports/")) {
+    return "scenario-runner";
+  }
+  if (rel.startsWith("reports/live-test-runs/")) return "live-test-runs";
+  if (rel.startsWith("reports/walkthrough/")) return "walkthrough";
+  if (rel.startsWith("reports/")) return "reports";
+  return "other";
+}
+
+export async function analyzeImageFile(filePath, sharp) {
+  void sharp;
+  return analyzeSharedImageFile(filePath);
+}
+
+export function summarizeTextPreview(text, maxLength = 1200) {
+  return String(text ?? "")
+    .replace(/\r/g, "")
+    .replace(ANSI_ESCAPE_PATTERN, "")
+    .slice(0, maxLength);
+}

scripts/evidence-review/reporter.mjs

@@ -0,0 +1,152 @@
+/**
+ * Human-speed streaming reporter and admin summary for the evidence matrix
+ * runner (run-matrix.mjs). Where run-matrix owns lane execution, this module
+ * owns the operator's view of it: it turns each lane's lifecycle
+ * (pending -> running -> passed/failed/skipped) into readable terminal lines as
+ * the matrix advances, then renders one end-of-run summary table an admin — or a
+ * coding agent parsing stdout — reads to see what passed, what failed, what was
+ * skipped and why, and where every artifact landed.
+ *
+ * All output goes through an injected `write` sink and an injected `now` clock
+ * so the reporter's exact status transitions and summary text are asserted
+ * against fixtures without spawning a real lane. Honesty is the invariant: a
+ * lane that cannot run (no device, no live model) is reported `skipped` with a
+ * reason and never rendered as passing.
+ */
+
+const STATUS_GLYPH = {
+  pending: "[ ]",
+  running: "[>]",
+  passed: "[+]",
+  failed: "[x]",
+  skipped: "[-]",
+};
+
+const STATUS_LABEL = {
+  pending: "PENDING",
+  running: "RUNNING",
+  passed: "PASS",
+  failed: "FAIL",
+  skipped: "SKIP",
+};
+
+export function formatDuration(ms) {
+  if (!Number.isFinite(ms) || ms < 0) return "—";
+  if (ms < 1000) return `${ms}ms`;
+  const seconds = ms / 1000;
+  if (seconds < 60) return `${seconds.toFixed(1)}s`;
+  const minutes = Math.floor(seconds / 60);
+  const rest = Math.round(seconds - minutes * 60);
+  return `${minutes}m${String(rest).padStart(2, "0")}s`;
+}
+
+/**
+ * Streaming reporter that emits one line per lane state change so an operator
+ * watching the terminal sees the matrix advance at human speed. Construct once
+ * per run with the lanes to be executed; drive it with laneStart/laneEnd/
+ * laneSkip as the runner processes each lane.
+ */
+export function createMatrixReporter({
+  write,
+  now = () => Date.now(),
+  total,
+} = {}) {
+  if (typeof write !== "function") {
+    throw new Error("createMatrixReporter requires a write(line) function");
+  }
+  if (!Number.isInteger(total) || total <= 0) {
+    throw new Error("createMatrixReporter requires a positive integer total");
+  }
+
+  let index = 0;
+  const startedAt = new Map();
+
+  const emit = (status, id, label, suffix) => {
+    const position = `[${index}/${total}]`;
+    const glyph = STATUS_GLYPH[status];
+    const badge = STATUS_LABEL[status];
+    write(`${glyph} ${position} ${badge} ${id} — ${label}${suffix}`);
+  };
+
+  return {
+    header() {
+      write(
+        `Running ${total} evidence lane${total === 1 ? "" : "s"} at human speed.`,
+      );
+    },
+    laneStart(step) {
+      index += 1;
+      startedAt.set(step.id, now());
+      emit("running", step.id, step.label, "");
+    },
+    laneEnd(step, status) {
+      const started = startedAt.get(step.id);
+      const elapsed = started == null ? null : now() - started;
+      const timing = elapsed == null ? "" : `  (${formatDuration(elapsed)})`;
+      emit(status, step.id, step.label, timing);
+    },
+    laneSkip(step, reason) {
+      index += 1;
+      emit("skipped", step.id, step.label, `  — ${reason}`);
+    },
+  };
+}
+
+function padEnd(value, width) {
+  const text = String(value);
+  return text.length >= width ? text : text + " ".repeat(width - text.length);
+}
+
+/**
+ * Render the single end-of-run summary table. `steps` are the finished lane
+ * records from the manifest (each with id/status/durationMs and optional
+ * skipReason/artifactPath). Returns the full multi-line string so callers can
+ * both print it and assert it. A `failed` lane appears explicitly as FAIL in the
+ * table and drives the returned overall status — it is never swallowed into a
+ * green summary.
+ */
+export function renderMatrixSummary(
+  steps,
+  { dashboardPath, manifestPath } = {},
+) {
+  const counts = { passed: 0, failed: 0, skipped: 0, planned: 0 };
+  for (const step of steps) {
+    counts[step.status] = (counts[step.status] ?? 0) + 1;
+  }
+  const overall =
+    counts.failed > 0
+      ? "FAILED"
+      : counts.planned === steps.length && steps.length > 0
+        ? "PLANNED"
+        : "PASSED";
+
+  const idWidth = Math.max(4, ...steps.map((s) => s.id.length));
+  const lines = [];
+  lines.push("");
+  lines.push("Evidence matrix summary");
+  lines.push("=".repeat(72));
+  for (const step of steps) {
+    const badge = STATUS_LABEL[step.status] ?? step.status.toUpperCase();
+    const timing = padEnd(formatDuration(step.durationMs), 8);
+    const note =
+      step.status === "skipped" && step.skipReason
+        ? `skip: ${step.skipReason}`
+        : step.artifactPath
+          ? `artifacts: ${step.artifactPath}`
+          : "";
+    lines.push(
+      `${padEnd(badge, 5)}  ${padEnd(step.id, idWidth)}  ${timing}  ${note}`.trimEnd(),
+    );
+  }
+  lines.push("-".repeat(72));
+  const tally =
+    overall === "PLANNED"
+      ? `${counts.planned} planned`
+      : `${counts.passed} passed, ${counts.failed} failed, ${counts.skipped} skipped`;
+  lines.push(`${overall}: ${tally}`);
+  if (manifestPath) lines.push(`Manifest: ${manifestPath}`);
+  if (dashboardPath) lines.push(`Evidence dashboard: ${dashboardPath}`);
+  lines.push("");
+
+  return { text: lines.join("\n"), overall, counts };
+}

scripts/evidence-review/reporter.test.mjs

@@ -0,0 +1,142 @@
+/**
+ * Unit tests for the human-speed streaming reporter and admin summary. Output
+ * is captured through an injected write sink and a fake monotonic clock so the
+ * exact status-transition lines and summary text are asserted deterministically,
+ * with explicit coverage that a failed lane surfaces as FAIL and is not
+ * swallowed into a green summary.
+ */
+
+import assert from "node:assert/strict";
+import test from "node:test";
+import {
+  createMatrixReporter,
+  formatDuration,
+  renderMatrixSummary,
+} from "./reporter.mjs";
+
+function collector() {
+  const lines = [];
+  return { lines, write: (line) => lines.push(line) };
+}
+
+test("formatDuration renders ms, seconds, and minutes", () => {
+  assert.equal(formatDuration(0), "0ms");
+  assert.equal(formatDuration(850), "850ms");
+  assert.equal(formatDuration(1500), "1.5s");
+  assert.equal(formatDuration(65000), "1m05s");
+  assert.equal(formatDuration(Number.NaN), "—");
+  assert.equal(formatDuration(-5), "—");
+});
+
+test("reporter streams pending->running->passed transitions with timing", () => {
+  const { lines, write } = collector();
+  let clock = 1000;
+  const reporter = createMatrixReporter({
+    write,
+    total: 2,
+    now: () => clock,
+  });
+  reporter.header();
+
+  reporter.laneStart({ id: "test-all", label: "Test matrix" });
+  clock = 3500; // 2.5s elapsed
+  reporter.laneEnd({ id: "test-all", label: "Test matrix" }, "passed");
+
+  reporter.laneStart({ id: "app-audit", label: "Visual audit" });
+  clock = 4200;
+  reporter.laneEnd({ id: "app-audit", label: "Visual audit" }, "passed");
+
+  assert.equal(lines[0], "Running 2 evidence lanes at human speed.");
+  assert.equal(lines[1], "[>] [1/2] RUNNING test-all — Test matrix");
+  assert.equal(lines[2], "[+] [1/2] PASS test-all — Test matrix  (2.5s)");
+  assert.equal(lines[3], "[>] [2/2] RUNNING app-audit — Visual audit");
+  assert.match(lines[4], /^\[\+\] \[2\/2\] PASS app-audit/);
+});
+
+test("reporter marks a failed lane as FAIL, not passed", () => {
+  const { lines, write } = collector();
+  const reporter = createMatrixReporter({ write, total: 1, now: () => 0 });
+  reporter.laneStart({ id: "e2e", label: "e2e" });
+  reporter.laneEnd({ id: "e2e", label: "e2e" }, "failed");
+  assert.match(lines.at(-1), /\[x\] \[1\/1\] FAIL e2e/);
+  assert.ok(!lines.some((l) => /PASS/.test(l)));
+});
+
+test("reporter surfaces a skipped lane with its reason", () => {
+  const { lines, write } = collector();
+  const reporter = createMatrixReporter({ write, total: 1, now: () => 0 });
+  reporter.laneSkip(
+    { id: "ios-sim-capture", label: "iOS capture" },
+    "no booted iOS Simulator",
+  );
+  assert.equal(
+    lines.at(-1),
+    "[-] [1/1] SKIP ios-sim-capture — iOS capture  — no booted iOS Simulator",
+  );
+});
+
+test("reporter rejects invalid construction", () => {
+  assert.throws(() => createMatrixReporter({ total: 1 }), /write/);
+  assert.throws(
+    () => createMatrixReporter({ write: () => {}, total: 0 }),
+    /positive integer total/,
+  );
+});
+
+test("summary tallies pass/fail/skip and reports overall FAILED on any failure", () => {
+  const steps = [
+    { id: "test-all", status: "passed", durationMs: 2500 },
+    {
+      id: "app-audit",
+      status: "failed",
+      durationMs: 900,
+      artifactPath: "packages/app/aesthetic-audit-output",
+    },
+    {
+      id: "ios-sim-capture",
+      status: "skipped",
+      durationMs: 0,
+      skipReason: "no booted iOS Simulator",
+    },
+  ];
+  const summary = renderMatrixSummary(steps, {
+    manifestPath: "/repo/evidence/matrix-run.json",
+    dashboardPath: "/repo/evidence/index.html",
+  });
+
+  assert.equal(summary.overall, "FAILED");
+  assert.deepEqual(summary.counts, {
+    passed: 1,
+    failed: 1,
+    skipped: 1,
+    planned: 0,
+  });
+  assert.match(summary.text, /FAIL {3}app-audit/);
+  assert.match(summary.text, /SKIP {3}ios-sim-capture/);
+  assert.match(summary.text, /skip: no booted iOS Simulator/);
+  assert.match(summary.text, /FAILED: 1 passed, 1 failed, 1 skipped/);
+  assert.match(
+    summary.text,
+    /Evidence dashboard: \/repo\/evidence\/index\.html/,
+  );
+});
+
+test("summary reads PASSED when no lane failed", () => {
+  const steps = [
+    { id: "test-all", status: "passed", durationMs: 10 },
+    { id: "app-audit", status: "skipped", durationMs: 0, skipReason: "off" },
+  ];
+  const summary = renderMatrixSummary(steps, {});
+  assert.equal(summary.overall, "PASSED");
+  assert.match(summary.text, /PASSED: 1 passed, 0 failed, 1 skipped/);
+});
+
+test("summary reads PLANNED when every lane is dry-run planned", () => {
+  const steps = [
+    { id: "test-all", status: "planned", durationMs: 0 },
+    { id: "app-audit", status: "planned", durationMs: 0 },
+  ];
+  const summary = renderMatrixSummary(steps, {});
+  assert.equal(summary.overall, "PLANNED");
+  assert.match(summary.text, /PLANNED: 2 planned/);
+});

scripts/evidence-review/run-matrix.mjs

@@ -0,0 +1,384 @@
+#!/usr/bin/env node
+/**
+ * Full evidence matrix runner for end-of-work verification. It executes the
+ * repo's real test, recording, audit, and device-capture lanes in sequence,
+ * streams each lane's status through the human-speed reporter (reporter.mjs) so
+ * an operator watches the run advance, writes one run manifest, opens the local
+ * evidence reviewer, and prints a single admin-readable summary of what passed,
+ * failed, or was skipped and where the artifacts landed.
+ *
+ * Device lanes whose simulator/emulator is unreachable are reported `skipped`
+ * with a reason (probeRequirement) — never dropped silently and never faked
+ * green — so the manifest is an honest record of what actually ran.
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { createMatrixReporter, renderMatrixSummary } from "./reporter.mjs";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = path.dirname(__filename);
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const DEFAULT_OUTPUT_DIR = path.join(REPO_ROOT, "evidence");
+
+export const MATRIX_STEPS = [
+  {
+    id: "test-all",
+    label: "Unit, integration, and e2e test matrix",
+    command: ["node", "packages/scripts/run-all-tests.mjs", "--all"],
+    tags: ["tests"],
+  },
+  {
+    id: "e2e-recordings",
+    label: "Recorded UI e2e sweep",
+    command: ["node", "scripts/e2e-recordings/run-all.mjs", "--review"],
+    tags: ["ui", "recordings"],
+  },
+  {
+    id: "app-audit",
+    label: "App visual audit",
+    command: ["bun", "run", "--cwd", "packages/app", "audit:app"],
+    tags: ["ui", "screenshots"],
+  },
+  {
+    id: "ios-sim-capture",
+    label: "iOS simulator capture",
+    command: ["bun", "run", "--cwd", "packages/app", "capture:ios-sim"],
+    tags: ["device", "ios"],
+    // Requires a booted iOS Simulator; probed via `xcrun simctl` so the lane is
+    // honestly skipped (not silently dropped) on a host without one.
+    requires: "ios-simulator",
+  },
+  {
+    id: "android-emu-capture",
+    label: "Android emulator capture",
+    command: ["bun", "run", "--cwd", "packages/app", "capture:android-emu"],
+    tags: ["device", "android"],
+    requires: "android-emulator",
+  },
+];
+
+/**
+ * Report whether a lane's external dependency (a device fleet member) is
+ * reachable. Returns `{ reachable, reason }`; `reason` is the operator-facing
+ * skip explanation when a device is absent. Kept side-effect-free apart from the
+ * cheap probe command so device lanes degrade to an honest SKIP rather than a
+ * fake pass or a silent drop.
+ */
+export function probeRequirement(requirement, { runProbe = spawnSync } = {}) {
+  if (!requirement) return { reachable: true, reason: null };
+  if (requirement === "ios-simulator") {
+    const result = runProbe("xcrun", ["simctl", "list", "devices", "booted"], {
+      encoding: "utf8",
+    });
+    const out = `${result.stdout ?? ""}`;
+    if (result.status === 0 && /\(Booted\)/.test(out)) {
+      return { reachable: true, reason: null };
+    }
+    return {
+      reachable: false,
+      reason: "no booted iOS Simulator (run `xcrun simctl boot <udid>`)",
+    };
+  }
+  if (requirement === "android-emulator") {
+    const result = runProbe("adb", ["devices"], { encoding: "utf8" });
+    const out = `${result.stdout ?? ""}`;
+    const hasDevice = out
+      .split("\n")
+      .slice(1)
+      .some((line) => /\tdevice$/.test(line.trim()));
+    if (result.status === 0 && hasDevice) {
+      return { reachable: true, reason: null };
+    }
+    return {
+      reachable: false,
+      reason:
+        "no attached Android device/emulator (run `emulator -avd <name>`)",
+    };
+  }
+  return {
+    reachable: false,
+    reason: `unknown requirement '${requirement}'`,
+  };
+}
+
+function printHelp() {
+  console.log(`Usage: node scripts/evidence-review/run-matrix.mjs [options]
+
+Options:
+  --only=<ids>             Comma-separated step ids to run.
+  --skip-devices           Skip iOS/Android device capture lanes.
+  --out=<dir>              Output directory for matrix-run.json and reviewer.
+  --review / --no-review   Generate the evidence reviewer after the matrix.
+  --open / --no-open       Open the reviewer after generation. Default: no-open.
+  --review-ocr=on          OCR mode passed to evidence:review. Packaged OCR is required.
+  --stop-on-failure        Stop after the first failed step.
+  --dry-run                Write a planned manifest without executing commands.
+  --help, -h               Show this help.`);
+}
+
+export function parseMatrixArgs(argv) {
+  const options = {
+    only: null,
+    skipDevices: false,
+    outputDir: DEFAULT_OUTPUT_DIR,
+    review: true,
+    open: false,
+    reviewOcr: "on",
+    stopOnFailure: false,
+    dryRun: false,
+  };
+
+  for (const arg of argv) {
+    if (arg === "--skip-devices") options.skipDevices = true;
+    else if (arg === "--review") options.review = true;
+    else if (arg === "--no-review") options.review = false;
+    else if (arg === "--open") options.open = true;
+    else if (arg === "--no-open") options.open = false;
+    else if (arg === "--stop-on-failure") options.stopOnFailure = true;
+    else if (arg === "--dry-run") options.dryRun = true;
+    else if (arg.startsWith("--only=")) {
+      options.only = arg
+        .slice("--only=".length)
+        .split(",")
+        .map((step) => step.trim())
+        .filter(Boolean);
+    } else if (arg.startsWith("--out=")) {
+      options.outputDir = path.resolve(REPO_ROOT, arg.slice("--out=".length));
+    } else if (arg.startsWith("--review-ocr=")) {
+      options.reviewOcr = arg.slice("--review-ocr=".length);
+    } else if (arg === "--help" || arg === "-h") {
+      options.help = true;
+    } else {
+      throw new Error(`unknown argument: ${arg}`);
+    }
+  }
+
+  if (options.reviewOcr !== "on") {
+    throw new Error(
+      "--review-ocr must be on; OCR is required for evidence review and uses the packaged tesseract.js dependency",
+    );
+  }
+  return options;
+}
+
+export function selectMatrixSteps(steps, options) {
+  const selected = steps.filter((step) => {
+    if (options.skipDevices && step.tags.includes("device")) return false;
+    if (options.only) return options.only.includes(step.id);
+    return true;
+  });
+
+  if (options.only) {
+    const known = new Set(steps.map((step) => step.id));
+    const unknown = options.only.filter((id) => !known.has(id));
+    if (unknown.length > 0) {
+      throw new Error(`unknown matrix step(s): ${unknown.join(", ")}`);
+    }
+  }
+
+  // A filter combination that selects nothing (e.g. `--skip-devices
+  // --only=ios-sim-capture`) is an operator mistake, not a passing run. Fail
+  // here with an actionable message instead of letting the empty set reach the
+  // reporter, which would throw the opaque "positive integer total" error.
+  if (selected.length === 0) {
+    throw new Error(
+      "no lanes selected - check --only/--skip filters (they exclude every matrix step)",
+    );
+  }
+  return selected;
+}
+
+function resolveCommand(command) {
+  const [bin, ...args] = command;
+  return [bin === "node" ? process.execPath : bin, args];
+}
+
+function formatCommand(command) {
+  return command.join(" ");
+}
+
+function runStep(step) {
+  const startedAt = new Date().toISOString();
+  const startedMs = Date.now();
+  const [bin, args] = resolveCommand(step.command);
+  const result = spawnSync(bin, args, {
+    cwd: REPO_ROOT,
+    stdio: "inherit",
+    env: { ...process.env },
+  });
+  const exitCode = result.status ?? 1;
+  return {
+    ...step,
+    command: formatCommand(step.command),
+    startedAt,
+    finishedAt: new Date().toISOString(),
+    durationMs: Date.now() - startedMs,
+    exitCode,
+    status: exitCode === 0 ? "passed" : "failed",
+  };
+}
+
+function plannedStep(step) {
+  return {
+    ...step,
+    command: formatCommand(step.command),
+    startedAt: null,
+    finishedAt: null,
+    durationMs: 0,
+    exitCode: null,
+    status: "planned",
+  };
+}
+
+function skippedStep(step, reason) {
+  return {
+    ...step,
+    command: formatCommand(step.command),
+    startedAt: null,
+    finishedAt: null,
+    durationMs: 0,
+    exitCode: null,
+    status: "skipped",
+    skipReason: reason,
+  };
+}
+
+function writeManifest(options, steps, reviewer) {
+  fs.mkdirSync(options.outputDir, { recursive: true });
+  const manifest = {
+    schemaVersion: 1,
+    generatedAt: new Date().toISOString(),
+    repoRoot: REPO_ROOT,
+    outputDir: options.outputDir,
+    options: {
+      skipDevices: options.skipDevices,
+      only: options.only,
+      review: options.review,
+      open: options.open,
+      reviewOcr: options.reviewOcr,
+      stopOnFailure: options.stopOnFailure,
+      dryRun: options.dryRun,
+    },
+    status: steps.some((step) => step.status === "failed")
+      ? "failed"
+      : options.dryRun
+        ? "planned"
+        : "passed",
+    steps,
+    reviewer,
+  };
+  const manifestPath = path.join(options.outputDir, "matrix-run.json");
+  fs.writeFileSync(manifestPath, JSON.stringify(manifest, null, 2), "utf8");
+  return { manifest, manifestPath };
+}
+
+function runReviewer(options) {
+  if (!options.review || options.dryRun) return null;
+  const script = path.join(
+    REPO_ROOT,
+    "scripts",
+    "evidence-review",
+    "generate.mjs",
+  );
+  const args = [
+    script,
+    `--out=${options.outputDir}`,
+    `--ocr=${options.reviewOcr}`,
+    options.open ? "--open" : "--no-open",
+  ];
+  const result = spawnSync(process.execPath, args, {
+    cwd: REPO_ROOT,
+    stdio: "inherit",
+    env: { ...process.env },
+  });
+  return {
+    command: `node ${args.join(" ")}`,
+    exitCode: result.status ?? 1,
+    status: result.status === 0 ? "passed" : "failed",
+    dashboardPath: path.join(options.outputDir, "index.html"),
+  };
+}
+
+/**
+ * Execute the selected lanes, driving the streaming reporter through each
+ * lane's lifecycle. Device lanes whose requirement is unreachable are recorded
+ * as `skipped` with the probe reason rather than run. Extracted from main() so
+ * the ordering of reporter transitions and lane records is unit-testable with
+ * injected reporter and probe.
+ */
+export function executeSteps(
+  steps,
+  options,
+  { reporter, probe = probeRequirement } = {},
+) {
+  const results = [];
+  for (const step of steps) {
+    if (options.dryRun) {
+      results.push(plannedStep(step));
+      continue;
+    }
+
+    const requirement = probe(step.requires);
+    if (!requirement.reachable) {
+      reporter?.laneSkip(step, requirement.reason);
+      results.push(skippedStep(step, requirement.reason));
+      continue;
+    }
+
+    reporter?.laneStart(step);
+    const result = runStep(step);
+    reporter?.laneEnd(step, result.status);
+    results.push(result);
+    if (result.status === "failed" && options.stopOnFailure) break;
+  }
+  return results;
+}
+
+async function main() {
+  const options = parseMatrixArgs(process.argv.slice(2));
+  if (options.help) {
+    printHelp();
+    return;
+  }
+
+  const steps = selectMatrixSteps(MATRIX_STEPS, options);
+
+  let reporter = null;
+  if (!options.dryRun) {
+    reporter = createMatrixReporter({
+      write: (line) => console.log(line),
+      total: steps.length,
+    });
+    reporter.header();
+  }
+
+  const results = executeSteps(steps, options, { reporter });
+
+  writeManifest(options, results, null);
+  const reviewer = runReviewer(options);
+  const { manifest, manifestPath } = writeManifest(options, results, reviewer);
+
+  const summary = renderMatrixSummary(results, {
+    manifestPath,
+    dashboardPath: reviewer?.dashboardPath ?? null,
+  });
+  console.log(summary.text);
+
+  if (
+    manifest.status === "failed" ||
+    (reviewer && reviewer.status === "failed")
+  ) {
+    process.exit(1);
+  }
+}
+
+const isMain = process.argv[1] && path.resolve(process.argv[1]) === __filename;
+if (isMain) {
+  main().catch((error) => {
+    console.error(error?.stack || error?.message || String(error));
+    process.exit(1);
+  });
+}

scripts/evidence-review/run-matrix.test.mjs

@@ -0,0 +1,223 @@
+/**
+ * Unit tests for the evidence matrix runner's planning and execution logic. The
+ * option parser and step selector are exercised with no side effects; execution
+ * is proven against lightweight fixture lanes — one that exits 0 and one that
+ * exits non-zero — driven through the real spawn path so the streamed reporter
+ * transitions, the honest device-lane skip, and the not-swallowed failure are
+ * all asserted without the expensive real matrix.
+ */
+
+import assert from "node:assert/strict";
+import test from "node:test";
+import {
+  executeSteps,
+  MATRIX_STEPS,
+  parseMatrixArgs,
+  probeRequirement,
+  selectMatrixSteps,
+} from "./run-matrix.mjs";
+
+function fixtureLane(id, exitCode) {
+  return {
+    id,
+    label: `fixture ${id}`,
+    command: ["node", "-e", `process.exit(${exitCode})`],
+    tags: ["fixture"],
+  };
+}
+
+function recordingReporter() {
+  const events = [];
+  return {
+    events,
+    header() {
+      events.push(["header"]);
+    },
+    laneStart(step) {
+      events.push(["start", step.id]);
+    },
+    laneEnd(step, status) {
+      events.push(["end", step.id, status]);
+    },
+    laneSkip(step, reason) {
+      events.push(["skip", step.id, reason]);
+    },
+  };
+}
+
+test("selects all real matrix lanes by default", () => {
+  const options = parseMatrixArgs([]);
+  const steps = selectMatrixSteps(MATRIX_STEPS, options);
+  assert.deepEqual(
+    steps.map((step) => step.id),
+    [
+      "test-all",
+      "e2e-recordings",
+      "app-audit",
+      "ios-sim-capture",
+      "android-emu-capture",
+    ],
+  );
+  assert.equal(options.review, true);
+  assert.equal(options.open, false);
+  assert.equal(options.reviewOcr, "on");
+});
+
+test("can skip device lanes while keeping test and visual evidence lanes", () => {
+  const options = parseMatrixArgs(["--skip-devices"]);
+  const steps = selectMatrixSteps(MATRIX_STEPS, options);
+  assert.deepEqual(
+    steps.map((step) => step.id),
+    ["test-all", "e2e-recordings", "app-audit"],
+  );
+});
+
+test("validates explicit step ids and requires OCR", () => {
+  const options = parseMatrixArgs([
+    "--only=e2e-recordings,app-audit",
+    "--review-ocr=on",
+    "--open",
+  ]);
+  assert.equal(options.reviewOcr, "on");
+  assert.equal(options.open, true);
+  assert.deepEqual(
+    selectMatrixSteps(MATRIX_STEPS, options).map((step) => step.id),
+    ["e2e-recordings", "app-audit"],
+  );
+  assert.throws(
+    () => selectMatrixSteps(MATRIX_STEPS, parseMatrixArgs(["--only=missing"])),
+    /unknown matrix step/,
+  );
+  assert.throws(
+    () => parseMatrixArgs(["--review-ocr=off"]),
+    /--review-ocr must be on/,
+  );
+  assert.throws(
+    () => parseMatrixArgs(["--review-ocr=auto"]),
+    /--review-ocr must be on/,
+  );
+});
+
+test("a filter combination selecting zero lanes fails with an actionable message", () => {
+  // --skip-devices drops the device lanes, --only keeps only a device lane, so
+  // the intersection is empty. This must be a clear error, not an opaque
+  // reporter-constructor throw and not a fake pass.
+  assert.throws(
+    () =>
+      selectMatrixSteps(
+        MATRIX_STEPS,
+        parseMatrixArgs(["--skip-devices", "--only=ios-sim-capture"]),
+      ),
+    /no lanes selected - check --only\/--skip filters/,
+  );
+});
+
+test("probeRequirement passes lanes with no external dependency", () => {
+  assert.deepEqual(probeRequirement(null), { reachable: true, reason: null });
+  assert.deepEqual(probeRequirement(undefined), {
+    reachable: true,
+    reason: null,
+  });
+});
+
+test("probeRequirement reports an honest skip reason when no device is booted", () => {
+  const noBooted = probeRequirement("ios-simulator", {
+    runProbe: () => ({ status: 0, stdout: "== Devices ==\n" }),
+  });
+  assert.equal(noBooted.reachable, false);
+  assert.match(noBooted.reason, /no booted iOS Simulator/);
+
+  const booted = probeRequirement("ios-simulator", {
+    runProbe: () => ({ status: 0, stdout: "iPhone 16 (ABC) (Booted)\n" }),
+  });
+  assert.deepEqual(booted, { reachable: true, reason: null });
+
+  const noAndroid = probeRequirement("android-emulator", {
+    runProbe: () => ({ status: 0, stdout: "List of devices attached\n\n" }),
+  });
+  assert.equal(noAndroid.reachable, false);
+  assert.match(noAndroid.reason, /no attached Android device/);
+
+  const android = probeRequirement("android-emulator", {
+    runProbe: () => ({
+      status: 0,
+      stdout: "List of devices attached\nemulator-5554\tdevice\n",
+    }),
+  });
+  assert.deepEqual(android, { reachable: true, reason: null });
+});
+
+test("executeSteps runs real fixture lanes and streams pass/fail transitions", () => {
+  const reporter = recordingReporter();
+  const steps = [fixtureLane("green-lane", 0), fixtureLane("red-lane", 3)];
+  const results = executeSteps(steps, parseMatrixArgs([]), {
+    reporter,
+    probe: () => ({ reachable: true, reason: null }),
+  });
+
+  assert.deepEqual(
+    results.map((r) => [r.id, r.status, r.exitCode]),
+    [
+      ["green-lane", "passed", 0],
+      ["red-lane", "failed", 3],
+    ],
+  );
+  // The failure is surfaced as a real FAIL transition, never swallowed.
+  assert.deepEqual(reporter.events, [
+    ["start", "green-lane"],
+    ["end", "green-lane", "passed"],
+    ["start", "red-lane"],
+    ["end", "red-lane", "failed"],
+  ]);
+});
+
+test("executeSteps stops after first failure when --stop-on-failure is set", () => {
+  const reporter = recordingReporter();
+  const steps = [fixtureLane("red-lane", 1), fixtureLane("never-runs", 0)];
+  const results = executeSteps(steps, parseMatrixArgs(["--stop-on-failure"]), {
+    reporter,
+    probe: () => ({ reachable: true, reason: null }),
+  });
+  assert.equal(results.length, 1);
+  assert.equal(results[0].status, "failed");
+  assert.ok(!reporter.events.some((e) => e[1] === "never-runs"));
+});
+
+test("executeSteps skips an unreachable device lane with a reason, never runs it", () => {
+  const reporter = recordingReporter();
+  const steps = [
+    fixtureLane("green-lane", 0),
+    { ...fixtureLane("ios-sim-capture", 1), requires: "ios-simulator" },
+  ];
+  const results = executeSteps(steps, parseMatrixArgs([]), {
+    reporter,
+    probe: (req) =>
+      req === "ios-simulator"
+        ? { reachable: false, reason: "no booted iOS Simulator" }
+        : { reachable: true, reason: null },
+  });
+
+  const ios = results.find((r) => r.id === "ios-sim-capture");
+  assert.equal(ios.status, "skipped");
+  assert.equal(ios.skipReason, "no booted iOS Simulator");
+  assert.equal(ios.exitCode, null);
+  assert.deepEqual(reporter.events, [
+    ["start", "green-lane"],
+    ["end", "green-lane", "passed"],
+    ["skip", "ios-sim-capture", "no booted iOS Simulator"],
+  ]);
+});
+
+test("executeSteps writes planned records without running under --dry-run", () => {
+  const reporter = recordingReporter();
+  const steps = [fixtureLane("green-lane", 0)];
+  const results = executeSteps(steps, parseMatrixArgs(["--dry-run"]), {
+    reporter,
+    probe: () => {
+      throw new Error("probe must not run in dry-run");
+    },
+  });
+  assert.equal(results[0].status, "planned");
+  assert.equal(results[0].exitCode, null);
+  assert.deepEqual(reporter.events, []);
+});

scripts/generate-view-heroes.mjs

@@ -0,0 +1,221 @@
+#!/usr/bin/env node
+/**
+ * Generate clean, brand-consistent SVG hero images for plugin views that lack
+ * one. Heroes are probed at request time from `<pluginDir>/assets/hero.<ext>`
+ * by `packages/agent/src/api/views-registry.ts` (`.svg` is a supported hero
+ * extension). All existing real heroes are 1024x1024.
+ *
+ * The art itself (frame, palette, icon glyphs) is the shared, single source of
+ * truth in `@elizaos/shared` (`view-hero-art.ts`) — the same generator the
+ * agent uses for its runtime hero fallback and that view scaffolding uses to
+ * seed a new plugin's icon. This script owns curated fallback config and checks
+ * the full manifest-derived app catalog so hero omissions cannot silently ship.
+ *
+ * Output is deterministic: re-running produces byte-identical files. Run with
+ * `node scripts/generate-view-heroes.mjs` (requires `@elizaos/shared` built).
+ */
+
+import { existsSync, readdirSync, readFileSync } from "node:fs";
+import { mkdir, writeFile } from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { renderViewHeroSvg, VIEW_HERO_ICONS } from "@elizaos/shared";
+
+const repoRoot = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "..",
+);
+
+/**
+ * Discover every plugin that declares an Eliza app surface (`elizaos.app` in its
+ * package.json) by scanning the plugins manifest — the same source the view
+ * catalog reads — so the generator can never silently omit a view-bearing
+ * plugin. Returns the plugin dir names (e.g. "plugin-calendar").
+ */
+function scanAppPluginDirs() {
+  const pluginsRoot = path.join(repoRoot, "plugins");
+  if (!existsSync(pluginsRoot)) return [];
+  const dirs = [];
+  for (const name of readdirSync(pluginsRoot)) {
+    const manifestPath = path.join(pluginsRoot, name, "package.json");
+    if (!existsSync(manifestPath)) continue;
+    try {
+      const manifest = JSON.parse(readFileSync(manifestPath, "utf8"));
+      if (manifest?.elizaos?.app) dirs.push(name);
+    } catch {
+      // Unparseable manifest — skip; not a regression we own here.
+    }
+  }
+  return dirs.sort();
+}
+
+/** True when a plugin dir already ships a hero asset (svg or png). */
+function pluginHasHeroAsset(pluginDir) {
+  const assetsDir = path.join(repoRoot, "plugins", pluginDir, "assets");
+  if (!existsSync(assetsDir)) return false;
+  return readdirSync(assetsDir).some((f) => /^hero.*\.(svg|png)$/.test(f));
+}
+
+/**
+ * Curated fallback config. The manifest coverage check below is the source of
+ * truth for completeness; this list controls generated art for app plugins that
+ * need a committed fallback hero.
+ */
+const views = [
+  {
+    out: "plugins/app-model-tester/assets/hero.svg",
+    id: "model-tester",
+    label: "Model Tester",
+    hue: 25,
+    icon: VIEW_HERO_ICONS.modelTester,
+  },
+  {
+    out: "plugins/plugin-app-control/assets/hero.svg",
+    id: "views",
+    label: "Views",
+    hue: 270,
+    icon: VIEW_HERO_ICONS.views,
+  },
+  {
+    out: "plugins/plugin-blocker/assets/hero.svg",
+    id: "focus",
+    label: "Focus",
+    hue: 348,
+    icon: VIEW_HERO_ICONS.focus,
+  },
+  {
+    out: "plugins/plugin-calendar/assets/hero.svg",
+    id: "calendar",
+    label: "Calendar",
+    hue: 12,
+    icon: VIEW_HERO_ICONS.calendar,
+  },
+  {
+    out: "plugins/plugin-native-settings/assets/hero.svg",
+    id: "device-settings",
+    label: "Device Settings",
+    hue: 96,
+    icon: VIEW_HERO_ICONS.views,
+  },
+  {
+    out: "plugins/plugin-facewear/assets/hero-facewear.svg",
+    id: "facewear",
+    label: "Facewear",
+    hue: 190,
+    icon: VIEW_HERO_ICONS.headphones,
+  },
+  {
+    out: "plugins/plugin-facewear/assets/hero-smartglasses.svg",
+    id: "smartglasses",
+    label: "Smartglasses",
+    hue: 300,
+    icon: VIEW_HERO_ICONS.glasses,
+  },
+  {
+    out: "plugins/plugin-finances/assets/hero.svg",
+    id: "finances",
+    label: "Finances",
+    hue: 150,
+    icon: VIEW_HERO_ICONS.finances,
+  },
+  {
+    out: "plugins/plugin-goals/assets/hero.svg",
+    id: "goals",
+    label: "Goals",
+    hue: 38,
+    icon: VIEW_HERO_ICONS.goals,
+  },
+  {
+    out: "plugins/plugin-health/assets/hero.svg",
+    id: "health",
+    label: "Health",
+    hue: 332,
+    icon: VIEW_HERO_ICONS.health,
+  },
+  {
+    out: "plugins/plugin-inbox/assets/hero.svg",
+    id: "inbox",
+    label: "Inbox",
+    hue: 168,
+    icon: VIEW_HERO_ICONS.inbox,
+  },
+  {
+    out: "plugins/plugin-messages/assets/hero.svg",
+    id: "messages",
+    label: "Messages",
+    hue: 256,
+    icon: VIEW_HERO_ICONS.messages,
+  },
+  {
+    out: "plugins/plugin-relationships/assets/hero.svg",
+    id: "relationships",
+    label: "Relationships",
+    hue: 286,
+    icon: VIEW_HERO_ICONS.vectorBrowser,
+  },
+  {
+    out: "plugins/plugin-todos/assets/hero.svg",
+    id: "todos",
+    label: "Todos",
+    hue: 52,
+    icon: VIEW_HERO_ICONS.todos,
+  },
+  {
+    out: "plugins/plugin-vector-browser/assets/hero.svg",
+    id: "vector-browser",
+    label: "Vector Browser",
+    hue: 286,
+    icon: VIEW_HERO_ICONS.vectorBrowser,
+  },
+];
+
+async function main() {
+  const written = [];
+  for (const view of views) {
+    const svg = renderViewHeroSvg({
+      id: view.id,
+      hue: view.hue,
+      iconSvg: view.icon,
+      label: view.label,
+    });
+    const absPath = path.resolve(repoRoot, view.out);
+    await mkdir(path.dirname(absPath), { recursive: true });
+    await writeFile(absPath, svg, "utf8");
+    written.push({ path: view.out, bytes: Buffer.byteLength(svg, "utf8") });
+  }
+
+  for (const entry of written) {
+    console.log(`${String(entry.bytes).padStart(6)}  ${entry.path}`);
+  }
+  console.log(`\nWrote ${written.length} hero SVG files.`);
+
+  // Coverage, sourced from the manifest scan (#8796): every plugin that
+  // declares an Eliza app surface must ship a hero asset — either generated
+  // above (curated hue/icon) or committed directly. A plugin without one is a
+  // gap the catalog would render as an icon-only fallback.
+  const curatedDirs = new Set(
+    views.map((v) => v.out.split("/")[1]).filter(Boolean),
+  );
+  const appPlugins = scanAppPluginDirs();
+  const missing = appPlugins.filter(
+    (dir) => !pluginHasHeroAsset(dir) && !curatedDirs.has(dir),
+  );
+  console.log(
+    `\nManifest scan: ${appPlugins.length} app plugins, ${appPlugins.length - missing.length} with a hero asset.`,
+  );
+  if (missing.length > 0) {
+    console.error(
+      `\n⚠️  ${missing.length} app plugin(s) declare a surface but ship no hero asset:\n${missing
+        .map((d) => `  - plugins/${d}`)
+        .join(
+          "\n",
+        )}\nAdd a curated entry above or commit plugins/<name>/assets/hero.svg.`,
+    );
+    process.exitCode = 1;
+  }
+}
+
+main().catch((err) => {
+  console.error(err);
+  process.exit(1);
+});

scripts/generated/static-asset-manifest.json

@@ -0,0 +1,130 @@
+{
+  "app": [
+    "packages/app/public/app-heroes/database-viewer.png",
+    "packages/app/public/app-heroes/log-viewer.png",
+    "packages/app/public/app-heroes/memory-viewer.png",
+    "packages/app/public/app-heroes/plugin-viewer.png",
+    "packages/app/public/app-heroes/relationship-viewer.png",
+    "packages/app/public/app-heroes/runtime-debugger.png",
+    "packages/app/public/app-heroes/skills-viewer.png",
+    "packages/app/public/app-heroes/trajectory-viewer.png",
+    "packages/app/public/brand/background/Clouds_Loop_HQ_1080p.mp4",
+    "packages/app/public/brand/background/Clouds_Loop_Mobile_480p.mp4",
+    "packages/app/public/brand/background/clouds_background.jpg",
+    "packages/app/public/brand/banners/eliza_banner.svg",
+    "packages/app/public/brand/banners/elizacloud_banner.svg",
+    "packages/app/public/brand/banners/elizaos_banner.svg",
+    "packages/app/public/brand/concepts/billboard_concept.jpg",
+    "packages/app/public/brand/concepts/billboard_concept_1200.jpg",
+    "packages/app/public/brand/concepts/chibi_usb_concept.jpg",
+    "packages/app/public/brand/concepts/chibi_usb_concept_900.jpg",
+    "packages/app/public/brand/concepts/concept_minipc.jpg",
+    "packages/app/public/brand/concepts/concept_minipc_900.jpg",
+    "packages/app/public/brand/concepts/concept_phone.jpg",
+    "packages/app/public/brand/concepts/concept_phone_800.jpg",
+    "packages/app/public/brand/concepts/concept_usbdrive.jpg",
+    "packages/app/public/brand/concepts/concept_usbdrive_900.jpg",
+    "packages/app/public/brand/favicons/android-chrome-192x192.png",
+    "packages/app/public/brand/favicons/android-chrome-512x512.png",
+    "packages/app/public/brand/favicons/apple-touch-icon.png",
+    "packages/app/public/brand/favicons/favicon-16x16.png",
+    "packages/app/public/brand/favicons/favicon-32x32.png",
+    "packages/app/public/brand/favicons/favicon.ico",
+    "packages/app/public/brand/favicons/favicon.svg",
+    "packages/app/public/brand/logos/elizaOS_text_black.svg",
+    "packages/app/public/brand/logos/elizaOS_text_white.svg",
+    "packages/app/public/brand/logos/eliza_logotext.svg",
+    "packages/app/public/brand/logos/eliza_logotext_black.svg",
+    "packages/app/public/brand/logos/eliza_text_black.svg",
+    "packages/app/public/brand/logos/eliza_text_white.svg",
+    "packages/app/public/brand/logos/elizacloud_logotext.svg",
+    "packages/app/public/brand/logos/elizacloud_logotext_black.svg",
+    "packages/app/public/brand/logos/elizacloud_text_black.svg",
+    "packages/app/public/brand/logos/elizacloud_text_white.svg",
+    "packages/app/public/brand/logos/elizaos_logotext.svg",
+    "packages/app/public/brand/logos/elizaos_logotext_black.svg",
+    "packages/app/public/brand/logos/logo_blue_blackbg.svg",
+    "packages/app/public/brand/logos/logo_blue_nobg.svg",
+    "packages/app/public/brand/logos/logo_orange_blackbg.svg",
+    "packages/app/public/brand/logos/logo_orange_nobg.svg",
+    "packages/app/public/brand/logos/logo_white_blackbg.svg",
+    "packages/app/public/brand/logos/logo_white_bluebg.svg",
+    "packages/app/public/brand/logos/logo_white_graybg.svg",
+    "packages/app/public/brand/logos/logo_white_nobg.svg",
+    "packages/app/public/brand/logos/logo_white_orangebg.svg",
+    "packages/app/public/logos/anthropic-icon-white.png",
+    "packages/app/public/logos/anthropic-icon.png",
+    "packages/app/public/logos/claude-icon.png",
+    "packages/app/public/logos/deepseek-icon.png",
+    "packages/app/public/logos/elizaos-icon.png",
+    "packages/app/public/logos/gemini-icon.png",
+    "packages/app/public/logos/grok-icon-white.png",
+    "packages/app/public/logos/grok-icon.png",
+    "packages/app/public/logos/groq-icon-white.png",
+    "packages/app/public/logos/groq-icon.png",
+    "packages/app/public/logos/mistral-icon.png",
+    "packages/app/public/logos/ollama-icon-white.png",
+    "packages/app/public/logos/ollama-icon.png",
+    "packages/app/public/logos/openai-icon-white.png",
+    "packages/app/public/logos/openai-icon.png",
+    "packages/app/public/logos/openrouter-icon-white.png",
+    "packages/app/public/logos/openrouter-icon.png",
+    "packages/app/public/logos/together-ai-icon.png",
+    "packages/app/public/logos/zai-icon-white.png",
+    "packages/app/public/logos/zai-icon.png",
+    "packages/app/public/og-image.png",
+    "packages/app/public/runners/eliza-tasks.js",
+    "packages/app/public/site.webmanifest",
+    "packages/app/public/sw.js"
+  ],
+  "homepage": [
+    "packages/homepage/public/_headers",
+    "packages/homepage/public/_redirects",
+    "packages/homepage/public/brand/background/Clouds_Loop_HQ_1080p.mp4",
+    "packages/homepage/public/brand/background/Clouds_Loop_Mobile_480p.mp4",
+    "packages/homepage/public/brand/background/clouds_background.jpg",
+    "packages/homepage/public/brand/favicons/android-chrome-192x192.png",
+    "packages/homepage/public/brand/favicons/android-chrome-512x512.png",
+    "packages/homepage/public/brand/favicons/apple-touch-icon.png",
+    "packages/homepage/public/brand/favicons/favicon-16x16.png",
+    "packages/homepage/public/brand/favicons/favicon-32x32.png",
+    "packages/homepage/public/brand/favicons/favicon.ico",
+    "packages/homepage/public/brand/favicons/favicon.svg",
+    "packages/homepage/public/brand/logos/elizaOS_text_black.svg",
+    "packages/homepage/public/brand/logos/elizaOS_text_white.svg",
+    "packages/homepage/public/brand/logos/eliza_logotext.svg",
+    "packages/homepage/public/brand/logos/eliza_logotext_black.svg",
+    "packages/homepage/public/brand/logos/eliza_text_black.svg",
+    "packages/homepage/public/brand/logos/eliza_text_white.svg",
+    "packages/homepage/public/brand/logos/elizacloud_logotext.svg",
+    "packages/homepage/public/brand/logos/elizacloud_logotext_black.svg",
+    "packages/homepage/public/brand/logos/elizacloud_text_black.svg",
+    "packages/homepage/public/brand/logos/elizacloud_text_white.svg",
+    "packages/homepage/public/brand/logos/elizaos_logotext.svg",
+    "packages/homepage/public/brand/logos/elizaos_logotext_black.svg",
+    "packages/homepage/public/brand/logos/logo_blue_blackbg.svg",
+    "packages/homepage/public/brand/logos/logo_blue_nobg.svg",
+    "packages/homepage/public/brand/logos/logo_orange_blackbg.svg",
+    "packages/homepage/public/brand/logos/logo_orange_nobg.svg",
+    "packages/homepage/public/brand/logos/logo_white_blackbg.svg",
+    "packages/homepage/public/brand/logos/logo_white_bluebg.svg",
+    "packages/homepage/public/brand/logos/logo_white_graybg.svg",
+    "packages/homepage/public/brand/logos/logo_white_nobg.svg",
+    "packages/homepage/public/brand/logos/logo_white_orangebg.svg",
+    "packages/homepage/public/brand/ogembeds/eliza_ogembed.png",
+    "packages/homepage/public/brand/ogembeds/eliza_ogembed.svg",
+    "packages/homepage/public/brand/ogembeds/elizacloud_ogembed.png",
+    "packages/homepage/public/brand/ogembeds/elizacloud_ogembed.svg",
+    "packages/homepage/public/brand/ogembeds/elizaos_ogembed.png",
+    "packages/homepage/public/brand/ogembeds/elizaos_ogembed.svg",
+    "packages/homepage/public/eliza-app-profile-image.png",
+    "packages/homepage/public/elizapfp.png",
+    "packages/homepage/public/elizawallpaper.jpeg",
+    "packages/homepage/public/install.ps1",
+    "packages/homepage/public/install.sh",
+    "packages/homepage/public/models/iphone.glb",
+    "packages/homepage/public/product/elizaos-usb-key-concept.png",
+    "packages/homepage/public/site.webmanifest",
+    "packages/homepage/public/tbg.jpg"
+  ]
+}

scripts/git-hooks/pre-commit-gitleaks.sh

@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+# pre-commit-gitleaks.sh — block secrets before they land in a commit.
+#
+# Installation is opt-in: run scripts/security/install-git-hooks.sh, which
+# wires this into .git/hooks/pre-commit. Not auto-installed from postinstall
+# (we don't silently rewrite contributors' git config).
+
+set -euo pipefail
+
+if ! command -v gitleaks >/dev/null 2>&1; then
+  echo "gitleaks not found on PATH." >&2
+  echo "  macOS:   brew install gitleaks" >&2
+  echo "  Linux:   https://github.com/gitleaks/gitleaks/releases" >&2
+  echo "  go:      go install github.com/gitleaks/gitleaks/v8@latest" >&2
+  exit 1
+fi
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+CONFIG="${REPO_ROOT}/.gitleaks.toml"
+
+cfg_args=()
+if [[ -f "$CONFIG" ]]; then
+  cfg_args+=(--config "$CONFIG")
+fi
+
+# Scan staged changes only. `protect --staged` exits non-zero on any finding.
+gitleaks protect --staged --redact --verbose "${cfg_args[@]}"

scripts/gpu-vision/README.md

@@ -0,0 +1,137 @@
+# GPU vision service (`scripts/gpu-vision/`)
+
+Stands up the local GPU vision lane for the evidence analyzer registry
+([#14543](https://github.com/elizaOS/eliza/issues/14543), epic
+[#14541](https://github.com/elizaOS/eliza/issues/14541)): **one resident
+`llama-server` + a job queue**, not a model load per image.
+
+The default model is **Baidu Unlimited-OCR** — a 3B DeepSeek-OCR-based VLM (MIT),
+served from GGUF `sahilchachra/Unlimited-OCR-GGUF` (Q4_K_M weights + F16 mmproj).
+Note the GGUF repo is a personal HF account's conversion of Baidu's released
+weights, not a Baidu-official artifact: the revision + sha256 pin in
+`models.lock.json` guarantees **immutability** (what we serve can never silently
+change), not upstream **provenance**. An optional **Qwen3-VL-4B-Instruct** GGUF
+(`Qwen/Qwen3-VL-4B-Instruct-GGUF`, Qwen-official) covers the offline VLM Q&A
+fallback. Both are served over the same OpenAI-compatible HTTP API.
+
+This directory is **only the service half** of #14543. The analyzer-registry
+integration (enqueue images, stream results into `analysis.json`, `skipped-tier`
+records when no GPU service is reachable) is a separate task in `packages/evidence`.
+
+## Scripts
+
+| Script | Purpose |
+| --- | --- |
+| `setup.mjs` | Idempotent, revision-pinned model download with sha256 verification against `models.lock.json`. |
+| `serve.mjs` | Launch / stop a resident `llama-server`; free-port bind, `/health` readiness, PID file, optional `--verify` re-hash. |
+| `smoke.mjs` | Real end-to-end OCR test: render a fixture PNG with known text, POST it, assert the text comes back. |
+| `lib.mjs` | Shared pure logic (model pins, lockfile, version gate, arg/port helpers, readiness poller). |
+| `models.lock.json` | Pinned sha256/size/URL per blob. First real download records it; later runs verify and fail loud on drift. |
+
+Root `package.json` wiring is limited to one entry — `test:gpu-vision`,
+mirroring `test:evidence-review` (the repo's mechanism for making a
+`scripts/<family>` unit suite runnable). The operational scripts are invoked
+directly by path: `node scripts/gpu-vision/<script>.mjs`.
+
+## Quick start
+
+```bash
+# 1. Fetch the OCR model (+ mmproj) into the cache, verified against the lockfile.
+node scripts/gpu-vision/setup.mjs
+#    …or also fetch the Qwen3-VL VLM:
+node scripts/gpu-vision/setup.mjs --with-vlm
+
+# 2. Serve it (resident, OpenAI-compatible). Prints the base URL + PID file.
+node scripts/gpu-vision/serve.mjs --parallel 2
+#    …or serve the VLM instead (second instance, its own PID entry):
+node scripts/gpu-vision/serve.mjs --vlm
+#    …optionally re-hash the blobs against models.lock.json before launch.
+#    Catches on-disk corruption between setup and serve, at the cost of hashing
+#    ~2.7 GiB (a few seconds); the default stays presence-only for boot speed:
+node scripts/gpu-vision/serve.mjs --verify
+
+# 3. Prove it actually transcribes pixels.
+node scripts/gpu-vision/smoke.mjs
+
+# Stop the resident server.
+node scripts/gpu-vision/serve.mjs --stop        # add --vlm to stop the VLM instance
+
+# One-shot: setup (if needed) → serve → smoke → stop.
+node scripts/gpu-vision/smoke.mjs --start
+```
+
+## Requirements
+
+- **llama.cpp ≥ build b8525** (2026-03-25, upstream PR 17400 — DeepSeek-OCR
+  support). `serve.mjs` parses `llama-server --version` and refuses to launch on
+  anything older, with an actionable `brew upgrade llama.cpp` message.
+  - macOS: `brew install llama.cpp` (Metal backend, no extra flags).
+  - Linux CUDA: build llama.cpp with `-DGGML_CUDA=on` (or install a CUDA-enabled
+    package) so the model runs on the GPU; the CPU backend works but is slow.
+- **`hf` CLI** (from `huggingface_hub`) is used for downloads when present
+  (resumable, revision-pinned). Without it, `setup.mjs` falls back to direct
+  HTTPS `resolve/<revision>` URLs. Either path enforces the same sha256 gate.
+- **`sharp`** (already a workspace dependency) renders the smoke fixture.
+
+## VRAM / RAM expectations
+
+| Model | Weights (Q4_K_M) | mmproj (F16) | Resident footprint |
+| --- | --- | --- | --- |
+| Unlimited-OCR (3B) | 1.82 GiB | 0.76 GiB | **~4–6 GiB** VRAM (Metal unified / CUDA) |
+| Qwen3-VL-4B-Instruct | 2.33 GiB | 0.78 GiB | ~5–7 GiB VRAM |
+
+On Apple Silicon these live in unified memory via Metal; an 8 GiB machine runs
+the OCR model comfortably, 16 GiB is roomy enough to co-resident both.
+
+## MLX alternative (Apple-Silicon-only environments)
+
+For Apple-Silicon local certifiers that prefer MLX over llama.cpp/Metal, the
+same personal HF account that publishes the GGUF conversion also publishes MLX
+quants (`sahilchachra/unlimited-ocr-*-mlx`); serve them with `mlx_vlm.server`
+(also OpenAI-compatible). These scripts target the GGUF + `llama-server` path
+because it is identical across macOS Metal and Linux CUDA (one code path for
+both certifier types); MLX is a drop-in swap at the endpoint level if a box has
+no working llama.cpp build.
+
+## How the analyzer registry consumes this
+
+The registry treats the service as an **OpenAI-compatible chat completions
+endpoint** and never loads a model itself:
+
+- **Endpoint**: `POST {baseUrl}/v1/chat/completions`, where `baseUrl` is
+  `http://127.0.0.1:<port>`. The running port/PID/model are recorded in
+  `~/.cache/eliza/gpu-vision/serve.json` (respecting `ELIZA_GPU_VISION_CACHE`);
+  a fixed port can be forced with `ELIZA_GPU_VISION_PORT`.
+- **Request shape**: one user message with a `text` part (the grounding OCR
+  prompt, exported as `OCR_PROMPT` from `lib.mjs`) and an `image_url` part whose
+  URL is a `data:image/png;base64,…` inline image. `temperature: 0` for
+  reproducible transcription.
+- **Output normalization**: Unlimited-OCR decorates its transcription with
+  grounding markers of its own accord — lines come back like
+  `title [50, 128, 595, 262]ELIZA VISION LANE`. This is model behavior, not
+  something `OCR_PROMPT` requests, so consumers must strip/normalize the
+  `<tag> [x1, y1, x2, y2]` decorations (or treat them as free layout metadata)
+  before exact-matching text. `smoke.mjs` handles it with a
+  lowercase/whitespace-collapse + substring check.
+- **Doctrine**: one resident server holds the model in memory and drains a queue
+  via `--parallel N` slots — **no per-job model loads, no GPU sharing between
+  containers**. When no service is reachable, the registry records `skipped-tier`
+  honestly rather than faking a result; the cpu tier is unaffected.
+
+## Environment variables
+
+| Variable | Default | Purpose |
+| --- | --- | --- |
+| `ELIZA_GPU_VISION_CACHE` | `~/.cache/eliza/gpu-vision` | Model + state cache root. |
+| `ELIZA_GPU_VISION_PORT` | free port | Force the serve port. `smoke.mjs` prefers the per-model `serve.json` entry and uses this only as a fallback (validated, never `:NaN`). |
+
+## Tests
+
+`bun run test:gpu-vision` (i.e. `node --test scripts/gpu-vision/*.test.mjs`,
+mirroring `test:evidence-review`) — covers the pure logic: lockfile
+reconciliation, the llama.cpp version-gate boundary (b8524 rejected / b8525
+accepted), port and arg parsing, the torn-download size floor, the setup
+skip-path wiring (a present-but-wrong blob still fails the sha256 gate), and
+the readiness poller against a real in-process HTTP stub (including the
+accepts-but-never-responds abort). The download, process launch, and OCR
+request are proven by the real `smoke.mjs` run, not mocked in the unit suite.

scripts/gpu-vision/gpu-vision.test.mjs

@@ -0,0 +1,252 @@
+/**
+ * Unit tests for the GPU vision service's pure logic: lockfile reconciliation
+ * (the fail-loud integrity gate), the llama.cpp version-gate boundary, port and
+ * arg parsing, the torn-download size floor, the setup skip-path wiring (a
+ * present-but-wrong blob must still fail the sha256 gate), and the readiness
+ * poller against a real in-process HTTP stub. The download, process launch, and
+ * OCR request are exercised by the real smoke run, not here.
+ */
+
+import assert from "node:assert/strict";
+import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
+import http from "node:http";
+import os from "node:os";
+import path from "node:path";
+import test from "node:test";
+import {
+  assertLlamaBuildSupported,
+  assertPlausibleSize,
+  formatBytes,
+  lockKey,
+  MODEL_SETS,
+  modelFilePath,
+  parseArgs,
+  parseLlamaBuild,
+  parsePort,
+  parsePositiveInteger,
+  reconcileLock,
+  sha256File,
+  waitForReady,
+} from "./lib.mjs";
+import { ensureFile } from "./setup.mjs";
+
+test("model sets pin exact repo, revision, and filenames", () => {
+  assert.equal(MODEL_SETS.ocr.repo, "sahilchachra/Unlimited-OCR-GGUF");
+  assert.equal(MODEL_SETS.ocr.files.model.name, "Unlimited-OCR-Q4_K_M.gguf");
+  assert.equal(
+    MODEL_SETS.ocr.files.mmproj.name,
+    "mmproj-Unlimited-OCR-F16.gguf",
+  );
+  assert.match(MODEL_SETS.ocr.revision, /^[0-9a-f]{40}$/);
+  assert.equal(MODEL_SETS.vlm.repo, "Qwen/Qwen3-VL-4B-Instruct-GGUF");
+  assert.match(MODEL_SETS.vlm.revision, /^[0-9a-f]{40}$/);
+});
+
+test("lockKey encodes repo, revision, and file", () => {
+  assert.equal(
+    lockKey("ocr", "model"),
+    "sahilchachra/Unlimited-OCR-GGUF@0dc781d8a23f52963918ebd5b2d1b9fe61504661/Unlimited-OCR-Q4_K_M.gguf",
+  );
+});
+
+test("reconcileLock records a new pin when none exists", () => {
+  const lock = {};
+  const result = reconcileLock(lock, "k", { sha256: "abc" });
+  assert.equal(result.status, "recorded");
+  assert.equal(result.entry.sha256, "abc");
+});
+
+test("reconcileLock verifies a matching pin", () => {
+  const lock = { k: { sha256: "abc" } };
+  const result = reconcileLock(lock, "k", { sha256: "abc" });
+  assert.equal(result.status, "verified");
+});
+
+test("reconcileLock throws loud on sha256 mismatch", () => {
+  const lock = { k: { sha256: "expected" } };
+  assert.throws(
+    () => reconcileLock(lock, "k", { sha256: "different" }),
+    /sha256 mismatch/,
+  );
+});
+
+test("parseLlamaBuild extracts the build integer", () => {
+  assert.equal(
+    parseLlamaBuild("version: 9870 (2d973636e)\nbuilt with ..."),
+    9870,
+  );
+  assert.equal(parseLlamaBuild("version: 8525 (abc)"), 8525);
+  assert.equal(parseLlamaBuild("no version here"), null);
+});
+
+test("version gate boundary: b8524 rejected, b8525 accepted, garbage throws", () => {
+  assert.throws(
+    () => assertLlamaBuildSupported("version: 8524 (deadbeef)"),
+    /too old for DeepSeek-OCR/,
+  );
+  assert.equal(assertLlamaBuildSupported("version: 8525 (deadbeef)"), 8525);
+  assert.equal(assertLlamaBuildSupported("version: 9870 (2d973636e)"), 9870);
+  assert.throws(
+    () => assertLlamaBuildSupported("not a version banner"),
+    /could not parse/,
+  );
+});
+
+test("parsePort validates flag and env port values", () => {
+  assert.equal(parsePort(undefined, "X"), undefined);
+  assert.equal(parsePort("", "X"), undefined);
+  assert.equal(parsePort("  ", "X"), undefined);
+  assert.equal(parsePort("8080", "X"), 8080);
+  assert.equal(parsePort(8080, "X"), 8080);
+  assert.throws(() => parsePort("not-a-port", "X"), /integer port/);
+  assert.throws(() => parsePort("0", "X"), /integer port/);
+  assert.throws(() => parsePort("70000", "X"), /integer port/);
+  assert.throws(() => parsePort("80.5", "X"), /integer port/);
+  // A bare `--port` flag parses to boolean true — a usage error, never port 1.
+  assert.throws(() => parsePort(true, "X"), /requires a port number/);
+});
+
+test("parsePositiveInteger rejects bare value flags instead of coercing them", () => {
+  assert.equal(parsePositiveInteger(undefined, "--parallel", 2), 2);
+  assert.equal(parsePositiveInteger("", "--parallel", 2), 2);
+  assert.equal(parsePositiveInteger("4", "--parallel", 2), 4);
+  assert.equal(parsePositiveInteger(4, "--parallel", 2), 4);
+  assert.throws(
+    () => parsePositiveInteger(true, "--parallel", 2),
+    /requires a positive integer/,
+  );
+  assert.throws(
+    () => parsePositiveInteger("0", "--parallel", 2),
+    /positive integer/,
+  );
+  assert.throws(
+    () => parsePositiveInteger("1.5", "--parallel", 2),
+    /positive integer/,
+  );
+});
+
+test("assertPlausibleSize accepts near-pinned sizes and rejects torn downloads", () => {
+  assert.doesNotThrow(() => assertPlausibleSize(1000, 1000, "f"));
+  assert.doesNotThrow(() => assertPlausibleSize(960, 1000, "f"));
+  assert.throws(() => assertPlausibleSize(500, 1000, "f"), /truncated/);
+  assert.throws(() => assertPlausibleSize(2000, 1000, "f"), /truncated/);
+});
+
+test("parseArgs handles flags, key=value, key value, and positionals", () => {
+  const { flags, positionals } = parseArgs(
+    ["--with-vlm", "--parallel", "4", "--port=9999", "fixture.png"],
+    { booleans: ["with-vlm"] },
+  );
+  assert.equal(flags["with-vlm"], true);
+  assert.equal(flags.parallel, "4");
+  assert.equal(flags.port, "9999");
+  assert.deepEqual(positionals, ["fixture.png"]);
+});
+
+test("parseArgs treats a bare boolean flag before a value flag correctly", () => {
+  const { flags } = parseArgs(["--stop", "--vlm"], {
+    booleans: ["stop", "vlm"],
+  });
+  assert.equal(flags.stop, true);
+  assert.equal(flags.vlm, true);
+});
+
+test("formatBytes renders GiB and MiB", () => {
+  assert.equal(formatBytes(2 * 1024 ** 3), "2.00 GiB");
+  assert.equal(formatBytes(512 * 1024 ** 2), "512.0 MiB");
+  assert.equal(formatBytes(Number.NaN), "unknown");
+});
+
+test("ensureFile skip-if-present path still enforces the lockfile sha256 gate", async (t) => {
+  const tmp = await mkdtemp(path.join(os.tmpdir(), "gpu-vision-test-"));
+  const prevCache = process.env.ELIZA_GPU_VISION_CACHE;
+  process.env.ELIZA_GPU_VISION_CACHE = tmp;
+  t.after(async () => {
+    if (prevCache === undefined) delete process.env.ELIZA_GPU_VISION_CACHE;
+    else process.env.ELIZA_GPU_VISION_CACHE = prevCache;
+    await rm(tmp, { recursive: true, force: true });
+  });
+
+  const destPath = modelFilePath("ocr", "model");
+  await mkdir(path.dirname(destPath), { recursive: true });
+  await writeFile(destPath, "definitely not the pinned model bytes");
+  const actualSha = await sha256File(destPath);
+  const key = lockKey("ocr", "model");
+
+  // Present file whose hash contradicts the pin must fail loud — the skip
+  // path re-hashing is what makes "already downloaded" trustworthy.
+  await assert.rejects(
+    ensureFile({
+      setKey: "ocr",
+      role: "model",
+      hfBin: null,
+      lock: { [key]: { sha256: "0".repeat(64) } },
+    }),
+    /sha256 mismatch/,
+  );
+
+  // Present file matching the pin is skipped without a download.
+  const result = await ensureFile({
+    setKey: "ocr",
+    role: "model",
+    hfBin: null,
+    lock: { [key]: { sha256: actualSha } },
+  });
+  assert.equal(result.downloaded, false);
+});
+
+test("waitForReady resolves once the stub returns 200", async () => {
+  let hits = 0;
+  const server = http.createServer((_req, res) => {
+    hits += 1;
+    // Fail the first probe, succeed the second — exercises the poll loop.
+    res.statusCode = hits >= 2 ? 200 : 503;
+    res.end("ok");
+  });
+  await new Promise((r) => server.listen(0, "127.0.0.1", r));
+  const { port } = server.address();
+  try {
+    const ready = await waitForReady(`http://127.0.0.1:${port}/health`, {
+      timeoutMs: 5000,
+      intervalMs: 20,
+    });
+    assert.equal(ready, true);
+    assert.ok(hits >= 2);
+  } finally {
+    server.close();
+  }
+});
+
+test("waitForReady throws on timeout with the last error", async () => {
+  // Nothing listening on this port; the poller must give up loud, not hang.
+  await assert.rejects(
+    waitForReady("http://127.0.0.1:1/health", {
+      timeoutMs: 200,
+      intervalMs: 50,
+    }),
+    /server not ready/,
+  );
+});
+
+test("waitForReady aborts a probe that accepts but never responds", async () => {
+  // A server that accepts the TCP connection and then goes silent must not
+  // hang the poller — the per-probe abort converts it into a poll failure.
+  const server = http.createServer(() => {
+    // never respond; the probe's AbortSignal is what ends this request
+  });
+  await new Promise((r) => server.listen(0, "127.0.0.1", r));
+  const { port } = server.address();
+  try {
+    await assert.rejects(
+      waitForReady(`http://127.0.0.1:${port}/health`, {
+        timeoutMs: 400,
+        intervalMs: 50,
+        probeTimeoutMs: 100,
+      }),
+      /server not ready/,
+    );
+  } finally {
+    server.closeAllConnections();
+    server.close();
+  }
+});

scripts/gpu-vision/lib.mjs

@@ -0,0 +1,348 @@
+/**
+ * Shared pure logic for the local GPU vision service scripts (setup/serve/smoke).
+ * These scripts stand up a resident `llama-server` for Baidu Unlimited-OCR (and
+ * optionally Qwen3-VL) so the evidence analyzer registry (#14541/#14543) can hit
+ * one OpenAI-compatible endpoint instead of loading a model per job. This module
+ * holds the filesystem/version/lockfile/poller pieces that have no side effects
+ * of their own so they can be unit-tested against stubs; the .mjs entrypoints
+ * own the actual downloads, process launch, and HTTP requests.
+ *
+ * Model identity — repo id, revision, and exact filenames — is pinned here so a
+ * silent upstream re-quant can never change what we serve without a code change.
+ * The observed sha256/size of each blob lives in the checked-in models.lock.json
+ * next to this file; the first real download records it, every later run
+ * verifies against it and fails loud on drift (no fabricated success).
+ */
+
+import { createHash } from "node:crypto";
+import { createReadStream } from "node:fs";
+import fs from "node:fs/promises";
+import net from "node:net";
+import os from "node:os";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+
+export const LOCKFILE_PATH = path.join(__dirname, "models.lock.json");
+
+/**
+ * llama.cpp gained DeepSeek-OCR family support in PR 17400, merged 2026-03-25;
+ * the first tagged release containing it is build b8525. Unlimited-OCR is a
+ * DeepSeek-OCR derivative, so an older `llama-server` will refuse or mis-load
+ * the mmproj. serve.mjs enforces this floor via assertLlamaBuildSupported.
+ */
+export const MIN_LLAMA_BUILD = 8525;
+
+/**
+ * Pinned model set. `ocr` is always fetched; `vlm` only under --with-vlm.
+ * `approxBytes` is the exact blob size observed at pin time, used as a
+ * torn-download floor (assertPlausibleSize) before the slower sha256 pass; the
+ * authoritative integrity check is the sha256 in the lockfile. Revisions are the
+ * commit shas observed on the HF repos so we pin to an immutable snapshot rather
+ * than a moving branch head.
+ */
+export const MODEL_SETS = {
+  ocr: {
+    key: "ocr",
+    label: "Baidu Unlimited-OCR (DeepSeek-OCR 3B, Q4_K_M)",
+    repo: "sahilchachra/Unlimited-OCR-GGUF",
+    revision: "0dc781d8a23f52963918ebd5b2d1b9fe61504661",
+    files: {
+      model: { name: "Unlimited-OCR-Q4_K_M.gguf", approxBytes: 1950326784 },
+      mmproj: { name: "mmproj-Unlimited-OCR-F16.gguf", approxBytes: 811876448 },
+    },
+  },
+  vlm: {
+    key: "vlm",
+    label: "Qwen3-VL-4B-Instruct (Q4_K_M)",
+    repo: "Qwen/Qwen3-VL-4B-Instruct-GGUF",
+    revision: "1cd86afb9a95c410a6038ab3b40d8b578c892266",
+    files: {
+      model: {
+        name: "Qwen3VL-4B-Instruct-Q4_K_M.gguf",
+        approxBytes: 2497281664,
+      },
+      mmproj: {
+        name: "mmproj-Qwen3VL-4B-Instruct-F16.gguf",
+        approxBytes: 836180256,
+      },
+    },
+  },
+};
+
+/** Maximum tolerated deviation between a blob's on-disk size and its pinned
+ * approxBytes before the download is rejected as torn/truncated. */
+export const SIZE_TOLERANCE = 0.05;
+
+/**
+ * Torn-download floor: reject a blob whose size deviates grossly from the
+ * pinned size before the (much slower) sha256 pass, so a truncated mirror
+ * response gets a friendly early error instead of a bare hash mismatch.
+ */
+export function assertPlausibleSize(bytes, approxBytes, name) {
+  const deviation = Math.abs(bytes - approxBytes) / approxBytes;
+  if (deviation > SIZE_TOLERANCE) {
+    throw new Error(
+      `[gpu-vision] ${name} is ${formatBytes(bytes)} on disk but ~${formatBytes(approxBytes)} was expected ` +
+        `(> ${SIZE_TOLERANCE * 100}% off) — the download looks truncated or the upstream blob changed. ` +
+        "Delete the file and re-run setup.",
+    );
+  }
+}
+
+/** Grounding OCR prompt sent to the server at temp 0. The analyzer registry uses
+ * this exact string so its OCR output is reproducible across runs. */
+export const OCR_PROMPT =
+  "Convert this document image to markdown. Transcribe every visible character exactly, preserving reading order. Output only the transcribed text with no commentary.";
+
+/** Resolve the on-disk cache root. `ELIZA_GPU_VISION_CACHE` wins so CI/vast runs
+ * can redirect to a scratch volume; otherwise the conventional per-user cache. */
+export function cacheDir() {
+  const override = process.env.ELIZA_GPU_VISION_CACHE;
+  if (override?.trim()) return path.resolve(override.trim());
+  return path.join(os.homedir(), ".cache", "eliza", "gpu-vision");
+}
+
+export function serveStatePath() {
+  return path.join(cacheDir(), "serve.json");
+}
+
+/** Absolute path a given model set's file lands at once downloaded. */
+export function modelFilePath(setKey, role) {
+  const set = MODEL_SETS[setKey];
+  if (!set) throw new Error(`[gpu-vision] unknown model set: ${setKey}`);
+  const file = set.files[role];
+  if (!file) throw new Error(`[gpu-vision] unknown file role: ${role}`);
+  return path.join(cacheDir(), setKey, file.name);
+}
+
+export async function sha256File(filePath) {
+  const hash = createHash("sha256");
+  await new Promise((resolve, reject) => {
+    const stream = createReadStream(filePath);
+    stream.on("error", reject);
+    stream.on("data", (chunk) => hash.update(chunk));
+    stream.on("end", resolve);
+  });
+  return hash.digest("hex");
+}
+
+export async function readLockfile() {
+  try {
+    const raw = await fs.readFile(LOCKFILE_PATH, "utf8");
+    return JSON.parse(raw);
+  } catch (err) {
+    if (err.code === "ENOENT") return {};
+    // A present-but-corrupt lockfile is a real fault, not an empty one: surface it.
+    throw new Error(
+      `[gpu-vision] models.lock.json is unreadable: ${err.message}`,
+    );
+  }
+}
+
+export async function writeLockfile(lock) {
+  const sorted = {};
+  for (const key of Object.keys(lock).sort()) sorted[key] = lock[key];
+  await fs.writeFile(
+    LOCKFILE_PATH,
+    `${JSON.stringify(sorted, null, 2)}\n`,
+    "utf8",
+  );
+}
+
+/** Stable lockfile identity for one blob: which repo revision and file it is. */
+export function lockKey(setKey, role) {
+  const set = MODEL_SETS[setKey];
+  const file = set.files[role];
+  return `${set.repo}@${set.revision}/${file.name}`;
+}
+
+/**
+ * Compare a freshly-hashed blob against the lockfile. Returns one of:
+ *   { status: "recorded", entry }  — no prior pin; caller should record it
+ *   { status: "verified", entry }  — hash matches the pin
+ * Throws on mismatch. This is the fail-loud integrity gate: a corrupt or
+ * swapped download can never masquerade as the pinned model.
+ */
+export function reconcileLock(lock, key, observed) {
+  const existing = lock[key];
+  if (!existing) return { status: "recorded", entry: observed };
+  if (existing.sha256 !== observed.sha256) {
+    throw new Error(
+      `[gpu-vision] sha256 mismatch for ${key}\n` +
+        `  expected ${existing.sha256}\n` +
+        `  observed ${observed.sha256}\n` +
+        "  The download is corrupt or the upstream file changed. Delete the cached " +
+        "file and re-run; if upstream legitimately changed, update models.lock.json deliberately.",
+    );
+  }
+  return { status: "verified", entry: existing };
+}
+
+/**
+ * Parse the build number out of `llama-server --version` output. The first line
+ * is `version: <build> (<shorthash>)`; we key the version gate on the monotonic
+ * build integer rather than the commit hash. Returns null when unparseable.
+ */
+export function parseLlamaBuild(versionOutput) {
+  const match = /version:\s*(\d+)/i.exec(versionOutput ?? "");
+  return match ? Number(match[1]) : null;
+}
+
+/**
+ * The DeepSeek-OCR version gate: throws with an actionable upgrade message when
+ * the `--version` output is unparseable or the build predates b8525, otherwise
+ * returns the build number. Pure so the boundary (b8524 rejected / b8525
+ * accepted) is unit-testable without spawning a binary.
+ */
+export function assertLlamaBuildSupported(versionOutput) {
+  const build = parseLlamaBuild(versionOutput);
+  if (build === null) {
+    throw new Error(
+      `[gpu-vision] could not parse llama-server version from:\n${versionOutput}`,
+    );
+  }
+  if (build < MIN_LLAMA_BUILD) {
+    throw new Error(
+      `[gpu-vision] llama-server build ${build} is too old for DeepSeek-OCR models.\n` +
+        `  Need build >= ${MIN_LLAMA_BUILD} (PR 17400, merged 2026-03-25).\n` +
+        "  Upgrade: `brew upgrade llama.cpp`.",
+    );
+  }
+  return build;
+}
+
+/**
+ * Validate a TCP port from a flag or env var. Returns undefined for absent or
+ * empty input; throws on anything that is not an integer in 1-65535 so a typo'd
+ * port can never silently become `http://127.0.0.1:NaN`.
+ */
+export function parsePort(value, label) {
+  if (value === undefined || value === null) return undefined;
+  if (typeof value === "boolean") {
+    throw new Error(`[gpu-vision] ${label} requires a port number`);
+  }
+  if (String(value).trim() === "") return undefined;
+  const port = Number(value);
+  if (!Number.isInteger(port) || port < 1 || port > 65535) {
+    throw new Error(
+      `[gpu-vision] ${label} must be an integer port 1-65535, got ${value}`,
+    );
+  }
+  return port;
+}
+
+/** Validate a positive integer CLI/env value. Bare value flags parse as boolean
+ * true, which must be rejected instead of becoming Number(true) === 1. */
+export function parsePositiveInteger(value, label, defaultValue) {
+  if (value === undefined || value === null || String(value).trim() === "") {
+    return defaultValue;
+  }
+  if (typeof value === "boolean") {
+    throw new Error(`[gpu-vision] ${label} requires a positive integer`);
+  }
+  const parsed = Number(value);
+  if (!Number.isInteger(parsed) || parsed < 1) {
+    throw new Error(
+      `[gpu-vision] ${label} must be a positive integer, got ${value}`,
+    );
+  }
+  return parsed;
+}
+
+/** Human-readable byte formatter for the setup/serve reports. */
+export function formatBytes(bytes) {
+  if (!Number.isFinite(bytes)) return "unknown";
+  const gib = bytes / 1024 ** 3;
+  if (gib >= 1) return `${gib.toFixed(2)} GiB`;
+  const mib = bytes / 1024 ** 2;
+  return `${mib.toFixed(1)} MiB`;
+}
+
+/**
+ * Minimal flag parser shared by the entrypoints. Recognises `--flag`,
+ * `--key value`, and `--key=value`; everything else becomes a positional. Kept
+ * deliberately tiny — these scripts have a handful of options and pulling in a
+ * CLI framework would be more surface than the whole feature.
+ */
+export function parseArgs(argv, { booleans = [] } = {}) {
+  const booleanSet = new Set(booleans);
+  const flags = {};
+  const positionals = [];
+  for (let i = 0; i < argv.length; i += 1) {
+    const arg = argv[i];
+    if (!arg.startsWith("--")) {
+      positionals.push(arg);
+      continue;
+    }
+    const body = arg.slice(2);
+    const eq = body.indexOf("=");
+    if (eq !== -1) {
+      flags[body.slice(0, eq)] = body.slice(eq + 1);
+      continue;
+    }
+    if (booleanSet.has(body)) {
+      flags[body] = true;
+      continue;
+    }
+    const next = argv[i + 1];
+    if (next !== undefined && !next.startsWith("--")) {
+      flags[body] = next;
+      i += 1;
+    } else {
+      flags[body] = true;
+    }
+  }
+  return { flags, positionals };
+}
+
+/** Ask the OS for a free TCP port by binding :0 and reading back the assignment.
+ * Used only when no explicit port is requested — never hardcode a busy port. */
+export function findFreePort() {
+  return new Promise((resolve, reject) => {
+    const server = net.createServer();
+    server.on("error", reject);
+    server.listen(0, "127.0.0.1", () => {
+      const { port } = server.address();
+      server.close((err) => (err ? reject(err) : resolve(port)));
+    });
+  });
+}
+
+/**
+ * Poll a readiness URL until it answers 200 or the deadline passes. `fetchImpl`
+ * is injectable so the poller is testable against a stub server without touching
+ * global fetch. Each probe carries its own abort timeout so a socket that
+ * accepts but never responds cannot hang the poll (or a smoke run) forever.
+ * Returns true on ready; throws with elapsed time on timeout so a
+ * hung/mis-launched server surfaces as a hard failure rather than a silent hang.
+ */
+export async function waitForReady(
+  url,
+  {
+    timeoutMs = 120000,
+    intervalMs = 500,
+    probeTimeoutMs = 3000,
+    fetchImpl = fetch,
+  } = {},
+) {
+  const start = Date.now();
+  let lastError = "no response";
+  while (Date.now() - start < timeoutMs) {
+    try {
+      const res = await fetchImpl(url, {
+        signal: AbortSignal.timeout(probeTimeoutMs),
+      });
+      if (res.ok) return true;
+      lastError = `HTTP ${res.status}`;
+    } catch (err) {
+      lastError = err.message;
+    }
+    await new Promise((r) => setTimeout(r, intervalMs));
+  }
+  const elapsed = ((Date.now() - start) / 1000).toFixed(1);
+  throw new Error(
+    `[gpu-vision] server not ready at ${url} after ${elapsed}s (last: ${lastError})`,
+  );
+}

scripts/gpu-vision/models.lock.json

@@ -0,0 +1,34 @@
+{
+  "Qwen/Qwen3-VL-4B-Instruct-GGUF@1cd86afb9a95c410a6038ab3b40d8b578c892266/Qwen3VL-4B-Instruct-Q4_K_M.gguf": {
+    "sha256": "66358cb18bb6b3b1b6675aa412c7a88ef01d228f481184d13668e5201c730a0a",
+    "bytes": 2497281664,
+    "repo": "Qwen/Qwen3-VL-4B-Instruct-GGUF",
+    "revision": "1cd86afb9a95c410a6038ab3b40d8b578c892266",
+    "file": "Qwen3VL-4B-Instruct-Q4_K_M.gguf",
+    "url": "https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/resolve/1cd86afb9a95c410a6038ab3b40d8b578c892266/Qwen3VL-4B-Instruct-Q4_K_M.gguf"
+  },
+  "Qwen/Qwen3-VL-4B-Instruct-GGUF@1cd86afb9a95c410a6038ab3b40d8b578c892266/mmproj-Qwen3VL-4B-Instruct-F16.gguf": {
+    "sha256": "256f3a43bd4205ffef48d6b92715e1e70b5b0e9aef06522584967513a9985331",
+    "bytes": 836180256,
+    "repo": "Qwen/Qwen3-VL-4B-Instruct-GGUF",
+    "revision": "1cd86afb9a95c410a6038ab3b40d8b578c892266",
+    "file": "mmproj-Qwen3VL-4B-Instruct-F16.gguf",
+    "url": "https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct-GGUF/resolve/1cd86afb9a95c410a6038ab3b40d8b578c892266/mmproj-Qwen3VL-4B-Instruct-F16.gguf"
+  },
+  "sahilchachra/Unlimited-OCR-GGUF@0dc781d8a23f52963918ebd5b2d1b9fe61504661/Unlimited-OCR-Q4_K_M.gguf": {
+    "sha256": "c8461bded976eac709a33f6b26e1414efcd2124a203f2ee93ee984a4c9e9265b",
+    "bytes": 1950326784,
+    "repo": "sahilchachra/Unlimited-OCR-GGUF",
+    "revision": "0dc781d8a23f52963918ebd5b2d1b9fe61504661",
+    "file": "Unlimited-OCR-Q4_K_M.gguf",
+    "url": "https://huggingface.co/sahilchachra/Unlimited-OCR-GGUF/resolve/0dc781d8a23f52963918ebd5b2d1b9fe61504661/Unlimited-OCR-Q4_K_M.gguf"
+  },
+  "sahilchachra/Unlimited-OCR-GGUF@0dc781d8a23f52963918ebd5b2d1b9fe61504661/mmproj-Unlimited-OCR-F16.gguf": {
+    "sha256": "4f28c295e1fcf67a97488e356f2b4372da4702b77fdfad0fa138b5821325743c",
+    "bytes": 811876448,
+    "repo": "sahilchachra/Unlimited-OCR-GGUF",
+    "revision": "0dc781d8a23f52963918ebd5b2d1b9fe61504661",
+    "file": "mmproj-Unlimited-OCR-F16.gguf",
+    "url": "https://huggingface.co/sahilchachra/Unlimited-OCR-GGUF/resolve/0dc781d8a23f52963918ebd5b2d1b9fe61504661/mmproj-Unlimited-OCR-F16.gguf"
+  }
+}

scripts/gpu-vision/serve.mjs

@@ -0,0 +1,316 @@
+#!/usr/bin/env node
+/**
+ * Launches and supervises a resident `llama-server` for the GPU vision lane. One
+ * server holds the model in VRAM and serves an OpenAI-compatible endpoint with
+ * `--parallel N` slots; the analyzer registry queues jobs against it rather than
+ * loading a model per image. Serves Unlimited-OCR by default, or Qwen3-VL under
+ * --vlm (a second instance keyed by model in the shared serve.json).
+ *
+ * serve.json is written only AFTER /health answers, so a discovery record always
+ * points at a server that actually became ready — a fast crash (port collision,
+ * corrupt gguf, OOM) is surfaced immediately by racing the child's exit against
+ * the readiness poll, and the spawned child is torn down on any launch failure.
+ * PIDs recycle, so both --stop and the already-running check verify the recorded
+ * pid is really a llama-server (ps comm) before trusting it; stale entries are
+ * discarded with a note, never obeyed.
+ *
+ * The DeepSeek-OCR mmproj needs llama.cpp >= b8525 (PR 17400, 2026-03-25); an
+ * older or missing binary fails here with an actionable upgrade message rather
+ * than a cryptic model-load error deep in the server.
+ *
+ * Usage:
+ *   node scripts/gpu-vision/serve.mjs [--vlm] [--parallel N] [--port P] [--verify]
+ *   node scripts/gpu-vision/serve.mjs --stop [--vlm]
+ */
+
+import { spawn, spawnSync } from "node:child_process";
+import { openSync } from "node:fs";
+import fs from "node:fs/promises";
+import path from "node:path";
+import {
+  assertLlamaBuildSupported,
+  cacheDir,
+  findFreePort,
+  lockKey,
+  MODEL_SETS,
+  modelFilePath,
+  parseArgs,
+  parsePort,
+  parsePositiveInteger,
+  readLockfile,
+  reconcileLock,
+  serveStatePath,
+  sha256File,
+  waitForReady,
+} from "./lib.mjs";
+
+const CONTEXT_SIZE = 8192;
+const DEFAULT_PARALLEL = 2;
+
+function requireLlamaServer() {
+  const probe = spawnSync("llama-server", ["--version"], { encoding: "utf8" });
+  if (probe.error || probe.status !== 0) {
+    throw new Error(
+      "[gpu-vision] llama-server not found on PATH.\n" +
+        "  Install it: `brew install llama.cpp` (macOS) or build llama.cpp with your GPU backend.\n" +
+        "  The DeepSeek-OCR mmproj requires build b8525 or newer (2026-03-25+).",
+    );
+  }
+  // llama.cpp prints version to stderr.
+  return assertLlamaBuildSupported(
+    `${probe.stdout || ""}${probe.stderr || ""}`,
+  );
+}
+
+async function readState() {
+  try {
+    return JSON.parse(await fs.readFile(serveStatePath(), "utf8"));
+  } catch (err) {
+    if (err.code === "ENOENT") return {};
+    throw err;
+  }
+}
+
+async function writeState(state) {
+  await fs.mkdir(cacheDir(), { recursive: true });
+  await fs.writeFile(
+    serveStatePath(),
+    `${JSON.stringify(state, null, 2)}\n`,
+    "utf8",
+  );
+}
+
+function processAlive(pid) {
+  try {
+    process.kill(pid, 0);
+    return true;
+  } catch {
+    // error-policy:J3 kill(pid, 0) throwing ESRCH/EPERM IS the "not ours/not
+    // alive" answer this probe exists to produce, not a failure to hide.
+    return false;
+  }
+}
+
+/** PIDs recycle; before trusting a recorded pid, confirm the process it names
+ * is actually a llama-server. `ps -o comm=` is portable across macOS/Linux. */
+function pidIsLlamaServer(pid) {
+  const probe = spawnSync("ps", ["-p", String(pid), "-o", "comm="], {
+    encoding: "utf8",
+  });
+  if (probe.error || probe.status !== 0) return false;
+  return probe.stdout.trim().toLowerCase().includes("llama-server");
+}
+
+function entryIsLive(entry) {
+  return processAlive(entry.pid) && pidIsLlamaServer(entry.pid);
+}
+
+async function stop(setKey) {
+  const state = await readState();
+  const entry = state[setKey];
+  if (!entry) {
+    process.stdout.write(
+      `[gpu-vision] no recorded ${setKey} server in serve.json\n`,
+    );
+    return;
+  }
+  if (!processAlive(entry.pid)) {
+    process.stdout.write(
+      `[gpu-vision] ${setKey} server pid ${entry.pid} already gone\n`,
+    );
+  } else if (!pidIsLlamaServer(entry.pid)) {
+    process.stdout.write(
+      `[gpu-vision] pid ${entry.pid} exists but is not a llama-server (pid recycled) — ` +
+        "discarding stale serve.json entry without signaling\n",
+    );
+  } else {
+    process.kill(entry.pid, "SIGTERM");
+    process.stdout.write(
+      `[gpu-vision] stopped ${setKey} server pid ${entry.pid} (port ${entry.port})\n`,
+    );
+  }
+  delete state[setKey];
+  await writeState(state);
+}
+
+/**
+ * --verify: re-hash both blobs against models.lock.json before launch. Guards
+ * against on-disk corruption/tampering between setup and serve at the cost of
+ * hashing ~2.7 GiB; the default stays presence-only for boot speed.
+ */
+async function verifyBlobs(setKey) {
+  const lock = await readLockfile();
+  for (const role of ["model", "mmproj"]) {
+    const key = lockKey(setKey, role);
+    if (!lock[key]) {
+      throw new Error(
+        `[gpu-vision] --verify: no pin for ${key} in models.lock.json — run setup.mjs first`,
+      );
+    }
+    const filePath = modelFilePath(setKey, role);
+    const sha256 = await sha256File(filePath);
+    reconcileLock(lock, key, { sha256 });
+    process.stdout.write(
+      `[gpu-vision]   verified ${path.basename(filePath)} sha256 ok\n`,
+    );
+  }
+}
+
+async function serve({ setKey, parallel, requestedPort, verify }) {
+  const build = requireLlamaServer();
+  const set = MODEL_SETS[setKey];
+  const modelPath = modelFilePath(setKey, "model");
+  const mmprojPath = modelFilePath(setKey, "mmproj");
+  for (const p of [modelPath, mmprojPath]) {
+    await fs.access(p).catch(() => {
+      throw new Error(
+        `[gpu-vision] missing model file: ${p}\n  Run: node scripts/gpu-vision/setup.mjs${setKey === "vlm" ? " --with-vlm" : ""}`,
+      );
+    });
+  }
+  if (verify) await verifyBlobs(setKey);
+
+  const state = await readState();
+  const existing = state[setKey];
+  if (existing) {
+    if (entryIsLive(existing)) {
+      throw new Error(
+        `[gpu-vision] a ${setKey} server is already running (pid ${existing.pid}, port ${existing.port}).\n` +
+          "  Stop it first: node scripts/gpu-vision/serve.mjs --stop" +
+          (setKey === "vlm" ? " --vlm" : ""),
+      );
+    }
+    process.stdout.write(
+      `[gpu-vision] discarding stale serve.json ${setKey} entry (pid ${existing.pid} is ` +
+        `${processAlive(existing.pid) ? "not a llama-server — pid recycled" : "gone"})\n`,
+    );
+    delete state[setKey];
+    await writeState(state);
+  }
+
+  const port = requestedPort ?? (await findFreePort());
+  const baseUrl = `http://127.0.0.1:${port}`;
+
+  process.stdout.write(`[gpu-vision] launching ${set.label}\n`);
+  process.stdout.write(
+    `[gpu-vision]   llama-server build ${build}, parallel=${parallel}, ctx=${CONTEXT_SIZE}, port=${port}\n`,
+  );
+
+  const args = [
+    "-m",
+    modelPath,
+    "--mmproj",
+    mmprojPath,
+    "-c",
+    String(CONTEXT_SIZE),
+    "--parallel",
+    String(parallel),
+    "--host",
+    "127.0.0.1",
+    "--port",
+    String(port),
+  ];
+  // The server is detached so it outlives this launcher, and its stdout/stderr
+  // go to a log file rather than being inherited — inheriting the pipe would
+  // keep this launcher's process alive after readiness, which is not what a
+  // "start it and hand back the shell" command should do.
+  const logPath = path.join(cacheDir(), `llama-server.${setKey}.log`);
+  const logFd = openSync(logPath, "a");
+  const child = spawn("llama-server", args, {
+    stdio: ["ignore", logFd, logFd],
+    detached: true,
+  });
+  child.unref();
+
+  // Race readiness against the child dying: a fast crash (port collision,
+  // corrupt gguf, OOM) fails immediately with a pointer at the log instead of
+  // burning the full readiness timeout on ECONNREFUSED.
+  const childExited = new Promise((_, reject) => {
+    child.once("exit", (code, signal) => {
+      reject(
+        new Error(
+          `[gpu-vision] llama-server exited ${signal ? `on signal ${signal}` : `with code ${code}`} ` +
+            `before becoming ready — see ${logPath}`,
+        ),
+      );
+    });
+  });
+
+  try {
+    await Promise.race([waitForReady(`${baseUrl}/health`), childExited]);
+  } catch (err) {
+    if (processAlive(child.pid)) {
+      try {
+        process.kill(child.pid, "SIGTERM");
+      } catch {
+        // error-policy:J6 best-effort teardown — the child may exit between the
+        // aliveness check and the signal; the launch failure below is the error.
+      }
+    }
+    throw err;
+  }
+  // error-policy:J5 the childExited rejection is observed in the race above;
+  // after readiness wins this promise must not surface as an unhandled rejection.
+  childExited.catch(() => {});
+
+  // Record discovery state only for a server that actually answered /health,
+  // so the analyzer registry can never pick up a never-ready instance.
+  state[setKey] = {
+    port,
+    pid: child.pid,
+    model: set.files.model.name,
+    repo: set.repo,
+    revision: set.revision,
+    logPath,
+    startedAt: new Date().toISOString(),
+  };
+  await writeState(state);
+
+  process.stdout.write(`\n[gpu-vision] ${setKey} server ready\n`);
+  process.stdout.write(`[gpu-vision]   base URL:      ${baseUrl}\n`);
+  process.stdout.write(
+    `[gpu-vision]   chat endpoint: ${baseUrl}/v1/chat/completions\n`,
+  );
+  process.stdout.write(`[gpu-vision]   pid:           ${child.pid}\n`);
+  process.stdout.write(`[gpu-vision]   server log:    ${logPath}\n`);
+  process.stdout.write(`[gpu-vision]   state file:    ${serveStatePath()}\n`);
+  process.stdout.write(
+    `[gpu-vision] stop with: node scripts/gpu-vision/serve.mjs --stop${setKey === "vlm" ? " --vlm" : ""}\n`,
+  );
+  // Exit explicitly so the launcher returns the shell; the server keeps running.
+  process.exit(0);
+}
+
+async function main() {
+  const { flags } = parseArgs(process.argv.slice(2), {
+    booleans: ["vlm", "stop", "verify"],
+  });
+  const setKey = flags.vlm ? "vlm" : "ocr";
+
+  if (flags.stop) {
+    await stop(setKey);
+    return;
+  }
+
+  const parallel = parsePositiveInteger(
+    flags.parallel,
+    "--parallel",
+    DEFAULT_PARALLEL,
+  );
+  const requestedPort =
+    flags.port !== undefined
+      ? parsePort(flags.port, "--port")
+      : parsePort(process.env.ELIZA_GPU_VISION_PORT, "ELIZA_GPU_VISION_PORT");
+
+  await serve({
+    setKey,
+    parallel,
+    requestedPort,
+    verify: flags.verify === true,
+  });
+}
+
+main().catch((err) => {
+  process.stderr.write(`${err.stack || err.message}\n`);
+  process.exit(1);
+});

scripts/gpu-vision/setup.mjs

@@ -0,0 +1,219 @@
+#!/usr/bin/env node
+/**
+ * Idempotent model fetcher for the local GPU vision service. Downloads the
+ * pinned Unlimited-OCR GGUF + its F16 mmproj (and, under --with-vlm, a small
+ * Qwen3-VL Instruct GGUF + mmproj) into the gpu-vision cache, verifies every
+ * blob's sha256 against models.lock.json, and records the pin on first download.
+ *
+ * Fetch strategy: the `hf` CLI when present (resumable, revision-pinned),
+ * otherwise a direct HTTPS `resolve/<revision>` download. Either way the integrity
+ * gate is the same lockfile check — a corrupt or upstream-changed file fails loud
+ * instead of being served. Already-present + verified files are skipped, so
+ * re-running is cheap and safe.
+ *
+ * Usage:
+ *   node scripts/gpu-vision/setup.mjs [--with-vlm]
+ */
+
+import { spawnSync } from "node:child_process";
+import { createWriteStream } from "node:fs";
+import fs from "node:fs/promises";
+import path from "node:path";
+import { Readable } from "node:stream";
+import { pipeline } from "node:stream/promises";
+import { fileURLToPath } from "node:url";
+import {
+  assertPlausibleSize,
+  cacheDir,
+  formatBytes,
+  lockKey,
+  MODEL_SETS,
+  modelFilePath,
+  parseArgs,
+  readLockfile,
+  reconcileLock,
+  sha256File,
+  writeLockfile,
+} from "./lib.mjs";
+
+function hfCliPath() {
+  for (const bin of ["hf", "huggingface-cli"]) {
+    const probe = spawnSync(bin, ["version"], { encoding: "utf8" });
+    if (probe.status === 0) return bin;
+  }
+  return null;
+}
+
+function resolveUrl(set, fileName) {
+  return `https://huggingface.co/${set.repo}/resolve/${set.revision}/${fileName}`;
+}
+
+async function fileExists(p) {
+  try {
+    await fs.stat(p);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+async function downloadViaHf(hfBin, set, fileName, destDir) {
+  const result = spawnSync(
+    hfBin,
+    [
+      "download",
+      set.repo,
+      fileName,
+      "--revision",
+      set.revision,
+      "--local-dir",
+      destDir,
+    ],
+    { stdio: "inherit", encoding: "utf8" },
+  );
+  if (result.status !== 0) {
+    throw new Error(
+      `[gpu-vision] hf download failed for ${set.repo}/${fileName} (exit ${result.status})`,
+    );
+  }
+  // hf places the file at destDir/<fileName>; our layout stores it flat there.
+  return path.join(destDir, fileName);
+}
+
+async function downloadViaHttps(set, fileName, destPath) {
+  const url = resolveUrl(set, fileName);
+  process.stdout.write(`  GET ${url}\n`);
+  const res = await fetch(url, { redirect: "follow" });
+  if (!res.ok || !res.body) {
+    throw new Error(`[gpu-vision] HTTP ${res.status} downloading ${url}`);
+  }
+  const tmp = `${destPath}.partial`;
+  await pipeline(Readable.fromWeb(res.body), createWriteStream(tmp));
+  await fs.rename(tmp, destPath);
+  return destPath;
+}
+
+/**
+ * Ensure one pinned blob is present and lockfile-verified. Exported so the unit
+ * suite can prove the skip-if-present path still runs the sha256 gate — a
+ * present-but-wrong file must fail here, never be served.
+ */
+export async function ensureFile({ setKey, role, hfBin, lock }) {
+  const set = MODEL_SETS[setKey];
+  const file = set.files[role];
+  const destPath = modelFilePath(setKey, role);
+  const destDir = path.dirname(destPath);
+  await fs.mkdir(destDir, { recursive: true });
+
+  const key = lockKey(setKey, role);
+  const pinned = lock[key];
+
+  if (await fileExists(destPath)) {
+    if (pinned) {
+      const observedSha = await sha256File(destPath);
+      reconcileLock(lock, key, { sha256: observedSha });
+      const { size } = await fs.stat(destPath);
+      process.stdout.write(
+        `  skip  ${file.name} (present, sha256 verified, ${formatBytes(size)})\n`,
+      );
+      return { bytes: size, downloaded: false };
+    }
+    // Present but unpinned (first-ever run interrupted mid-record): re-hash and pin below.
+  } else {
+    process.stdout.write(
+      `  fetch ${file.name} from ${set.repo}@${set.revision.slice(0, 8)}\n`,
+    );
+    if (hfBin) {
+      const landed = await downloadViaHf(hfBin, set, file.name, destDir);
+      if (landed !== destPath && (await fileExists(landed))) {
+        await fs.rename(landed, destPath);
+      }
+    } else {
+      await downloadViaHttps(set, file.name, destPath);
+    }
+  }
+
+  // Cheap torn-download floor before the slower hash pass.
+  const { size } = await fs.stat(destPath);
+  assertPlausibleSize(size, file.approxBytes, file.name);
+
+  const observedSha = await sha256File(destPath);
+  const outcome = reconcileLock(lock, key, {
+    sha256: observedSha,
+    bytes: size,
+    repo: set.repo,
+    revision: set.revision,
+    file: file.name,
+    url: resolveUrl(set, file.name),
+  });
+  if (outcome.status === "recorded") {
+    lock[key] = outcome.entry;
+    process.stdout.write(
+      `  WARN  recorded NEW pin for ${file.name} — review the models.lock.json diff before committing\n` +
+        `        sha256=${observedSha} (${formatBytes(size)})\n`,
+    );
+  } else {
+    process.stdout.write(
+      `  ok    ${file.name} sha256 verified (${formatBytes(size)})\n`,
+    );
+  }
+  return { bytes: size, downloaded: true };
+}
+
+async function main() {
+  const { flags } = parseArgs(process.argv.slice(2), {
+    booleans: ["with-vlm"],
+  });
+  const sets = flags["with-vlm"] ? ["ocr", "vlm"] : ["ocr"];
+
+  process.stdout.write(`[gpu-vision] cache: ${cacheDir()}\n`);
+  process.stdout.write(`[gpu-vision] model sets: ${sets.join(", ")}\n`);
+
+  const hfBin = hfCliPath();
+  process.stdout.write(
+    hfBin
+      ? `[gpu-vision] using ${hfBin} for downloads\n`
+      : "[gpu-vision] hf CLI not found; using direct HTTPS resolve URLs\n",
+  );
+
+  const lock = await readLockfile();
+  let totalBytes = 0;
+  let downloadedAny = false;
+
+  for (const setKey of sets) {
+    const set = MODEL_SETS[setKey];
+    process.stdout.write(`\n[gpu-vision] ${set.label}\n`);
+    for (const role of Object.keys(set.files)) {
+      const { bytes, downloaded } = await ensureFile({
+        setKey,
+        role,
+        hfBin,
+        lock,
+      });
+      totalBytes += bytes;
+      downloadedAny = downloadedAny || downloaded;
+    }
+  }
+
+  await writeLockfile(lock);
+  process.stdout.write(
+    `\n[gpu-vision] done. total on disk: ${formatBytes(totalBytes)} across ${sets.length} set(s).\n`,
+  );
+  if (!downloadedAny) {
+    process.stdout.write(
+      "[gpu-vision] nothing to download — all files already present + verified.\n",
+    );
+  }
+}
+
+// Run only when invoked directly — the unit suite imports ensureFile from this
+// module and must not trigger a download.
+const isMain =
+  process.argv[1] &&
+  path.resolve(process.argv[1]) === fileURLToPath(import.meta.url);
+if (isMain) {
+  main().catch((err) => {
+    process.stderr.write(`${err.stack || err.message}\n`);
+    process.exit(1);
+  });
+}

scripts/gpu-vision/smoke.mjs

@@ -0,0 +1,236 @@
+#!/usr/bin/env node
+/**
+ * Real end-to-end smoke test for the GPU vision service. It renders a
+ * deterministic fixture PNG containing known text, POSTs it to a running
+ * llama-server's OpenAI-compatible chat endpoint with the grounding OCR prompt
+ * at temp 0, and asserts the known strings come back in the transcription. This
+ * proves the actual model is loaded and transcribing pixels — not a mock — which
+ * is the bar the evidence harness holds every analyzer to.
+ *
+ * Fixture is generated in-process with sharp (SVG text → PNG, a repo dependency)
+ * so there is no binary asset to check in and the expected text is defined right
+ * next to the assertion. On mismatch the script prints the expected-vs-got diff
+ * and exits nonzero; capture latency and token counts are printed on success.
+ *
+ * The server base URL is taken from --url, else the instance recorded in
+ * serve.json (keyed per model, so --vlm finds the right one), else a validated
+ * ELIZA_GPU_VISION_PORT. --start is the one-shot convenience path: setup (if
+ * needed) → serve → smoke → stop.
+ *
+ * Usage:
+ *   node scripts/gpu-vision/smoke.mjs [--url http://127.0.0.1:PORT] [--vlm]
+ *   node scripts/gpu-vision/smoke.mjs --start [--vlm] [--with-vlm]
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import sharp from "sharp";
+import {
+  cacheDir,
+  MODEL_SETS,
+  OCR_PROMPT,
+  parseArgs,
+  parsePort,
+  serveStatePath,
+  waitForReady,
+} from "./lib.mjs";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const SCRIPTS = {
+  setup: path.join(__dirname, "setup.mjs"),
+  serve: path.join(__dirname, "serve.mjs"),
+};
+
+// Distinctive tokens: an unlikely alphanumeric code plus two plain words, so a
+// pass requires transcribing arbitrary characters, not echoing a common phrase.
+const FIXTURE_LINES = [
+  "ELIZA VISION LANE",
+  "OCR-CODE-4F2A9",
+  "unlimited ocr smoke",
+];
+
+async function renderFixture() {
+  const width = 900;
+  const lineHeight = 90;
+  const height = lineHeight * (FIXTURE_LINES.length + 1);
+  const texts = FIXTURE_LINES.map(
+    (line, i) =>
+      `<text x="50" y="${lineHeight * (i + 1)}" font-family="Helvetica, Arial, sans-serif" font-size="52" fill="#000000">${line}</text>`,
+  ).join("");
+  const svg = `<svg xmlns="http://www.w3.org/2000/svg" width="${width}" height="${height}"><rect width="100%" height="100%" fill="#ffffff"/>${texts}</svg>`;
+  return sharp(Buffer.from(svg)).png().toBuffer();
+}
+
+async function readServeState() {
+  try {
+    return JSON.parse(await fs.readFile(serveStatePath(), "utf8"));
+  } catch (err) {
+    // No serve.json simply means nothing was launched; any other read/parse
+    // failure is a real fault and must surface, not read as "no server".
+    if (err.code === "ENOENT") return {};
+    throw err;
+  }
+}
+
+/**
+ * Discovery order: --url wins; then the per-model serve.json entry (keyed, so
+ * --vlm never hits the OCR instance a fixed env port might point at); then a
+ * validated ELIZA_GPU_VISION_PORT for servers started outside serve.mjs.
+ */
+async function resolveBaseUrl(flags, setKey) {
+  if (flags.url) return String(flags.url).replace(/\/$/, "");
+  const state = await readServeState();
+  const entry = state[setKey];
+  if (entry) return `http://127.0.0.1:${entry.port}`;
+  const envPort = parsePort(
+    process.env.ELIZA_GPU_VISION_PORT,
+    "ELIZA_GPU_VISION_PORT",
+  );
+  if (envPort !== undefined) return `http://127.0.0.1:${envPort}`;
+  throw new Error(
+    "[gpu-vision] no running server found. Pass --url, set ELIZA_GPU_VISION_PORT, " +
+      "or start one: node scripts/gpu-vision/serve.mjs" +
+      (setKey === "vlm" ? " --vlm" : ""),
+  );
+}
+
+async function runOcr(baseUrl, pngBuffer) {
+  const dataUrl = `data:image/png;base64,${pngBuffer.toString("base64")}`;
+  const body = {
+    model: "gpu-vision",
+    temperature: 0,
+    messages: [
+      {
+        role: "user",
+        content: [
+          { type: "text", text: OCR_PROMPT },
+          { type: "image_url", image_url: { url: dataUrl } },
+        ],
+      },
+    ],
+  };
+  const started = Date.now();
+  // Generous but bounded: a cold first OCR pass takes seconds, while a socket
+  // that accepts and never responds must not hang the smoke run forever.
+  const res = await fetch(`${baseUrl}/v1/chat/completions`, {
+    method: "POST",
+    headers: { "content-type": "application/json" },
+    body: JSON.stringify(body),
+    signal: AbortSignal.timeout(120000),
+  });
+  const latencyMs = Date.now() - started;
+  if (!res.ok) {
+    const text = await res.text().catch(() => "");
+    throw new Error(
+      `[gpu-vision] chat completion HTTP ${res.status}: ${text.slice(0, 400)}`,
+    );
+  }
+  const json = await res.json();
+  const text = json?.choices?.[0]?.message?.content;
+  if (typeof text !== "string") {
+    throw new Error(
+      `[gpu-vision] unexpected response shape: ${JSON.stringify(json).slice(0, 400)}`,
+    );
+  }
+  return { text, latencyMs, usage: json.usage ?? null };
+}
+
+/** Case-insensitive, whitespace-collapsed substring check so minor formatting
+ * (extra spaces, casing) doesn't fail a genuinely-correct transcription. */
+function normalize(s) {
+  return s.toLowerCase().replace(/\s+/g, " ").trim();
+}
+
+function assertContainsAll(transcription, expectedLines) {
+  const hay = normalize(transcription);
+  const missing = expectedLines.filter(
+    (line) => !hay.includes(normalize(line)),
+  );
+  return { ok: missing.length === 0, missing };
+}
+
+async function smoke(baseUrl) {
+  await waitForReady(`${baseUrl}/health`, { timeoutMs: 15000 });
+  const png = await renderFixture();
+  process.stdout.write(
+    `[gpu-vision] fixture: ${png.length} byte PNG, expecting ${FIXTURE_LINES.length} lines\n`,
+  );
+
+  const { text, latencyMs, usage } = await runOcr(baseUrl, png);
+  process.stdout.write(
+    `\n[gpu-vision] transcription (${latencyMs} ms):\n${text}\n\n`,
+  );
+
+  const { ok, missing } = assertContainsAll(text, FIXTURE_LINES);
+  if (usage) {
+    process.stdout.write(
+      `[gpu-vision] tokens: prompt=${usage.prompt_tokens ?? "?"} completion=${usage.completion_tokens ?? "?"} total=${usage.total_tokens ?? "?"}\n`,
+    );
+  }
+  process.stdout.write(`[gpu-vision] latency: ${latencyMs} ms\n`);
+
+  if (!ok) {
+    process.stderr.write(
+      "\n[gpu-vision] SMOKE FAILED — expected strings not found in transcription:\n",
+    );
+    for (const line of missing)
+      process.stderr.write(`  MISSING: ${JSON.stringify(line)}\n`);
+    process.stderr.write(
+      `\n  expected all of: ${JSON.stringify(FIXTURE_LINES)}\n`,
+    );
+    process.stderr.write(`  got: ${JSON.stringify(text)}\n`);
+    process.exit(1);
+  }
+  process.stdout.write(
+    "\n[gpu-vision] SMOKE PASSED — all expected strings transcribed.\n",
+  );
+}
+
+function runScript(scriptPath, args) {
+  const result = spawnSync("node", [scriptPath, ...args], { stdio: "inherit" });
+  if (result.status !== 0) {
+    throw new Error(
+      `[gpu-vision] ${path.basename(scriptPath)} ${args.join(" ")} exited ${result.status}`,
+    );
+  }
+}
+
+async function main() {
+  const { flags } = parseArgs(process.argv.slice(2), {
+    booleans: ["vlm", "start", "with-vlm"],
+  });
+  const setKey = flags.vlm ? "vlm" : "ocr";
+
+  if (flags.start) {
+    // One-shot: fetch models if needed, serve, smoke, then always stop. serve
+    // runs inside the try so a spawn that got as far as recording an instance
+    // is always torn down; --stop no-ops harmlessly when nothing was recorded.
+    runScript(
+      SCRIPTS.setup,
+      flags["with-vlm"] || setKey === "vlm" ? ["--with-vlm"] : [],
+    );
+    const instanceArgs = setKey === "vlm" ? ["--vlm"] : [];
+    try {
+      runScript(SCRIPTS.serve, instanceArgs);
+      const baseUrl = await resolveBaseUrl(flags, setKey);
+      await smoke(baseUrl);
+    } finally {
+      runScript(SCRIPTS.serve, ["--stop", ...instanceArgs]);
+    }
+    return;
+  }
+
+  const baseUrl = await resolveBaseUrl(flags, setKey);
+  process.stdout.write(
+    `[gpu-vision] target: ${baseUrl} (${MODEL_SETS[setKey].label})\n`,
+  );
+  process.stdout.write(`[gpu-vision] cache: ${cacheDir()}\n`);
+  await smoke(baseUrl);
+}
+
+main().catch((err) => {
+  process.stderr.write(`${err.stack || err.message}\n`);
+  process.exit(1);
+});

scripts/hitl/hitl-manifest.mjs

@@ -0,0 +1,238 @@
+/**
+ * HITL manifest (#14381).
+ *
+ * Maps human-decision points to the EXISTING e2e recording suites
+ * (`scripts/e2e-recordings/suites.mjs`) that stage the frames a human must
+ * eyeball. This is the tag layer the runner uses to select the onboarding /
+ * first-chat / login subset and to know which decision points are only
+ * satisfiable on a real device (routed to the Seeker pass, not faked headless).
+ *
+ * Marks:
+ *   frame  — headless can stage + screenshot the state; human judges the frame
+ *   device — needs a real device capability; NOT covered headless
+ *   auto   — machine-decidable; listed for completeness, no human needed
+ *
+ * See docs/testing/hitl-inventory.md for the full rationale per row.
+ */
+
+/** @typedef {"frame"|"device"|"auto"} HitlMark */
+
+/**
+ * @typedef {Object} HitlDecisionPoint
+ * @property {string} id           stable id for the decision point
+ * @property {string} group        onboarding | first-chat | login | wallet | notifications | visual | lifeops-live
+ * @property {string} label        human-readable
+ * @property {HitlMark} mark
+ * @property {string} why          why a human/device is required
+ * @property {string[]} suites     UI_E2E_SUITES names that stage this (empty if device-only)
+ * @property {string[]} [scripts]  specific package test scripts that stage frames
+ * @property {number[]} [issues]   related GitHub issues
+ */
+
+/** @type {HitlDecisionPoint[]} */
+export const HITL_DECISION_POINTS = [
+  // ── Onboarding ──────────────────────────────────────────────────────────
+  {
+    id: "onboarding-welcome",
+    group: "onboarding",
+    label: "First-run welcome + name/style pick",
+    mark: "frame",
+    why: "first impression, copy, pacing — not machine-decidable",
+    suites: ["app", "android-emu", "ios-sim"],
+    scripts: ["test:ftu-home-e2e"],
+    issues: [14382, 14168],
+  },
+  {
+    id: "onboarding-provider",
+    group: "onboarding",
+    label: "Provider / model selection grid",
+    mark: "frame",
+    why: "catalog correctness + visual density",
+    suites: ["app"],
+    issues: [14382],
+  },
+  {
+    id: "onboarding-replay",
+    group: "onboarding",
+    label: "Re-run onboarding on a real agent (no wipe)",
+    mark: "frame",
+    why: "historically impossible without nuking memories — see the dev-gated replay entry",
+    suites: ["app"],
+    issues: [14382],
+  },
+  {
+    id: "onboarding-persist",
+    group: "onboarding",
+    label: "Onboarding persists across restart",
+    mark: "auto",
+    why: "machine-decided by first-run-persistence.restart.test.ts",
+    suites: [],
+    issues: [14382],
+  },
+
+  // ── First chat ──────────────────────────────────────────────────────────
+  {
+    id: "first-chat-send",
+    group: "first-chat",
+    label: "First message send → streamed reply",
+    mark: "frame",
+    why: "cadence/feel; real inference needs credits (stubbed frames here)",
+    suites: ["app"],
+    issues: [14424],
+  },
+  {
+    id: "first-chat-suggestions",
+    group: "first-chat",
+    label: "Suggestions / FTU home widgets",
+    mark: "frame",
+    why: "visual + relevance",
+    suites: ["app"],
+    scripts: ["test:suggestions-e2e", "test:ftu-home-e2e"],
+  },
+  {
+    id: "first-chat-scroll",
+    group: "first-chat",
+    label: "Chat scroll / momentum / sheet detents",
+    mark: "frame",
+    why: "gesture feel",
+    suites: ["app"],
+    scripts: [
+      "test:chat-scroll-web-e2e",
+      "test:chat-sheet-e2e",
+      "test:chatux-gesture-e2e",
+    ],
+    issues: [14380],
+  },
+
+  // ── Login / auth ────────────────────────────────────────────────────────
+  {
+    id: "login-cloud-mock",
+    group: "login",
+    label: "Cloud sign-in (mock OAuth round-trip)",
+    mark: "frame",
+    why: "mockable frames only; REAL sign-in is device-only per FLEET.md",
+    suites: ["cloud-e2e"],
+    issues: [13609],
+  },
+  {
+    id: "login-real-device",
+    group: "login",
+    label: "Real cloud sign-in golden path",
+    mark: "device",
+    why: "real OAuth/token/tenant — injected state is banned as evidence",
+    suites: [],
+    issues: [13611, 13610, 13609],
+  },
+  {
+    id: "login-stale-token-spam",
+    group: "login",
+    label: "Stale-token / no-credits resume spam guard",
+    mark: "auto",
+    why: "machine-decided by use-first-run-conductor.test.ts (#14387)",
+    suites: [],
+    issues: [14387],
+  },
+
+  // ── Wallet / notifications (mostly device) ──────────────────────────────
+  {
+    id: "wallet-empty-render",
+    group: "wallet",
+    label: "Wallet view render (no leaked 'Not found')",
+    mark: "frame",
+    why: "error-string leak visible only to the eye (#14426)",
+    suites: ["app"],
+    issues: [14426],
+  },
+  {
+    id: "notifications-priming",
+    group: "notifications",
+    label: "Permission priming prompt",
+    mark: "frame",
+    why: "copy + timing",
+    suites: ["app"],
+    scripts: ["test:permission-priming-e2e"],
+  },
+
+  // ── LifeOps live validation (#11632) ────────────────────────────────────
+  // The operator lane for the live, account/device-backed LifeOps pass.
+  // Deliberately OUTSIDE HITL_GOLDEN_GROUPS: it needs real credentials and
+  // devices, so it is selected explicitly via --groups=lifeops-live.
+  {
+    id: "lifeops-cred-intake",
+    group: "lifeops-live",
+    label: "Connector credential intake / readiness dashboard",
+    mark: "frame",
+    why: "a human confirms which connector creds are present in .env before the live lanes run",
+    suites: [],
+    scripts: ["scripts/lifeops/hitl-credential-dashboard.mjs"],
+    issues: [11632],
+  },
+  {
+    id: "lifeops-matrix-credentialed",
+    group: "lifeops-live",
+    label: "9-state OWNER/AGENT permission matrix (credentialed)",
+    mark: "auto",
+    why: "machine-decided by owner-agent-permission-matrix.integration.test.ts under LIFEOPS_PERMISSION_MATRIX=1",
+    suites: [],
+    issues: [11632],
+  },
+  {
+    id: "lifeops-live-connector-suites",
+    group: "lifeops-live",
+    label: "Live connector suites (creds present → live; absent → clean skip)",
+    mark: "auto",
+    why: "machine-decided; each suite describeIf-gates on its own credentials",
+    suites: [],
+    scripts: ["scripts/lifeops/run-11632-live-lanes.mjs"],
+    issues: [11632],
+  },
+  {
+    id: "lifeops-populated-views",
+    group: "lifeops-live",
+    label: "LifeOps split views populated with live account data",
+    mark: "frame",
+    why: "populated/empty/error render quality with real data is judged by eye",
+    suites: ["app"],
+    issues: [11632],
+  },
+  {
+    id: "lifeops-ios-native",
+    group: "lifeops-live",
+    label: "iOS/macOS native flows (HealthKit, Family Controls, SelfControl)",
+    mark: "device",
+    why: "real OS permission dialogs + native capabilities — not fakeable headless",
+    suites: [],
+    issues: [11632],
+  },
+  {
+    id: "lifeops-android-native",
+    group: "lifeops-live",
+    label:
+      "Android native flows (Health Connect, SMS default-role, Usage Access)",
+    mark: "device",
+    why: "real OS permission dialogs + SIM-backed capabilities — not fakeable headless",
+    suites: [],
+    issues: [11632],
+  },
+];
+
+/** The core golden-path groups this session's harness targets. */
+export const HITL_GOLDEN_GROUPS = ["onboarding", "first-chat", "login"];
+
+/** All e2e suite names referenced by frame-mark decision points (dedup). */
+export function hitlFrameSuites(groups = HITL_GOLDEN_GROUPS) {
+  const names = new Set();
+  for (const dp of HITL_DECISION_POINTS) {
+    if (dp.mark !== "frame") continue;
+    if (groups && !groups.includes(dp.group)) continue;
+    for (const s of dp.suites) names.add(s);
+  }
+  return [...names];
+}
+
+/** Decision points that are device-only (routed to the Seeker pass, not faked). */
+export function hitlDeviceOnly(groups = HITL_GOLDEN_GROUPS) {
+  return HITL_DECISION_POINTS.filter(
+    (dp) => dp.mark === "device" && (!groups || groups.includes(dp.group)),
+  );
+}

scripts/hitl/run-hitl.mjs

@@ -0,0 +1,201 @@
+#!/usr/bin/env node
+/**
+ * run-hitl.mjs — HITL golden-path harness skeleton (#14381).
+ *
+ * Walks the onboarding → first-chat → login decision points and produces a
+ * HITL review packet: it drives the relevant EXISTING e2e recording suites
+ * (through `scripts/e2e-recordings/run-all.mjs`, which already extracts frames,
+ * builds contact sheets, and a viewer), then emits a `hitl-report.json` +
+ * `hitl-report.md` that labels each staged frame with its decision point and a
+ * pass / fail / blocked slot — so a developer skims a contact sheet during
+ * development instead of manually driving the app or waiting for a device pass.
+ *
+ * It does NOT reinvent capture. Device-only decision points (real login,
+ * wallet signing, push) are emitted as `blocked (device)` and routed to the
+ * Seeker pass rather than faked headless.
+ *
+ * Usage:
+ *   node scripts/hitl/run-hitl.mjs                # plan + report from existing recordings
+ *   node scripts/hitl/run-hitl.mjs --record       # also (re)run the frame suites first
+ *   node scripts/hitl/run-hitl.mjs --groups=onboarding,login
+ *   node scripts/hitl/run-hitl.mjs --out=<dir>
+ *
+ * Honesty: the packages/ui e2e env is known-flaky. `--record` best-efforts the
+ * suites and reports which ran / skipped / failed; the report is still emitted
+ * from whatever frames exist so a partial run is still reviewable.
+ */
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import {
+  HITL_DECISION_POINTS,
+  HITL_GOLDEN_GROUPS,
+  hitlDeviceOnly,
+  hitlFrameSuites,
+} from "./hitl-manifest.mjs";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const RECORDINGS_DIR = path.join(REPO_ROOT, "e2e-recordings");
+
+// ── args ────────────────────────────────────────────────────────────────────
+const args = process.argv.slice(2);
+const flags = new Map();
+for (const a of args) {
+  const [k, v] = a.replace(/^--/, "").split("=");
+  flags.set(k, v ?? true);
+}
+const doRecord = flags.get("record") === true || flags.get("record") === "true";
+const groups = flags.has("groups")
+  ? String(flags.get("groups"))
+      .split(",")
+      .map((s) => s.trim())
+  : HITL_GOLDEN_GROUPS;
+const outDir = flags.has("out")
+  ? path.resolve(String(flags.get("out")))
+  : path.join(RECORDINGS_DIR, "hitl");
+
+function log(...m) {
+  console.log("[hitl]", ...m);
+}
+
+// ── plan ──────────────────────────────────────────────────────────────────
+const frameSuites = hitlFrameSuites(groups);
+const deviceOnly = hitlDeviceOnly(groups);
+const framePoints = HITL_DECISION_POINTS.filter(
+  (dp) => dp.mark === "frame" && groups.includes(dp.group),
+);
+const autoPoints = HITL_DECISION_POINTS.filter(
+  (dp) => dp.mark === "auto" && groups.includes(dp.group),
+);
+
+log(`groups: ${groups.join(", ")}`);
+log(
+  `frame decision points: ${framePoints.length} (suites: ${frameSuites.join(", ") || "none"})`,
+);
+log(`device-only (blocked, routed to Seeker): ${deviceOnly.length}`);
+log(`auto (machine-decided, no human): ${autoPoints.length}`);
+
+// ── optionally (re)run the frame suites through the existing pipeline ────────
+const recordResult = { attempted: false, ran: [], failed: [], skipped: [] };
+if (doRecord && frameSuites.length) {
+  recordResult.attempted = true;
+  const runAll = path.join(
+    REPO_ROOT,
+    "scripts",
+    "e2e-recordings",
+    "run-all.mjs",
+  );
+  log(
+    `recording frame suites via ${path.relative(REPO_ROOT, runAll)} --packages=${frameSuites.join(",")}`,
+  );
+  const res = spawnSync(
+    "node",
+    [runAll, `--packages=${frameSuites.join(",")}`],
+    { cwd: REPO_ROOT, stdio: "inherit", env: process.env },
+  );
+  // run-all uses exit code 77 for "skipped" per its contract.
+  if (res.status === 0) recordResult.ran = frameSuites;
+  else if (res.status === 77) recordResult.skipped = frameSuites;
+  else recordResult.failed = frameSuites;
+  log(`record exit=${res.status ?? "signal:" + res.signal}`);
+} else if (doRecord) {
+  log("no frame suites for the selected groups — nothing to record");
+}
+
+// ── collect whatever frames exist (partial runs still reviewable) ────────────
+function readManifest() {
+  const p = path.join(RECORDINGS_DIR, "manifest.json");
+  if (!fs.existsSync(p)) return null;
+  try {
+    return JSON.parse(fs.readFileSync(p, "utf8"));
+  } catch {
+    return null;
+  }
+}
+const manifest = readManifest();
+const haveContactSheets = fs.existsSync(
+  path.join(RECORDINGS_DIR, "contact-sheets"),
+);
+
+// ── build the HITL review packet ─────────────────────────────────────────────
+const report = {
+  generatedAt: new Date().toISOString(),
+  groups,
+  record: recordResult,
+  contactSheetsPresent: haveContactSheets,
+  manifestPackages: manifest ? Object.keys(manifest).length : 0,
+  decisionPoints: [],
+};
+
+for (const dp of HITL_DECISION_POINTS) {
+  if (!groups.includes(dp.group)) continue;
+  let state;
+  if (dp.mark === "auto") state = "auto (no human needed)";
+  else if (dp.mark === "device") state = "blocked (device) → Seeker pass";
+  else
+    state = haveContactSheets
+      ? "review (frame staged)"
+      : "review (no frames yet — run --record)";
+  report.decisionPoints.push({
+    id: dp.id,
+    group: dp.group,
+    label: dp.label,
+    mark: dp.mark,
+    why: dp.why,
+    suites: dp.suites,
+    scripts: dp.scripts ?? [],
+    issues: dp.issues ?? [],
+    result: null, // human fills: pass | fail | blocked
+    state,
+  });
+}
+
+fs.mkdirSync(outDir, { recursive: true });
+const jsonPath = path.join(outDir, "hitl-report.json");
+fs.writeFileSync(jsonPath, JSON.stringify(report, null, 2));
+
+// markdown review sheet (what a developer skims)
+const lines = [];
+lines.push("# HITL review sheet");
+lines.push("");
+lines.push(`Generated: ${report.generatedAt}`);
+lines.push(`Groups: ${groups.join(", ")}`);
+lines.push(
+  `Contact sheets: ${haveContactSheets ? "present (open e2e-recordings/viewer)" : "NOT present — run `node scripts/hitl/run-hitl.mjs --record`"}`,
+);
+lines.push("");
+lines.push("| # | Group | Decision point | Mark | Result (fill in) | State |");
+lines.push("|---|---|---|---|---|---|");
+report.decisionPoints.forEach((dp, i) => {
+  lines.push(
+    `| ${i + 1} | ${dp.group} | ${dp.label} | ${dp.mark} | ☐ pass ☐ fail ☐ blocked | ${dp.state} |`,
+  );
+});
+lines.push("");
+if (deviceOnly.length) {
+  lines.push("## Device-only (route to Seeker pass — not faked headless)");
+  for (const dp of deviceOnly) {
+    lines.push(
+      `- **${dp.label}** — ${dp.why} (issues: ${(dp.issues ?? []).map((n) => `#${n}`).join(", ") || "—"})`,
+    );
+  }
+  lines.push("");
+}
+lines.push("_See docs/testing/hitl-inventory.md for full rationale._");
+const mdPath = path.join(outDir, "hitl-report.md");
+fs.writeFileSync(mdPath, lines.join("\n") + "\n");
+
+log(`wrote ${path.relative(REPO_ROOT, jsonPath)}`);
+log(`wrote ${path.relative(REPO_ROOT, mdPath)}`);
+log(
+  `summary: ${report.decisionPoints.length} decision points ` +
+    `(${framePoints.length} frame / ${deviceOnly.length} device / ${autoPoints.length} auto)`,
+);
+
+// non-zero only if --record was asked and the suites hard-failed, so CI can gate
+// on capture health while a plan-only run always succeeds.
+if (recordResult.attempted && recordResult.failed.length) {
+  process.exitCode = 1;
+}

scripts/lifeops-bench/export-action-manifest.ts

@@ -0,0 +1,486 @@
+#!/usr/bin/env node
+/**
+ * Exports the LifeOps planner action manifest as JSON for the LifeOps bench.
+ *
+ * Loads the personal-assistant plus its connector plugins (bluebubbles,
+ * contacts, imessage, phone, todos), builds the planner tool definitions from
+ * their actions, and emits each as a function-tool entry annotated with the
+ * source plugin and planner metadata (tags, contexts, priority, surfaces,
+ * risk/cost). The bench consumes the manifest to drive and score action selection.
+ */
+import { spawnSync } from "node:child_process";
+import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { dirname, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import { buildPlannerToolsFromActions } from "../../packages/core/src/actions/to-tool.ts";
+import type { Action } from "../../packages/core/src/types/components.ts";
+import type { Plugin } from "../../packages/core/src/types/plugin.ts";
+import blueBubblesPlugin from "../../plugins/plugin-bluebubbles/src/index.ts";
+import { appContactsPlugin } from "../../plugins/plugin-contacts/src/plugin.ts";
+import imessagePlugin from "../../plugins/plugin-imessage/src/index.ts";
+import { personalAssistantPlugin } from "../../plugins/plugin-personal-assistant/src/plugin.ts";
+import { appPhonePlugin } from "../../plugins/plugin-phone/src/plugin.ts";
+import { todosPlugin } from "../../plugins/plugin-todos/src/index.ts";
+
+type JsonObject = Record<string, unknown>;
+
+type ManifestEntry = {
+  type: "function";
+  function: {
+    name: string;
+    description: string;
+    parameters: JsonObject;
+    strict: true;
+  };
+  _plugin: string;
+  _tags: string[];
+  _contexts: string[];
+  _priority: number;
+  _examples_count: number;
+  _domain: string | null;
+  _capabilities: string[];
+  _surfaces: string[];
+  _risk: string | null;
+  _cost: string | null;
+};
+
+type Manifest = {
+  schemaVersion: 1;
+  generator: string;
+  sourcePlugins: string[];
+  filters: {
+    domains: string[];
+    capabilities: string[];
+    surfaces: string[];
+    excludeRisks: string[];
+    benchUmbrellaAugment: boolean;
+  };
+  actions: ManifestEntry[];
+};
+
+type CliOptions = {
+  out: string;
+  summaryOut: string | null;
+  domains: string[];
+  capabilities: string[];
+  surfaces: string[];
+  excludeRisks: string[];
+  benchUmbrellaAugment: boolean;
+};
+
+const scriptPath = fileURLToPath(import.meta.url);
+const repoRoot = resolve(dirname(scriptPath), "../..");
+const defaultManifestPath = resolve(
+  repoRoot,
+  "packages/benchmarks/lifeops-bench/manifests/actions.manifest.json",
+);
+const defaultSummaryPath = resolve(
+  repoRoot,
+  "packages/benchmarks/lifeops-bench/manifests/actions.summary.md",
+);
+const benchmarkPackageRoot = resolve(
+  repoRoot,
+  "packages/benchmarks/lifeops-bench",
+);
+
+function usage(): string {
+  return [
+    "Usage: node --conditions=eliza-source --conditions=development --import tsx scripts/lifeops-bench/export-action-manifest.ts [options]",
+    "",
+    "Options:",
+    "  --out <path>             Output manifest JSON path.",
+    "  --summary-out <path>     Output summary Markdown path. Use 'none' to skip.",
+    "  --domain <name>          Keep actions with domain:<name>. Repeatable.",
+    "  --capability <name>      Keep actions with capability:<name>. Repeatable.",
+    "  --surface <name>         Keep actions with surface:<name>. Repeatable.",
+    "  --exclude-risk <name>    Drop actions with risk:<name>. Repeatable.",
+    "  --skip-bench-augment     Do not add LifeOpsBench-only umbrella verbs.",
+    "  --help                   Print this help.",
+  ].join("\n");
+}
+
+function readValue(args: string[], index: number, flag: string): string {
+  const value = args[index + 1];
+  if (!value || value.startsWith("--")) {
+    throw new Error(`${flag} requires a value`);
+  }
+  return value;
+}
+
+function parseArgs(argv: string[]): CliOptions {
+  const options: CliOptions = {
+    out: defaultManifestPath,
+    summaryOut: defaultSummaryPath,
+    domains: [],
+    capabilities: [],
+    surfaces: [],
+    excludeRisks: [],
+    benchUmbrellaAugment: true,
+  };
+
+  for (let index = 0; index < argv.length; index += 1) {
+    const arg = argv[index];
+    switch (arg) {
+      case "--help":
+      case "-h":
+        console.log(usage());
+        process.exit(0);
+        break;
+      case "--out":
+        options.out = resolve(process.cwd(), readValue(argv, index, arg));
+        index += 1;
+        break;
+      case "--summary-out": {
+        const value = readValue(argv, index, arg);
+        options.summaryOut =
+          value === "none" ? null : resolve(process.cwd(), value);
+        index += 1;
+        break;
+      }
+      case "--domain":
+        options.domains.push(
+          normalizeTaggedValue("domain", readValue(argv, index, arg)),
+        );
+        index += 1;
+        break;
+      case "--capability":
+        options.capabilities.push(
+          normalizeTaggedValue("capability", readValue(argv, index, arg)),
+        );
+        index += 1;
+        break;
+      case "--surface":
+        options.surfaces.push(
+          normalizeTaggedValue("surface", readValue(argv, index, arg)),
+        );
+        index += 1;
+        break;
+      case "--exclude-risk":
+        options.excludeRisks.push(
+          normalizeTaggedValue("risk", readValue(argv, index, arg)),
+        );
+        index += 1;
+        break;
+      case "--skip-bench-augment":
+        options.benchUmbrellaAugment = false;
+        break;
+      default:
+        throw new Error(`Unknown argument: ${arg}\n\n${usage()}`);
+    }
+  }
+
+  return options;
+}
+
+function normalizeTaggedValue(prefix: string, value: string): string {
+  const trimmed = value.trim();
+  if (!trimmed) {
+    throw new Error(`Empty ${prefix} filter`);
+  }
+  return trimmed.includes(":") ? trimmed : `${prefix}:${trimmed}`;
+}
+
+function partitionTags(
+  tags: string[],
+): Pick<
+  ManifestEntry,
+  "_domain" | "_capabilities" | "_surfaces" | "_risk" | "_cost"
+> {
+  return {
+    _domain: tags.find((tag) => tag.startsWith("domain:")) ?? null,
+    _capabilities: tags.filter((tag) => tag.startsWith("capability:")),
+    _surfaces: tags.filter((tag) => tag.startsWith("surface:")),
+    _risk: tags.find((tag) => tag.startsWith("risk:")) ?? null,
+    _cost: tags.find((tag) => tag.startsWith("cost:")) ?? null,
+  };
+}
+
+function examplesCount(action: Action): number {
+  return Array.isArray(action.examples) ? action.examples.length : 0;
+}
+
+function actionToEntry(action: Action, pluginName: string): ManifestEntry {
+  const tool = buildPlannerToolsFromActions([action])[0];
+  if (!tool) {
+    throw new Error(`Failed to render planner tool for ${action.name}`);
+  }
+  const tags = [...(action.tags ?? [])];
+  const taxonomy = partitionTags(tags);
+  return {
+    type: "function",
+    function: {
+      name: tool.name,
+      description: tool.description,
+      parameters: tool.parameters as JsonObject,
+      strict: true,
+    },
+    _plugin: pluginName,
+    _tags: tags,
+    _contexts: [...(action.contexts ?? ["general"])],
+    _priority: action.priority ?? 100,
+    _examples_count: examplesCount(action),
+    ...taxonomy,
+  };
+}
+
+function collectEntries(plugins: Plugin[]): ManifestEntry[] {
+  const entries: ManifestEntry[] = [];
+  const seen = new Set<string>();
+  for (const plugin of plugins) {
+    for (const action of plugin.actions ?? []) {
+      const entry = actionToEntry(action, plugin.name);
+      const name = entry.function.name;
+      if (seen.has(name)) {
+        throw new Error(`Duplicate action name in exported registry: ${name}`);
+      }
+      seen.add(name);
+      entries.push(entry);
+    }
+  }
+  entries.sort((left, right) =>
+    left.function.name.localeCompare(right.function.name),
+  );
+  return entries;
+}
+
+function matchesAnyFilter(value: string | null, filters: string[]): boolean {
+  return filters.length === 0 || (value !== null && filters.includes(value));
+}
+
+function includesAllFilters(values: string[], filters: string[]): boolean {
+  return (
+    filters.length === 0 || filters.every((filter) => values.includes(filter))
+  );
+}
+
+function filterEntries(
+  entries: ManifestEntry[],
+  options: CliOptions,
+): ManifestEntry[] {
+  return entries.filter((entry) => {
+    if (!matchesAnyFilter(entry._domain, options.domains)) {
+      return false;
+    }
+    if (!includesAllFilters(entry._capabilities, options.capabilities)) {
+      return false;
+    }
+    if (!includesAllFilters(entry._surfaces, options.surfaces)) {
+      return false;
+    }
+    if (entry._risk !== null && options.excludeRisks.includes(entry._risk)) {
+      return false;
+    }
+    return true;
+  });
+}
+
+function writeJson(path: string, value: unknown): void {
+  mkdirSync(dirname(path), { recursive: true });
+  writeFileSync(path, `${JSON.stringify(value, null, 2)}\n`, "utf8");
+}
+
+function augmentWithBenchUmbrellas(path: string): void {
+  const result = spawnSync(
+    "python3",
+    ["-m", "eliza_lifeops_bench.manifest_export", path],
+    {
+      cwd: benchmarkPackageRoot,
+      encoding: "utf8",
+      stdio: "pipe",
+    },
+  );
+  if (result.status !== 0) {
+    throw new Error(
+      [
+        "LifeOpsBench umbrella manifest augment failed.",
+        result.stdout.trim(),
+        result.stderr.trim(),
+      ]
+        .filter(Boolean)
+        .join("\n"),
+    );
+  }
+  if (result.stdout.trim()) {
+    console.log(result.stdout.trim());
+  }
+}
+
+function formatJsonWithBiome(path: string): void {
+  const result = spawnSync(
+    "bunx",
+    ["biome", "format", "--stdin-file-path", "actions.manifest.json"],
+    {
+      cwd: repoRoot,
+      encoding: "utf8",
+      input: readFileSync(path, "utf8"),
+      stdio: "pipe",
+    },
+  );
+  if (result.status !== 0) {
+    throw new Error(
+      [
+        `Biome failed to format generated manifest at ${path}.`,
+        result.stdout.trim(),
+        result.stderr.trim(),
+      ]
+        .filter(Boolean)
+        .join("\n"),
+    );
+  }
+  writeFileSync(path, result.stdout, "utf8");
+}
+
+function renderList(values: string[]): string {
+  return values.length === 0 ? "none" : values.join(", ");
+}
+
+function renderSummary(manifest: Manifest): string {
+  const actions = [...manifest.actions].sort((left, right) => {
+    const leftDomain = left._domain ?? "(untagged)";
+    const rightDomain = right._domain ?? "(untagged)";
+    return (
+      leftDomain.localeCompare(rightDomain) ||
+      left.function.name.localeCompare(right.function.name)
+    );
+  });
+
+  const byPlugin = countBy(actions, (entry) => entry._plugin);
+  const byDomain = countBy(actions, (entry) => entry._domain ?? "(untagged)");
+  const byRisk = countBy(actions, (entry) => entry._risk ?? "(no risk)");
+  const domains = [...byDomain.keys()].sort();
+
+  const lines: string[] = [
+    "# LifeOps Action Manifest - Summary",
+    "",
+    "Generated by `scripts/lifeops-bench/export-action-manifest.ts`.",
+    `Filter: domains=[${renderList(manifest.filters.domains)}] capabilities=[${renderList(
+      manifest.filters.capabilities,
+    )}] surfaces=[${renderList(manifest.filters.surfaces)}] excludeRisks=[${renderList(
+      manifest.filters.excludeRisks,
+    )}] benchUmbrellaAugment=${manifest.filters.benchUmbrellaAugment}`,
+    `Total actions: ${actions.length}`,
+    "",
+    "## Plugin breakdown",
+    "",
+    "| Plugin | Actions |",
+    "| --- | ---: |",
+    ...renderCountRows(byPlugin),
+    "",
+    "## Domain breakdown",
+    "",
+    "| Domain | Actions |",
+    "| --- | ---: |",
+    ...renderCountRows(byDomain),
+    "",
+    "## Risk breakdown",
+    "",
+    "| Risk | Actions |",
+    "| --- | ---: |",
+    ...renderCountRows(byRisk),
+    "",
+    "## Actions by domain",
+    "",
+  ];
+
+  for (const domain of domains) {
+    lines.push(`### ${domain}`, "");
+    lines.push(
+      "| Action | Plugin | Risk | Capabilities | Surfaces | Description |",
+    );
+    lines.push("| --- | --- | :---: | --- | --- | --- |");
+    for (const entry of actions.filter(
+      (action) => (action._domain ?? "(untagged)") === domain,
+    )) {
+      lines.push(
+        [
+          `\`${entry.function.name}\``,
+          entry._plugin,
+          entry._risk ?? "-",
+          entry._capabilities
+            .map((tag) => tag.replace("capability:", ""))
+            .join(", "),
+          entry._surfaces.map((tag) => tag.replace("surface:", "")).join(", "),
+          truncateForTable(entry.function.description),
+        ]
+          .join(" | ")
+          .replace(/^/, "| ")
+          .replace(/$/, " |"),
+      );
+    }
+    lines.push("");
+  }
+
+  while (lines.at(-1) === "") {
+    lines.pop();
+  }
+  return `${lines.join("\n")}\n`;
+}
+
+function countBy<T>(
+  items: T[],
+  keyFn: (item: T) => string,
+): Map<string, number> {
+  const counts = new Map<string, number>();
+  for (const item of items) {
+    const key = keyFn(item);
+    counts.set(key, (counts.get(key) ?? 0) + 1);
+  }
+  return counts;
+}
+
+function renderCountRows(counts: Map<string, number>): string[] {
+  return [...counts.entries()]
+    .sort(([left], [right]) => left.localeCompare(right))
+    .map(([key, count]) => `| ${key} | ${count} |`);
+}
+
+function truncateForTable(value: string): string {
+  const escaped = value.replace(/\s+/g, " ").replace(/\|/g, "\\|").trim();
+  return escaped.length <= 96 ? escaped : `${escaped.slice(0, 95)}...`;
+}
+
+function main(): void {
+  const options = parseArgs(process.argv.slice(2));
+  const sourcePlugins = [
+    appContactsPlugin,
+    personalAssistantPlugin,
+    appPhonePlugin,
+    blueBubblesPlugin,
+    imessagePlugin,
+    todosPlugin,
+  ];
+  const entries = filterEntries(collectEntries(sourcePlugins), options);
+  const manifest: Manifest = {
+    schemaVersion: 1,
+    generator: "scripts/lifeops-bench/export-action-manifest.ts",
+    sourcePlugins: sourcePlugins.map((plugin) => plugin.name),
+    filters: {
+      domains: options.domains,
+      capabilities: options.capabilities,
+      surfaces: options.surfaces,
+      excludeRisks: options.excludeRisks,
+      benchUmbrellaAugment: options.benchUmbrellaAugment,
+    },
+    actions: entries,
+  };
+
+  writeJson(options.out, manifest);
+  if (options.benchUmbrellaAugment) {
+    augmentWithBenchUmbrellas(options.out);
+  }
+  formatJsonWithBiome(options.out);
+
+  const finalManifest = JSON.parse(
+    readFileSync(options.out, "utf8"),
+  ) as Manifest;
+  if (options.summaryOut) {
+    mkdirSync(dirname(options.summaryOut), { recursive: true });
+    writeFileSync(options.summaryOut, renderSummary(finalManifest), "utf8");
+  }
+  console.log(
+    `wrote ${options.out} (${finalManifest.actions.length} actions)` +
+      (options.summaryOut ? ` and ${options.summaryOut}` : ""),
+  );
+}
+
+main();

scripts/lifeops/collect-11632-live-validation-status.mjs

@@ -0,0 +1,414 @@
+#!/usr/bin/env node
+/**
+ * Collects the live-validation readiness status for the LifeOps HITL work
+ * (issue #11632) by probing each connector group's required env vars (model
+ * provider, Google, and others) and writing a status.json under reports/.
+ * Reports which connector groups are configured so the live
+ * validation run knows what it can actually exercise.
+ *
+ * CONNECTOR_GROUPS is also imported by the lane driver (run-11632-live-lanes.mjs
+ * derives its model gate from the model group), so the CLI body only runs when
+ * this file is the entrypoint (import.meta.main). As an entrypoint it hydrates
+ * process.env from the layered load shared with the HITL dashboard and lane
+ * driver (env-layers.mjs: process.env > repo .env > ~/.eliza/.env), so all
+ * three surfaces report the same readiness. The HITL
+ * dashboard renders per-auth-path rows from connector-paths.mjs instead.
+ */
+import { spawnSync } from "node:child_process";
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+import { applyLayeredEnvToProcess } from "./env-layers.mjs";
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+const DEFAULT_OUT = join(
+  ROOT,
+  "reports/lifeops-live-validation/11632-status/status.json",
+);
+const LIFEOPS_REPORT_ROOT = "reports/lifeops-live-validation";
+
+export const CONNECTOR_GROUPS = [
+  {
+    id: "model",
+    label: "Live model provider",
+    requiredAny: [
+      "OPENAI_API_KEY",
+      "ANTHROPIC_API_KEY",
+      "CEREBRAS_API_KEY",
+      "ELIZA_LIVE_TEST_LOCAL_LLAMA_CPP_BASE_URL",
+    ],
+  },
+  {
+    id: "google",
+    label: "Google Calendar / Gmail",
+    requiredAll: [
+      "GOOGLE_CLIENT_ID",
+      "GOOGLE_CLIENT_SECRET",
+      "GOOGLE_REDIRECT_URI",
+    ],
+  },
+  {
+    id: "discord",
+    label: "Discord",
+    requiredAny: ["DISCORD_API_TOKEN", "DISCORD_BOT_TOKEN"],
+  },
+  {
+    id: "telegram",
+    label: "Telegram",
+    requiredAll: ["TELEGRAM_BOT_TOKEN"],
+    optional: ["TELEGRAM_TEST_CHAT_ID", "TELEGRAM_ALLOWED_CHATS"],
+  },
+  {
+    id: "slack",
+    label: "Slack",
+    requiredAll: ["SLACK_BOT_TOKEN", "SLACK_APP_TOKEN"],
+    optional: ["SLACK_USER_TOKEN", "SLACK_CHANNEL_IDS", "SLACK_SIGNING_SECRET"],
+  },
+  {
+    id: "signal",
+    label: "Signal",
+    requiredAll: ["SIGNAL_ACCOUNT_NUMBER"],
+    requiredAny: ["SIGNAL_HTTP_URL", "SIGNAL_CLI_PATH"],
+  },
+  {
+    id: "whatsapp",
+    label: "WhatsApp",
+    requiredAll: [
+      "ELIZA_WHATSAPP_ACCESS_TOKEN",
+      "ELIZA_WHATSAPP_PHONE_NUMBER_ID",
+    ],
+  },
+  {
+    id: "x",
+    label: "X",
+    // plugin-x live tests use env-mode OAuth 1.0a; bearer-only cannot satisfy
+    // users/me and is not enough to mark the lane ready.
+    requiredAll: [
+      "TWITTER_API_KEY",
+      "TWITTER_API_SECRET_KEY",
+      "TWITTER_ACCESS_TOKEN",
+      "TWITTER_ACCESS_TOKEN_SECRET",
+    ],
+  },
+  {
+    id: "twilio",
+    label: "Phone / SMS / Voice",
+    requiredAll: ["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN"],
+    optional: ["TWILIO_PHONE_NUMBER", "TWILIO_WEBHOOK_URL"],
+  },
+  {
+    id: "health",
+    label: "Health",
+    requiredAny: [
+      "ELIZA_HEALTHKIT_CLI_PATH",
+      "ELIZA_GOOGLE_FIT_ACCESS_TOKEN",
+      "FITBIT_ACCESS_TOKEN",
+      "OURA_ACCESS_TOKEN",
+      "STRAVA_ACCESS_TOKEN",
+      "WITHINGS_ACCESS_TOKEN",
+    ],
+  },
+  {
+    id: "finance",
+    label: "Finances",
+    requiredAny: [
+      "LIFEOPS_FINANCE_CSV_FIXTURE",
+      "PLAID_CLIENT_ID",
+      "PLAID_SECRET",
+      "PAYPAL_CLIENT_ID",
+      "PAYPAL_CLIENT_SECRET",
+    ],
+  },
+  {
+    id: "native_ios_macos",
+    label: "iOS / macOS native permissions",
+    requiredAny: [
+      "ELIZA_IMESSAGE_BACKEND",
+      "ELIZA_NATIVE_PERMISSIONS_DYLIB",
+      "ELIZA_HEALTHKIT_CLI_PATH",
+    ],
+  },
+  {
+    id: "native_android",
+    label: "Android native permissions",
+    requiredAny: ["ANDROID_SERIAL"],
+    optional: ["ANDROID_HOME"],
+  },
+];
+
+function parseArgs(argv) {
+  const args = { out: DEFAULT_OUT };
+  for (let i = 0; i < argv.length; i += 1) {
+    const arg = argv[i];
+    if (arg === "--out") {
+      args.out = resolve(argv[++i]);
+    } else if (arg === "--help") {
+      console.log(
+        "Usage: node scripts/lifeops/collect-11632-live-validation-status.mjs [--out <status.json>]",
+      );
+      process.exit(0);
+    } else {
+      throw new Error(`Unknown argument: ${arg}`);
+    }
+  }
+  return args;
+}
+
+function hasEnv(name) {
+  const value = process.env[name];
+  return typeof value === "string" && value.trim().length > 0;
+}
+
+function commandAvailable(command, args = ["--version"]) {
+  const result = spawnSync(command, args, {
+    cwd: ROOT,
+    encoding: "utf8",
+    timeout: 10_000,
+  });
+  return {
+    command,
+    available: result.status === 0,
+    status: result.status,
+    signal: result.signal,
+    summary: summarizeOutput(`${result.stdout ?? ""}\n${result.stderr ?? ""}`),
+  };
+}
+
+function summarizeOutput(value) {
+  return value
+    .split("\n")
+    .map((line) => line.trim())
+    .filter(Boolean)
+    .slice(0, 5)
+    .join("\n");
+}
+
+export function groupStatus(group) {
+  const requiredAll = group.requiredAll ?? [];
+  const requiredAny = group.requiredAny ?? [];
+  const optional = group.optional ?? [];
+  const present = [...requiredAll, ...requiredAny, ...optional]
+    .filter(hasEnv)
+    .sort();
+  const missingRequiredAll = requiredAll.filter((name) => !hasEnv(name));
+  const anySatisfied = requiredAny.length === 0 || requiredAny.some(hasEnv);
+  const ready = missingRequiredAll.length === 0 && anySatisfied;
+  return {
+    id: group.id,
+    label: group.label,
+    readyForOperatorRun: ready,
+    requiredAll,
+    requiredAny,
+    optional,
+    present,
+    missingRequiredAll,
+    missingRequiredAny:
+      requiredAny.length > 0 && !anySatisfied ? requiredAny : [],
+  };
+}
+
+function adbHasOnlineSerial(adbSummary, serial) {
+  if (!serial) return false;
+  return adbSummary.split("\n").some((line) => {
+    const [deviceSerial, state] = line.trim().split(/\s+/, 2);
+    return deviceSerial === serial && state === "device";
+  });
+}
+
+function applyDeviceReadiness(envGroups, devices) {
+  const androidGroup = envGroups.find((group) => group.id === "native_android");
+  if (!androidGroup?.readyForOperatorRun) return;
+
+  // biome-ignore lint/suspicious/noUndeclaredEnvVars: evidence collector probes the local operator device outside Turbo tasks.
+  const androidSerial = process.env.ANDROID_SERIAL?.trim();
+  const online = adbHasOnlineSerial(devices.adb.summary, androidSerial);
+  androidGroup.deviceReady = online;
+  if (online) return;
+
+  androidGroup.readyForOperatorRun = false;
+  androidGroup.missingRequiredAny = [
+    ...androidGroup.missingRequiredAny,
+    `online adb device${androidSerial ? ` ${androidSerial}` : ""}`,
+  ];
+}
+
+function fileStatus(path) {
+  const full = join(ROOT, path);
+  return { path, exists: existsSync(full) };
+}
+
+function parseEvidenceLog(path, patterns) {
+  const full = join(ROOT, path);
+  if (!existsSync(full)) return { path, exists: false, matched: [] };
+  const text = readFileSync(full, "utf8");
+  return {
+    path,
+    exists: true,
+    matched: patterns.filter((pattern) => pattern.test(text)).map(String),
+  };
+}
+
+function buildStatus() {
+  const envGroups = CONNECTOR_GROUPS.map(groupStatus);
+  const existingEvidence = [
+    fileStatus(`${LIFEOPS_REPORT_ROOT}/README.md`),
+    fileStatus(`${LIFEOPS_REPORT_ROOT}/2026-07-02-keyless-run/README.md`),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/2026-07-02-keyless-run/owner-agent-permission-matrix.txt`,
+      [/20\/20 pass/i, /20 passed/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/2026-07-02-keyless-run/connector-keyless-suites.txt`,
+      [/all green/i, /pass/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/owner-agent-permission-matrix.txt`,
+      [/20\/20 pass/i, /20 passed/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/android-build-after-resolved-appdir.txt`,
+      [/BUILD SUCCESSFUL/i, /android sideload artifact audit passed/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/android-app-actions-test.txt`,
+      [/12 pass/i, /0 fail/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/biome-edited-files.txt`,
+      [/Checked \d+ files/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/core-build-node.txt`,
+      [/Node-only build complete/i],
+    ),
+    parseEvidenceLog(`${LIFEOPS_REPORT_ROOT}/11632-status/core-typecheck.txt`, [
+      /tsgo --noEmit -p \.\/tsconfig\.json/i,
+    ]),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/agent-typecheck.txt`,
+      [/tsgo --noEmit -p tsconfig\.json/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/plugin-discord-typecheck.txt`,
+      [/tsgo --noEmit/i],
+    ),
+    parseEvidenceLog(
+      `${LIFEOPS_REPORT_ROOT}/11632-status/plugin-google-live.txt`,
+      [/pass/i, /skip/i],
+    ),
+    parseEvidenceLog(`${LIFEOPS_REPORT_ROOT}/11632-status/plugin-x-live.txt`, [
+      /pass/i,
+      /skip/i,
+    ]),
+  ];
+  const devices = {
+    adb: commandAvailable("adb", ["devices", "-l"]),
+    xcrun: commandAvailable("xcrun", [
+      "simctl",
+      "list",
+      "devices",
+      "available",
+    ]),
+    devicectl: commandAvailable("devicectl", ["list", "devices"]),
+  };
+  applyDeviceReadiness(envGroups, devices);
+  const operatorReadyGroups = envGroups.filter(
+    (group) => group.readyForOperatorRun,
+  );
+  const blockedGroups = envGroups.filter((group) => !group.readyForOperatorRun);
+  const liveRowsProven = existingEvidence.some(
+    (entry) =>
+      entry.exists &&
+      (entry.path?.includes("plugin-google-live") ||
+        entry.path?.includes("plugin-x-live")),
+  );
+
+  return {
+    issue: 11632,
+    generatedAt: new Date().toISOString(),
+    verdict: {
+      closeable: false,
+      reason:
+        "This collector is read-only status evidence. #11632 still requires live OAuth/account/device artifacts unless a future run attaches those rows.",
+      operatorReadyGroups: operatorReadyGroups.map((group) => group.id),
+      blockedGroups: blockedGroups.map((group) => group.id),
+      liveRowsProven,
+    },
+    envGroups,
+    devices,
+    existingEvidence,
+    nextCommands: [
+      "LIFEOPS_PERMISSION_MATRIX=1 bunx vitest run --config packages/test/vitest/integration.config.ts plugins/plugin-personal-assistant/test/owner-agent-permission-matrix.integration.test.ts",
+      "TEST_LANE=post-merge ELIZA_LIVE_TEST=1 bun run --cwd plugins/plugin-google test",
+      "TEST_LANE=post-merge ELIZA_LIVE_TEST=1 bun run --cwd plugins/plugin-x test",
+      "bun run dev && bun run --cwd packages/app audit:views",
+      "bun run --cwd packages/app capture:android-emu",
+      "bun run --cwd packages/app capture:ios-sim",
+    ],
+  };
+}
+
+function renderMarkdown(status) {
+  const cell = (value) =>
+    String(value || "n/a")
+      .replaceAll("|", "\\|")
+      .replace(/\r?\n/g, "<br>");
+  const rows = status.envGroups
+    .map((group) => {
+      const required =
+        group.requiredAll.length > 0
+          ? group.requiredAll.join(", ")
+          : `one of: ${group.requiredAny.join(", ")}`;
+      const present =
+        group.present.length > 0 ? group.present.join(", ") : "none";
+      const missing = [
+        ...group.missingRequiredAll,
+        ...group.missingRequiredAny,
+      ].join(", ");
+      return `| ${cell(group.label)} | ${group.readyForOperatorRun ? "ready" : "blocked"} | ${cell(required)} | ${cell(present)} | ${cell(missing || "none")} |`;
+    })
+    .join("\n");
+  return `# #11632 LifeOps Live-Validation Status
+
+Generated: ${status.generatedAt}
+
+Verdict: **not closeable**. This is a read-only status artifact; live OAuth,
+account, sandbox, and physical-device rows still need operator evidence.
+
+| Surface | Status | Required | Present env names | Missing |
+|---|---|---|---|---|
+${rows}
+
+## Device Tooling
+
+| Tool | Available | Summary |
+|---|---:|---|
+| adb | ${status.devices.adb.available ? "yes" : "no"} | ${cell(status.devices.adb.summary)} |
+| xcrun | ${status.devices.xcrun.available ? "yes" : "no"} | ${cell(status.devices.xcrun.summary)} |
+| devicectl | ${status.devices.devicectl.available ? "yes" : "no"} | ${cell(status.devices.devicectl.summary)} |
+
+## Existing Evidence
+
+${status.existingEvidence.map((entry) => `- ${entry.exists ? "present" : "missing"}: \`${entry.path}\``).join("\n")}
+`;
+}
+
+const IS_MAIN =
+  import.meta.main || process.argv[1] === fileURLToPath(import.meta.url);
+
+if (IS_MAIN) {
+  applyLayeredEnvToProcess();
+  const args = parseArgs(process.argv.slice(2));
+  const status = buildStatus();
+  mkdirSync(dirname(args.out), { recursive: true });
+  writeFileSync(args.out, `${JSON.stringify(status, null, 2)}\n`, "utf8");
+  writeFileSync(
+    join(dirname(args.out), "README.md"),
+    renderMarkdown(status),
+    "utf8",
+  );
+  console.log(`[11632-status] wrote ${args.out}`);
+  console.log(
+    `[11632-status] closeable=${status.verdict.closeable} blocked=${status.verdict.blockedGroups.join(",")}`,
+  );
+}

scripts/lifeops/connector-paths.test.mjs

@@ -0,0 +1,711 @@
+/**
+ * Unit tests for the connector auth-path registry: structural invariants of
+ * the shipped CONNECTOR_PATHS, owner/agent slot naming against the repo's two
+ * conventions, and the declarative availability evaluator driven through a
+ * fully injected machine context (no real fs/PATH/exec needed to simulate
+ * machine states like "signal-cli installed but unregistered").
+ */
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import { resolve } from "node:path";
+import test from "node:test";
+import {
+  appBase,
+  CONNECTOR_PATH_ENV_NAMES,
+  CONNECTOR_PATH_KINDS,
+  CONNECTOR_PATHS,
+  checkAvailability,
+  DEFAULT_APP_BASE,
+  evaluateConnectorPaths,
+  getFamilies,
+  getPathsForFamily,
+  resolveDeepLink,
+  validateConnectorPaths,
+} from "./connector-paths.mjs";
+import { isSecretEnvName, PROBEABLE_PATH_IDS } from "./credential-probes.mjs";
+
+/** Deterministic machine context; override per scenario. */
+function fakeCtx(overrides = {}) {
+  return {
+    env: {},
+    platform: "darwin",
+    home: "/Users/op",
+    existsSync: () => false,
+    commandInPath: () => false,
+    runCommand: () => ({ ok: false }),
+    ...overrides,
+  };
+}
+
+const byId = (id) => {
+  const path = CONNECTOR_PATHS.find((entry) => entry.id === id);
+  assert.ok(path, `registry is missing ${id}`);
+  return path;
+};
+
+function markdownCells(row) {
+  return row
+    .replaceAll("\\|", "__ESCAPED_PIPE__")
+    .split("|")
+    .slice(1, -1)
+    .map((cell) => cell.replaceAll("__ESCAPED_PIPE__", "|").trim());
+}
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+const IDENTITY_SLOT_CATALOG = resolve(
+  ROOT,
+  "docs/testing/hitl-identity-slots.md",
+);
+
+function slotModel(path) {
+  if (path.rolesVia === "env-slots") return "env slots";
+  if (path.rolesVia === "oauth-requested-role") return "OAuth requestedRole";
+  if (path.rolesVia === "separate-real-accounts")
+    return "separate real account";
+  return "single/slotless";
+}
+
+function markdownList(values) {
+  return values.length > 0 ? values.join("<br>") : "n/a";
+}
+
+function parseIdentitySlotCatalog() {
+  const rows = new Map();
+  const text = readFileSync(IDENTITY_SLOT_CATALOG, "utf8");
+  for (const line of text.split(/\r?\n/)) {
+    if (!line.startsWith("| `")) continue;
+    const cells = markdownCells(line);
+    const id = cells[0]?.replace(/^`|`$/g, "");
+    assert.equal(cells.length, 8, `${id} row must have 8 columns`);
+    rows.set(id, {
+      family: cells[1],
+      kind: cells[2],
+      slotModel: cells[3],
+      ownerVars: cells[4],
+      agentVars: cells[5],
+      gateVars: cells[6],
+      notes: cells[7],
+    });
+  }
+  return rows;
+}
+
+// --- registry shape ------------------------------------------------------------
+
+test("shipped registry passes every structural invariant", () => {
+  assert.deepEqual(validateConnectorPaths(CONNECTOR_PATHS), []);
+});
+
+test("wired per-path probes have registry metadata and documented rows", () => {
+  const byPath = new Map(CONNECTOR_PATHS.map((path) => [path.id, path]));
+  const probeable = new Set(PROBEABLE_PATH_IDS);
+
+  for (const pathId of probeable) {
+    const path = byPath.get(pathId);
+    assert.ok(path, `PATH_PROBES contains unknown path ${pathId}`);
+    assert.notEqual(path.probeId, null, `${pathId} probeId is missing`);
+  }
+  for (const path of CONNECTOR_PATHS) {
+    if (path.probeId !== null) {
+      assert.ok(probeable.has(path.id), `${path.id} claims unwired probeId`);
+    }
+  }
+
+  const doc = readFileSync(
+    new URL("../../docs/testing/hitl-probes.md", import.meta.url),
+    "utf8",
+  );
+  const rows = doc.split("\n").filter((line) => /^\| `[^`]+` \|/.test(line));
+  const docIds = rows.map((row) => markdownCells(row)[0].replaceAll("`", ""));
+  assert.deepEqual(
+    docIds.sort(),
+    CONNECTOR_PATHS.map((path) => path.id).sort(),
+  );
+
+  for (const row of rows) {
+    const cells = markdownCells(row);
+    assert.equal(cells.length, 10, `wrong cell count in ${row}`);
+    const pathId = cells[0].replaceAll("`", "");
+    const probeState = cells[4];
+    const expected = probeable.has(pathId) ? "wired" : "documented-skip";
+    assert.equal(probeState, expected, `${pathId} doc probe state`);
+    assert.ok(
+      cells.every((cell) => cell.length > 0),
+      `${pathId} has blanks`,
+    );
+  }
+});
+
+test("validateConnectorPaths flags duplicates, bad kinds, bad probe ids, missing endpoints", () => {
+  const problems = validateConnectorPaths([
+    {
+      id: "fam.a",
+      family: "fam",
+      kind: "bot",
+      label: "ok",
+      requiredAll: [],
+      requiredAny: [],
+      optional: [],
+      ownerVars: [],
+      agentVars: [],
+      rolesVia: null,
+      probeId: "telegram",
+      probeEndpoint: "GET x",
+      oneClick: null,
+      availability: { type: "always" },
+    },
+    {
+      id: "fam.a",
+      family: "other",
+      kind: "carrier-pigeon",
+      label: "",
+      requiredAll: ["lower_case"],
+      requiredAny: [],
+      optional: [],
+      ownerVars: ["OWNER_VAR"],
+      agentVars: [],
+      rolesVia: null,
+      probeId: "nope",
+      probeEndpoint: "",
+      oneClick: { type: "telepathy" },
+      availability: { type: "mystery", outputFilter: "telepathy" },
+    },
+  ]);
+  for (const needle of [
+    "duplicate id: fam.a",
+    "id must be <family>.<slug>",
+    "invalid kind carrier-pigeon",
+    "missing label",
+    "unknown probeId nope",
+    "free/cheap probe endpoint",
+    "invalid oneClick type telepathy",
+    "ownerVars/agentVars imply rolesVia env-slots",
+    "malformed env name lower_case",
+    "unknown availability type mystery",
+    "unknown command-output filter telepathy",
+  ]) {
+    assert.ok(
+      problems.some((problem) => problem.includes(needle)),
+      `expected a problem matching ${JSON.stringify(needle)} in ${JSON.stringify(problems)}`,
+    );
+  }
+});
+
+test("every required family is present with multiple-path families intact", () => {
+  const families = getFamilies();
+  for (const family of [
+    "model",
+    "elizacloud",
+    "github",
+    "google",
+    "telegram",
+    "discord",
+    "slack",
+    "signal",
+    "whatsapp",
+    "imessage",
+    "x",
+    "twilio",
+    "health",
+    "finance",
+    "crypto",
+  ]) {
+    assert.ok(families.includes(family), `missing family ${family}`);
+  }
+  assert.equal(getPathsForFamily("model").length, 3);
+  assert.ok(getPathsForFamily("github").length >= 3);
+  assert.ok(getPathsForFamily("signal").length >= 2);
+});
+
+test("kinds are constrained to the declared vocabulary", () => {
+  for (const path of CONNECTOR_PATHS) {
+    assert.ok(
+      CONNECTOR_PATH_KINDS.includes(path.kind),
+      `${path.id} kind ${path.kind}`,
+    );
+  }
+});
+
+test("identity-slot catalog is in lockstep with every connector path", () => {
+  const rows = parseIdentitySlotCatalog();
+  assert.deepEqual(
+    [...rows.keys()].sort(),
+    CONNECTOR_PATHS.map((path) => path.id).sort(),
+  );
+  for (const path of CONNECTOR_PATHS) {
+    const row = rows.get(path.id);
+    assert.ok(row, `missing identity-slot catalog row for ${path.id}`);
+    assert.equal(row.family, path.family, `${path.id} family drift`);
+    assert.equal(row.kind, path.kind, `${path.id} kind drift`);
+    assert.equal(row.slotModel, slotModel(path), `${path.id} slot model drift`);
+    assert.equal(
+      row.ownerVars,
+      markdownList(path.ownerVars),
+      `${path.id} owner vars drift`,
+    );
+    assert.equal(
+      row.agentVars,
+      markdownList(path.agentVars),
+      `${path.id} agent vars drift`,
+    );
+    assert.equal(
+      row.gateVars,
+      markdownList([...path.requiredAll, ...path.requiredAny]),
+      `${path.id} gate vars drift`,
+    );
+    assert.ok(row.notes.length > 0, `${path.id} notes cell must not be blank`);
+  }
+});
+
+// --- owner/agent conventions ------------------------------------------------------
+
+test("github.pat carries the concrete two-slot env names from plugin-github", () => {
+  const path = byId("github.pat");
+  assert.equal(path.rolesVia, "env-slots");
+  assert.deepEqual(path.ownerVars, [
+    "GITHUB_USER_PAT",
+    "ELIZA_E2E_GITHUB_USER_PAT",
+  ]);
+  assert.deepEqual(path.agentVars, [
+    "GITHUB_AGENT_PAT",
+    "ELIZA_E2E_GITHUB_AGENT_PAT",
+  ]);
+});
+
+test("google owner/agent rows use oauth requestedRole, never invented env slots", () => {
+  for (const id of ["google.oauth-owner", "google.oauth-agent"]) {
+    const path = byId(id);
+    assert.equal(path.rolesVia, "oauth-requested-role");
+    assert.deepEqual(path.ownerVars, []);
+    assert.deepEqual(path.agentVars, []);
+  }
+});
+
+test("telegram user-client gates on the owner session key with a legacy alias", () => {
+  const path = byId("telegram.user-client");
+  assert.deepEqual(path.requiredAll, ["TELEGRAM_API_ID", "TELEGRAM_API_HASH"]);
+  assert.deepEqual(path.requiredAny, [
+    "TELEGRAM_OWNER_SESSION",
+    "TELEGRAM_USER_SESSION",
+  ]);
+  assert.match(path.notes, /TELEGRAM_OWNER_SESSION/);
+
+  const missing = checkAvailability(path.availability, fakeCtx());
+  assert.equal(missing.available, false);
+  assert.match(missing.reason, /TELEGRAM_API_ID/);
+
+  const sessionWithoutApiCredentials = checkAvailability(
+    path.availability,
+    fakeCtx({ env: { TELEGRAM_OWNER_SESSION: "session" } }),
+  );
+  assert.equal(sessionWithoutApiCredentials.available, false);
+  assert.match(sessionWithoutApiCredentials.reason, /TELEGRAM_API_ID/);
+
+  const ownerSession = checkAvailability(
+    path.availability,
+    fakeCtx({
+      env: {
+        TELEGRAM_API_ID: "12345",
+        TELEGRAM_API_HASH: "hash",
+        TELEGRAM_OWNER_SESSION: "session",
+      },
+    }),
+  );
+  assert.equal(ownerSession.available, true);
+
+  const legacySession = checkAvailability(
+    path.availability,
+    fakeCtx({
+      env: {
+        TELEGRAM_API_ID: "12345",
+        TELEGRAM_API_HASH: "hash",
+        TELEGRAM_USER_SESSION: "session",
+      },
+    }),
+  );
+  assert.equal(legacySession.available, true);
+
+  const savedSession = checkAvailability(
+    path.availability,
+    fakeCtx({
+      env: { TELEGRAM_API_ID: "12345", TELEGRAM_API_HASH: "hash" },
+      existsSync: (filePath) => filePath.endsWith("telegram-user.session"),
+    }),
+  );
+  assert.equal(savedSession.available, true);
+});
+
+test("x agent slot is a separate real account, permanently skipped with the matrix reason", () => {
+  const path = byId("x.agent-account");
+  assert.equal(path.rolesVia, "separate-real-accounts");
+  const { available, reason } = checkAvailability(path.availability, fakeCtx());
+  assert.equal(available, false);
+  assert.match(reason, /separate real account/);
+});
+
+test("env-name union covers the dashboard allowlist extensions", () => {
+  for (const name of [
+    "GITHUB_USER_PAT",
+    "GITHUB_AGENT_PAT",
+    "ELIZA_CLOUD_API_KEY",
+    "BLUEBUBBLES_SERVER_URL",
+    "EVM_PRIVATE_KEY",
+    "TWITTER_ACCESS_TOKEN_SECRET",
+  ]) {
+    assert.ok(CONNECTOR_PATH_ENV_NAMES.has(name), `missing ${name}`);
+  }
+});
+
+// --- availability evaluator ---------------------------------------------------------
+
+test("leaf spec types evaluate against the injected ctx", () => {
+  const ctx = fakeCtx({
+    env: { SET_VAR: "x", BLANK: "  " },
+    existsSync: (path) => path === "/Users/op/thing",
+    commandInPath: (command) => command === "have-me",
+    runCommand: (command) => ({ ok: command === "gh" }),
+    platform: "linux",
+  });
+  assert.equal(checkAvailability({ type: "always" }, ctx).available, true);
+  assert.deepEqual(checkAvailability({ type: "never", reason: "no" }, ctx), {
+    available: false,
+    reason: "no",
+  });
+  assert.equal(
+    checkAvailability({ type: "env-present", names: ["SET_VAR"] }, ctx)
+      .available,
+    true,
+  );
+  assert.equal(
+    checkAvailability({ type: "env-present", names: ["BLANK"] }, ctx).available,
+    false,
+  );
+  assert.equal(
+    checkAvailability({ type: "env-all", names: ["SET_VAR", "BLANK"] }, ctx)
+      .available,
+    false,
+  );
+  assert.match(
+    checkAvailability({ type: "env-all", names: ["SET_VAR", "BLANK"] }, ctx)
+      .reason,
+    /BLANK/,
+  );
+  assert.equal(
+    checkAvailability({ type: "file-exists", path: "~/thing" }, ctx).available,
+    true,
+  );
+  assert.equal(
+    checkAvailability({ type: "dir-exists", path: "/elsewhere" }, ctx)
+      .available,
+    false,
+  );
+  assert.equal(
+    checkAvailability({ type: "command-in-path", command: "have-me" }, ctx)
+      .available,
+    true,
+  );
+  assert.equal(
+    checkAvailability(
+      { type: "command-ok", command: "gh", args: ["auth", "token"] },
+      ctx,
+    ).available,
+    true,
+  );
+  assert.equal(
+    checkAvailability({ type: "platform", platform: "darwin" }, ctx).available,
+    false,
+  );
+  assert.throws(
+    () => checkAvailability({ type: "mystery" }, ctx),
+    /Unknown availability/,
+  );
+});
+
+test("any-of aggregates all branch reasons; all-of reports the first failure", () => {
+  const ctx = fakeCtx();
+  const anyOf = checkAvailability(
+    {
+      type: "any-of",
+      specs: [
+        { type: "never", reason: "first missing" },
+        { type: "never", reason: "second missing" },
+      ],
+    },
+    ctx,
+  );
+  assert.equal(anyOf.available, false);
+  assert.equal(anyOf.reason, "first missing; second missing");
+  const allOf = checkAvailability(
+    {
+      type: "all-of",
+      specs: [
+        { type: "always" },
+        { type: "never", reason: "blocker" },
+        { type: "never", reason: "unreached" },
+      ],
+    },
+    ctx,
+  );
+  assert.deepEqual(allOf, { available: false, reason: "blocker" });
+});
+
+test("signal.cli scenario: installed-but-unrunnable CLI skips distinctly", () => {
+  const spec = byId("signal.cli").availability;
+  // This machine's GROUND state: signal-cli in PATH, no data dir, no REST URL.
+  const broken = checkAvailability(
+    spec,
+    fakeCtx({ commandInPath: (command) => command === "signal-cli" }),
+  );
+  assert.equal(broken.available, false);
+  assert.match(broken.reason, /installed but not runnable/);
+  // REST bridge configured -> available regardless of the local CLI.
+  const viaRest = checkAvailability(
+    spec,
+    fakeCtx({ env: { SIGNAL_HTTP_URL: "http://x" } }),
+  );
+  assert.equal(viaRest.available, true);
+  // CLI in PATH plus a successful read-only account listing -> available.
+  const registered = checkAvailability(
+    spec,
+    fakeCtx({
+      commandInPath: (command) => command === "signal-cli",
+      runCommand: (_command, args) => ({
+        ok: args[0] === "--version" || args[0] === "listAccounts",
+        stdout:
+          args[0] === "listAccounts"
+            ? "Number: +15551234567\n"
+            : "signal-cli 0.13\n",
+      }),
+    }),
+  );
+  assert.equal(registered.available, true);
+});
+
+test("Signal Desktop and signal-cli unavailability shapes stay distinct", () => {
+  const desktop = byId("signal.desktop-bridge").availability;
+  const absent = checkAvailability(desktop, fakeCtx());
+  assert.equal(absent.available, false);
+  assert.equal(absent.reason, "Signal Desktop not installed");
+  const noProfile = checkAvailability(
+    desktop,
+    fakeCtx({ existsSync: (path) => path === "/Applications/Signal.app" }),
+  );
+  assert.equal(noProfile.available, false);
+  assert.equal(noProfile.reason, "no Signal Desktop profile directory");
+
+  const cli = byId("signal.cli").availability;
+  const noBinary = checkAvailability(cli, fakeCtx());
+  assert.match(noBinary.reason, /signal-cli not in PATH/);
+  const noAccount = checkAvailability(
+    cli,
+    fakeCtx({
+      commandInPath: () => true,
+      runCommand: (_command, args) => ({
+        ok: args[0] === "--version" || args[0] === "listAccounts",
+        stdout: args[0] === "listAccounts" ? "" : "signal-cli 0.13\n",
+      }),
+    }),
+  );
+  assert.match(noAccount.reason, /no linked Signal account/);
+});
+
+test("command-output-nonempty requires both success and actual stdout", () => {
+  const spec = {
+    type: "command-output-nonempty",
+    command: "signal-cli",
+    args: ["listAccounts"],
+    reason: "no linked account",
+  };
+  assert.deepEqual(
+    checkAvailability(
+      spec,
+      fakeCtx({ runCommand: () => ({ ok: true, stdout: "" }) }),
+    ),
+    { available: false, reason: "no linked account" },
+  );
+  assert.deepEqual(
+    checkAvailability(
+      spec,
+      fakeCtx({
+        runCommand: () => ({ ok: true, stdout: "Number: +15551234567\n" }),
+      }),
+    ),
+    { available: true, reason: null },
+  );
+  // A command that fails to run reports the run failure, never the spec's
+  // empty-output reason — even when the injected ctx omits status/stderr.
+  assert.deepEqual(
+    checkAvailability(spec, fakeCtx({ runCommand: () => ({ ok: false }) })),
+    {
+      available: false,
+      reason: "signal-cli listAccounts failed (status=spawn-error)",
+    },
+  );
+  assert.deepEqual(
+    checkAvailability(
+      spec,
+      fakeCtx({
+        runCommand: () => ({
+          ok: false,
+          status: 3,
+          stdout: "",
+          stderr: "Config file is in use by another instance\nsecond line",
+        }),
+      }),
+    ),
+    {
+      available: false,
+      reason:
+        "signal-cli listAccounts failed (status=3): Config file is in use by another instance",
+    },
+  );
+  // An unknown filter name is a malformed spec — a bug, not a skip.
+  assert.throws(
+    () =>
+      checkAvailability(
+        { ...spec, outputFilter: "telepathy" },
+        fakeCtx({ runCommand: () => ({ ok: true, stdout: "x\n" }) }),
+      ),
+    /Unknown command-output filter: "telepathy"/,
+  );
+});
+
+test("signal.cli scenario: warning-only listAccounts stdout reads as no linked account", () => {
+  const spec = byId("signal.cli").availability;
+  const signalCtx = (listAccountsStdout) =>
+    fakeCtx({
+      commandInPath: (command) => command === "signal-cli",
+      runCommand: (_command, args) => ({
+        ok: true,
+        stdout:
+          args[0] === "listAccounts" ? listAccountsStdout : "signal-cli 0.13\n",
+      }),
+    });
+  // signal-cli builds that emit warnings to stdout with zero linked accounts
+  // must skip exactly like an empty listing — probeSignal already rejects
+  // this shape, and the coarse availability leaf shares its parser.
+  const warningOnly = checkAvailability(
+    spec,
+    signalCtx("WARN no account configured\n"),
+  );
+  assert.equal(warningOnly.available, false);
+  assert.match(warningOnly.reason, /no linked Signal account/);
+  // Warnings interleaved with a real account line still count as linked.
+  const mixed = checkAvailability(
+    spec,
+    signalCtx("WARN storage version is newer\nNumber: +15551234567\n"),
+  );
+  assert.deepEqual(mixed, { available: true, reason: null });
+  // A u:username account handle counts the same as an E.164 number.
+  const username = checkAvailability(spec, signalCtx("u:agent.01\n"));
+  assert.deepEqual(username, { available: true, reason: null });
+});
+
+test("signal.cli scenario: listAccounts crash after --version reports the run failure", () => {
+  const spec = byId("signal.cli").availability;
+  const crashed = checkAvailability(
+    spec,
+    fakeCtx({
+      commandInPath: (command) => command === "signal-cli",
+      runCommand: (_command, args) =>
+        args[0] === "listAccounts"
+          ? {
+              ok: false,
+              status: 1,
+              stdout: "",
+              stderr: "Config file is in use by another instance",
+            }
+          : { ok: true, stdout: "signal-cli 0.13\n" },
+    }),
+  );
+  assert.equal(crashed.available, false);
+  // Locked account storage is a run failure, not evidence of a missing
+  // account — the dashboard reason must carry the exit code and stderr.
+  assert.match(crashed.reason, /signal-cli listAccounts failed \(status=1\)/);
+  assert.match(crashed.reason, /Config file is in use/);
+  assert.doesNotMatch(crashed.reason, /no linked Signal account/);
+});
+
+test("oauth-app-dependent rows skip with an explanatory reason until an app is configured", () => {
+  const discord = checkAvailability(
+    byId("discord.user-oauth").availability,
+    fakeCtx(),
+  );
+  assert.equal(discord.available, false);
+  assert.match(discord.reason, /no Discord user-OAuth app configured/);
+  const github = checkAvailability(
+    byId("github.user-oauth").availability,
+    fakeCtx(),
+  );
+  assert.equal(github.available, false);
+  assert.match(github.reason, /no OAuth app configured/);
+  const configured = checkAvailability(
+    byId("github.user-oauth").availability,
+    fakeCtx({
+      env: { GITHUB_OAUTH_CLIENT_ID: "id", GITHUB_OAUTH_CLIENT_SECRET: "s" },
+    }),
+  );
+  assert.equal(configured.available, true);
+});
+
+test("github.gh-cli requires gh in PATH and an authenticated keyring", () => {
+  const spec = byId("github.gh-cli").availability;
+  assert.match(checkAvailability(spec, fakeCtx()).reason, /not in PATH/);
+  const unauthed = checkAvailability(
+    spec,
+    fakeCtx({ commandInPath: () => true, runCommand: () => ({ ok: false }) }),
+  );
+  assert.match(unauthed.reason, /not authenticated/);
+  const authed = checkAvailability(
+    spec,
+    fakeCtx({ commandInPath: () => true, runCommand: () => ({ ok: true }) }),
+  );
+  assert.equal(authed.available, true);
+});
+
+test("github device login is a designed owner-setup state until a client id exists", () => {
+  const path = byId("github.device-oauth");
+  assert.equal(path.oneClick.type, "github-device");
+  const missing = checkAvailability(path.availability, fakeCtx());
+  assert.equal(missing.available, false);
+  assert.match(missing.reason, /needs owner setup/);
+  const configured = checkAvailability(
+    path.availability,
+    fakeCtx({ env: { GITHUB_OAUTH_CLIENT_ID: "device-client" } }),
+  );
+  assert.equal(configured.available, true);
+});
+
+// --- evaluation output safety ---------------------------------------------------------
+
+test("evaluateConnectorPaths emits env names and reasons, never env values", () => {
+  const secret = "sk-super-secret-value-1234";
+  const rows = evaluateConnectorPaths(
+    fakeCtx({ env: { OPENAI_API_KEY: secret, SIGNAL_HTTP_URL: secret } }),
+  );
+  assert.equal(rows.length, CONNECTOR_PATHS.length);
+  const serialized = JSON.stringify(rows);
+  assert.equal(serialized.includes(secret), false);
+  for (const row of rows) {
+    assert.equal(typeof row.available, "boolean");
+    assert.ok(row.available || typeof row.reason === "string");
+  }
+});
+
+test("Telegram session env names are classified as secrets", () => {
+  assert.equal(isSecretEnvName("TELEGRAM_OWNER_SESSION"), true);
+  assert.equal(isSecretEnvName("TELEGRAM_USER_SESSION"), true);
+});
+
+// --- deep links -------------------------------------------------------------------------
+
+test("deep-link rows resolve against ELIZA_APP_BASE_URL with the v1 default", () => {
+  assert.equal(appBase({}), DEFAULT_APP_BASE);
+  assert.equal(appBase({ ELIZA_APP_BASE_URL: "http://h:9" }), "http://h:9");
+  const owner = byId("google.oauth-owner");
+  assert.equal(
+    resolveDeepLink(owner, {}),
+    `${DEFAULT_APP_BASE}/settings?section=connectors`,
+  );
+  assert.equal(resolveDeepLink(byId("model.openai-key"), {}), null);
+});

scripts/lifeops/credential-probes.test.mjs

@@ -0,0 +1,92 @@
+/** Signal credential probes exercise linked-account checks without network. */
+import assert from "node:assert/strict";
+import test from "node:test";
+import { probeSignal } from "./credential-probes.mjs";
+
+function lookup(values) {
+  return (name) => values[name];
+}
+
+test("Signal REST probe requires at least one linked account", async () => {
+  const urls = [];
+  const result = await probeSignal(
+    lookup({ SIGNAL_HTTP_URL: "http://signal" }),
+    {
+      fetchJsonFn: async (url) => {
+        urls.push(url);
+        return url.endsWith("/v1/about")
+          ? { httpOk: true, status: 200, body: { version: "0.13" } }
+          : { httpOk: true, status: 200, body: [] };
+      },
+    },
+  );
+  assert.deepEqual(urls, [
+    "http://signal/v1/about",
+    "http://signal/v1/accounts",
+  ]);
+  assert.equal(result.ok, false);
+  assert.equal(result.detail, "no linked Signal account");
+});
+
+test("Signal REST probe verifies the selected account without exposing it", async () => {
+  const account = "+15551234567";
+  const result = await probeSignal(
+    lookup({
+      SIGNAL_HTTP_URL: "http://signal/",
+      SIGNAL_ACCOUNT_NUMBER: account,
+    }),
+    {
+      fetchJsonFn: async (url) => ({
+        httpOk: true,
+        status: 200,
+        body: url.endsWith("/v1/accounts") ? [account] : { version: "0.13" },
+      }),
+    },
+  );
+  assert.equal(result.ok, true);
+  assert.match(result.detail, /1 linked account/);
+  assert.match(result.detail, /…4567 present/);
+  assert.equal(result.detail.includes(account), false);
+});
+
+test("Signal CLI probe runs version and listAccounts before passing", async () => {
+  const calls = [];
+  const result = await probeSignal(
+    lookup({
+      SIGNAL_CLI_PATH: "/opt/signal-cli",
+      SIGNAL_ACCOUNT_NUMBER: "+15551234567",
+    }),
+    {
+      spawnSyncFn: (command, args) => {
+        calls.push([command, ...args]);
+        return args[0] === "--version"
+          ? { status: 0, stdout: "signal-cli 0.13.20\n" }
+          : { status: 0, stdout: "Number: +15551234567\n" };
+      },
+    },
+  );
+  assert.deepEqual(calls, [
+    ["/opt/signal-cli", "--version"],
+    ["/opt/signal-cli", "listAccounts"],
+  ]);
+  assert.equal(result.ok, true);
+  assert.match(result.detail, /…4567 present/);
+  assert.equal(result.detail.includes("+15551234567"), false);
+});
+
+test("Signal CLI probe rejects warning-only account output", async () => {
+  const result = await probeSignal(
+    lookup({
+      SIGNAL_CLI_PATH: "signal-cli",
+      SIGNAL_ACCOUNT_NUMBER: "+15551234567",
+    }),
+    {
+      spawnSyncFn: (_command, args) =>
+        args[0] === "--version"
+          ? { status: 0, stdout: "signal-cli 0.13.20\n" }
+          : { status: 0, stdout: "WARN no account configured\n" },
+    },
+  );
+  assert.equal(result.ok, false);
+  assert.equal(result.detail, "no linked Signal account");
+});

scripts/lifeops/env-layers.mjs

@@ -0,0 +1,271 @@
+#!/usr/bin/env node
+/**
+ * Layered .env resolution for the LifeOps HITL credential tooling (#11632).
+ * Credentials can live in three places on an operator machine, and this module
+ * is the single arbiter of which one wins: process.env > the current repo's
+ * .env > ~/.eliza/.env.
+ * The HITL dashboard and lane drivers consume loadLayeredEnv()/listPresent()
+ * so a probe sees the same value a paste-and-save produced, no matter which
+ * worktree the operator happens to be in.
+ *
+ * Saves default to ~/.eliza/.env — the layer that survives worktree churn —
+ * with repo .env as the per-save alternative; writes are atomic (tmp file
+ * mode 600 + rename) and preserve unrelated lines and comments. The parse,
+ * merge, and upsert primitives are exported separately so they stay
+ * unit-testable without touching the real filesystem or git. Values returned
+ * by loadLayeredEnv are real secrets: callers must never render them — the
+ * display-safe surface is listPresent(), which only reports presence and the
+ * winning source layer.
+ */
+import {
+  existsSync,
+  mkdirSync,
+  readFileSync,
+  renameSync,
+  writeFileSync,
+} from "node:fs";
+import { homedir } from "node:os";
+import { dirname, join, resolve } from "node:path";
+import { fileURLToPath } from "node:url";
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+
+/** Home-layer file: shared across every checkout and worktree of this repo. */
+export const HOME_ENV_PATH = join(homedir(), ".eliza", ".env");
+
+/** Precedence order, highest first; the values of the `sources` map. */
+export const ENV_LAYER_SOURCES = ["process", "repo", "home"];
+
+const ENV_KEY_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/;
+
+// --- pure primitives ---------------------------------------------------------
+
+/**
+ * Parse dotenv text: KEY=value with optional `export ` prefix, surrounding
+ * single/double quotes stripped, comments and malformed lines skipped.
+ * Identical semantics to the v1 dashboard parser so a file written by either
+ * tool reads back the same.
+ */
+export function parseDotenv(text) {
+  const out = {};
+  for (const line of text.split(/\r?\n/)) {
+    const trimmed = line.trim();
+    if (!trimmed || trimmed.startsWith("#")) continue;
+    const match = /^(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=\s*(.*)$/.exec(
+      trimmed,
+    );
+    if (!match) continue;
+    let value = match[2].trim();
+    if (
+      (value.startsWith('"') && value.endsWith('"') && value.length >= 2) ||
+      (value.startsWith("'") && value.endsWith("'") && value.length >= 2)
+    ) {
+      value = value.slice(1, -1);
+    }
+    out[match[1]] = value;
+  }
+  return out;
+}
+
+/**
+ * Merge layers ordered highest-precedence first; the first layer that defines
+ * a key wins. "Defined" means a string value — including the empty string, so
+ * an exported-but-empty process.env variable shadows a file value exactly like
+ * dotenv's override:false behavior.
+ */
+export function mergeEnvLayers(layers) {
+  const values = {};
+  const sources = {};
+  for (const layer of layers) {
+    for (const [key, value] of Object.entries(layer.values)) {
+      if (typeof value !== "string") continue;
+      if (Object.hasOwn(sources, key)) continue;
+      values[key] = value;
+      sources[key] = layer.source;
+    }
+  }
+  return { values, sources };
+}
+
+/**
+ * Replace KEY=value lines in dotenv text, preserving unrelated lines and
+ * comments, appending keys that were not present. Always ends with a single
+ * trailing newline.
+ */
+export function upsertEnvContent(existingText, entries) {
+  const lines = existingText.length > 0 ? existingText.split("\n") : [];
+  const remaining = new Map(Object.entries(entries));
+  const nextLines = lines.map((line) => {
+    const match = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=/.exec(line);
+    if (match && remaining.has(match[1])) {
+      const value = remaining.get(match[1]);
+      remaining.delete(match[1]);
+      return `${match[1]}=${value}`;
+    }
+    return line;
+  });
+  while (
+    nextLines.length > 0 &&
+    nextLines[nextLines.length - 1].trim() === ""
+  ) {
+    nextLines.pop();
+  }
+  for (const [key, value] of remaining) nextLines.push(`${key}=${value}`);
+  return `${nextLines.join("\n")}\n`;
+}
+
+/**
+ * Load and merge every env layer. Returns:
+ *   values  — merged KEY -> value (real secrets; never render these),
+ *   sources — KEY -> 'process' | 'repo' | 'home' (winning layer),
+ *   layers  — [{ source, path, exists }] for display ("loaded from ...").
+ * All roots/paths are injectable for tests; by default the repo root is this
+ * checkout.
+ */
+export function loadLayeredEnv(options = {}) {
+  const {
+    processEnv = process.env,
+    repoRoot = ROOT,
+    homeEnvPath = HOME_ENV_PATH,
+  } = options;
+  const filePaths = [];
+  const pushUnique = (source, path) => {
+    if (path && !filePaths.some((layer) => layer.path === path)) {
+      filePaths.push({ source, path });
+    }
+  };
+  pushUnique("repo", join(repoRoot, ".env"));
+  pushUnique("home", homeEnvPath);
+  const layers = [
+    { source: "process", path: null, exists: true, values: processEnv },
+    ...filePaths.map(({ source, path }) => {
+      const exists = existsSync(path);
+      return {
+        source,
+        path,
+        exists,
+        values: exists ? parseDotenv(readFileSync(path, "utf8")) : {},
+      };
+    }),
+  ];
+  const { values, sources } = mergeEnvLayers(layers);
+  return {
+    values,
+    sources,
+    layers: layers.map(({ source, path, exists }) => ({
+      source,
+      path,
+      exists,
+    })),
+  };
+}
+
+/**
+ * Load the layered env and fill process.env with every file-layer value whose
+ * key the process does not already define. The lane driver and status
+ * collector call this once at startup so their own readiness checks AND the
+ * test suites they spawn observe exactly the resolution the dashboard
+ * displays; the dashboard itself never calls this (it keeps process.env
+ * pristine and reads the merged map instead). Returns the loadLayeredEnv
+ * result for layer display.
+ */
+export function applyLayeredEnvToProcess(options = {}) {
+  const loaded = loadLayeredEnv(options);
+  const processEnv = options.processEnv ?? process.env;
+  for (const [key, value] of Object.entries(loaded.values)) {
+    if (processEnv[key] === undefined) processEnv[key] = value;
+  }
+  return loaded;
+}
+
+/**
+ * Display-safe presence report for the given env names: present means a
+ * non-empty value after trimming; source is the winning layer (attributed even
+ * for empty-but-defined values, null when no layer defines the key). Never
+ * returns values.
+ */
+export function listPresent(names, options = {}) {
+  const { values, sources } = loadLayeredEnv(options);
+  return names.map((name) => {
+    const value = values[name];
+    return {
+      name,
+      present: typeof value === "string" && value.trim().length > 0,
+      source: sources[name] ?? null,
+    };
+  });
+}
+
+function atomicWriteEnvFile(path, content) {
+  mkdirSync(dirname(path), { recursive: true });
+  // Mode 600 on the tmp file carries through the rename, so the final file is
+  // owner-only even when it replaces a pre-existing looser one.
+  const tmp = `${path}.${process.pid}.${Date.now()}.tmp`;
+  writeFileSync(tmp, content, { encoding: "utf8", mode: 0o600 });
+  renameSync(tmp, path);
+}
+
+/**
+ * Upsert one KEY=value into the chosen layer file — scope 'home'
+ * (~/.eliza/.env, created on first save; the default because it survives
+ * worktree churn) or 'repo' (this checkout's .env). Atomic tmp+rename write,
+ * mode 600. Also sets the key on processEnv so probes running in the same
+ * process observe the save immediately. Values must be single-line; multi-line
+ * values would corrupt the dotenv format and are rejected.
+ */
+export function writeSecret(key, value, options = {}) {
+  const {
+    scope = "home",
+    repoRoot = ROOT,
+    homeEnvPath = HOME_ENV_PATH,
+    processEnv = process.env,
+  } = options;
+  if (typeof key !== "string" || !ENV_KEY_PATTERN.test(key)) {
+    throw new Error(`writeSecret: invalid env key ${JSON.stringify(key)}`);
+  }
+  if (typeof value !== "string" || /[\r\n]/.test(value)) {
+    throw new Error(`writeSecret(${key}): value must be a single-line string`);
+  }
+  if (scope !== "home" && scope !== "repo") {
+    throw new Error(
+      `writeSecret(${key}): scope must be "home" or "repo", got ${JSON.stringify(scope)}`,
+    );
+  }
+  const path = scope === "home" ? homeEnvPath : join(repoRoot, ".env");
+  const existing = existsSync(path) ? readFileSync(path, "utf8") : "";
+  atomicWriteEnvFile(path, upsertEnvContent(existing, { [key]: value }));
+  processEnv[key] = value;
+  return { key, scope, path };
+}
+
+export function saveEnvVar(key, value, target = "home", options = {}) {
+  const saved = writeSecret(key, value, { ...options, scope: target });
+  return { key: saved.key, target: saved.scope, path: saved.path };
+}
+
+// --- CLI: presence/source inspection (never prints values) -------------------
+
+const IS_MAIN =
+  import.meta.main || process.argv[1] === fileURLToPath(import.meta.url);
+
+if (IS_MAIN) {
+  const args = process.argv.slice(2);
+  const json = args.includes("--json");
+  const names = args.filter((arg) => !arg.startsWith("--"));
+  const { layers } = loadLayeredEnv();
+  const rows = names.length > 0 ? listPresent(names) : [];
+  if (json) {
+    console.log(JSON.stringify({ layers, present: rows }, null, 2));
+  } else {
+    for (const layer of layers) {
+      console.log(
+        `${layer.source.padEnd(8)} ${layer.path ?? "(process.env)"}${layer.exists ? "" : " (absent)"}`,
+      );
+    }
+    for (const row of rows) {
+      console.log(
+        `${row.present ? "present" : "absent "} [${row.source ?? "-"}] ${row.name}`,
+      );
+    }
+  }
+}

scripts/lifeops/env-layers.test.mjs

@@ -0,0 +1,395 @@
+/**
+ * Unit tests for layered .env resolution: pure parse/merge/upsert primitives,
+ * real-filesystem load/save against temp dirs (mode 600 asserted), and a real
+ * linked-worktree fixture proving an empty worktree .env falls through to the
+ * home-scoped layer.
+ */
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import {
+  mkdirSync,
+  mkdtempSync,
+  readFileSync,
+  realpathSync,
+  rmSync,
+  statSync,
+  writeFileSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join, resolve } from "node:path";
+import test from "node:test";
+import {
+  applyLayeredEnvToProcess,
+  listPresent,
+  loadLayeredEnv,
+  mergeEnvLayers,
+  parseDotenv,
+  saveEnvVar,
+  upsertEnvContent,
+  writeSecret,
+} from "./env-layers.mjs";
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+
+// realpath so git-canonicalized paths (macOS /var -> /private/var) compare equal.
+function tempDir(prefix) {
+  return realpathSync(mkdtempSync(join(tmpdir(), prefix)));
+}
+
+function git(cwd, args) {
+  const result = spawnSync("git", args, { cwd, encoding: "utf8" });
+  assert.equal(
+    result.status,
+    0,
+    `git ${args.join(" ")} failed: ${result.stderr}`,
+  );
+  return result.stdout.trim();
+}
+
+// --- parseDotenv --------------------------------------------------------------
+
+test("parseDotenv handles comments, export prefix, quotes, and CRLF", () => {
+  const parsed = parseDotenv(
+    [
+      "# comment",
+      "PLAIN=value",
+      "export EXPORTED=exported-value",
+      'DQ="double quoted"',
+      "SQ='single quoted'",
+      "SPACED =  padded  ",
+      "not a valid line",
+      "EMPTY=",
+      "1BAD=starts-with-digit",
+    ].join("\r\n"),
+  );
+  assert.deepEqual(parsed, {
+    PLAIN: "value",
+    EXPORTED: "exported-value",
+    DQ: "double quoted",
+    SQ: "single quoted",
+    SPACED: "padded",
+    EMPTY: "",
+  });
+});
+
+// --- mergeEnvLayers -------------------------------------------------------------
+
+test("mergeEnvLayers: first (highest-precedence) definition wins, sources attributed", () => {
+  const { values, sources } = mergeEnvLayers([
+    { source: "process", values: { A: "proc", EMPTYWIN: "" } },
+    { source: "repo", values: { A: "repo", B: "repo", EMPTYWIN: "file" } },
+    { source: "home", values: { C: "home", D: "home", SKIPPED: undefined } },
+  ]);
+  assert.deepEqual(values, {
+    A: "proc",
+    B: "repo",
+    C: "home",
+    D: "home",
+    EMPTYWIN: "",
+  });
+  assert.deepEqual(sources, {
+    A: "process",
+    B: "repo",
+    C: "home",
+    D: "home",
+    EMPTYWIN: "process",
+  });
+});
+
+// --- upsertEnvContent ------------------------------------------------------------
+
+test("upsertEnvContent replaces in place, preserves comments, appends new keys", () => {
+  const before = [
+    "# keep me",
+    "KEEP=old-keep",
+    "REPLACE=old",
+    "",
+    "export ALSO=old-also",
+  ].join("\n");
+  const after = upsertEnvContent(before, {
+    REPLACE: "new",
+    ALSO: "new-also",
+    ADDED: "fresh",
+  });
+  assert.equal(
+    after,
+    [
+      "# keep me",
+      "KEEP=old-keep",
+      "REPLACE=new",
+      "",
+      "ALSO=new-also",
+      "ADDED=fresh",
+      "",
+    ].join("\n"),
+  );
+});
+
+test("upsertEnvContent on empty text emits just the entries", () => {
+  assert.equal(upsertEnvContent("", { A: "1" }), "A=1\n");
+});
+
+// --- loadLayeredEnv / listPresent ---------------------------------------------------
+
+test("loadLayeredEnv merges process > repo > home and reports layers", () => {
+  const base = tempDir("env-layers-load-");
+  try {
+    const repoRoot = join(base, "repo");
+    const homeEnvPath = join(base, "home", ".eliza", ".env");
+    for (const dir of [repoRoot, join(base, "home", ".eliza")]) {
+      mkdirSync(dir, { recursive: true });
+    }
+    writeFileSync(join(repoRoot, ".env"), "A=repo\nB=repo\n");
+    writeFileSync(homeEnvPath, "B=home\nC=home\n");
+    const { values, sources, layers } = loadLayeredEnv({
+      processEnv: { A: "proc" },
+      repoRoot,
+      homeEnvPath,
+    });
+    assert.equal(values.A, "proc");
+    assert.equal(values.B, "repo");
+    assert.equal(values.C, "home");
+    assert.deepEqual(sources, {
+      A: "process",
+      B: "repo",
+      C: "home",
+    });
+    assert.deepEqual(
+      layers.map((layer) => [layer.source, layer.exists]),
+      [
+        ["process", true],
+        ["repo", true],
+        ["home", true],
+      ],
+    );
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("loadLayeredEnv: missing files are graceful", () => {
+  const base = tempDir("env-layers-absent-");
+  try {
+    const repoRoot = join(base, "repo");
+    mkdirSync(repoRoot, { recursive: true });
+    const { values, sources, layers } = loadLayeredEnv({
+      processEnv: {},
+      repoRoot,
+      homeEnvPath: join(base, "nonexistent", ".env"),
+    });
+    assert.deepEqual(values, {});
+    assert.deepEqual(sources, {});
+    assert.deepEqual(
+      layers.map((layer) => layer.source),
+      ["process", "repo", "home"],
+    );
+    assert.equal(
+      layers.every((layer) => layer.source === "process" || !layer.exists),
+      true,
+    );
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("applyLayeredEnvToProcess hydrates only keys the process does not define", () => {
+  const base = tempDir("env-layers-apply-");
+  try {
+    const repoRoot = join(base, "repo");
+    mkdirSync(repoRoot, { recursive: true });
+    writeFileSync(join(repoRoot, ".env"), "FROM_REPO=repo\nKEPT=shadowed\n");
+    const homeEnvPath = join(base, "home.env");
+    writeFileSync(homeEnvPath, "FROM_HOME=home\nFROM_REPO=home-loses\n");
+    const processEnv = { KEPT: "process-wins", EMPTY: "" };
+    const loaded = applyLayeredEnvToProcess({
+      processEnv,
+      repoRoot,
+      homeEnvPath,
+    });
+    assert.equal(processEnv.FROM_REPO, "repo");
+    assert.equal(processEnv.FROM_HOME, "home");
+    assert.equal(processEnv.KEPT, "process-wins");
+    assert.equal(processEnv.EMPTY, "", "empty-but-defined keys stay untouched");
+    assert.equal(loaded.sources.FROM_REPO, "repo");
+    assert.equal(loaded.sources.KEPT, "process");
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("fresh linked worktree with empty repo .env uses home-scoped secrets, not main checkout .env", () => {
+  const base = tempDir("env-layers-worktree-home-");
+  try {
+    const mainRoot = join(base, "main");
+    const wtRoot = join(base, "wt");
+    const homeEnvPath = join(base, "home", ".eliza", ".env");
+    git(base, ["init", "-b", "main", "main"]);
+    writeFileSync(join(mainRoot, "seed.txt"), "seed\n");
+    writeFileSync(join(mainRoot, ".env"), "TOKEN=stale-main\n");
+    git(mainRoot, ["add", "seed.txt"]);
+    git(mainRoot, [
+      "-c",
+      "user.email=test@example.com",
+      "-c",
+      "user.name=Test",
+      "commit",
+      "-m",
+      "seed",
+    ]);
+    git(mainRoot, ["worktree", "add", wtRoot]);
+    mkdirSync(dirname(homeEnvPath), { recursive: true });
+    writeFileSync(homeEnvPath, "TOKEN=home-secret\n");
+
+    const loaded = loadLayeredEnv({
+      processEnv: {},
+      repoRoot: wtRoot,
+      homeEnvPath,
+    });
+    assert.equal(loaded.values.TOKEN, "home-secret");
+    assert.equal(loaded.sources.TOKEN, "home");
+    assert.deepEqual(
+      loaded.layers.map((layer) => layer.source),
+      ["process", "repo", "home"],
+    );
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("listPresent attributes each source and never returns values", () => {
+  const base = tempDir("env-layers-present-");
+  try {
+    const repoRoot = join(base, "repo");
+    const homeEnvPath = join(base, "home.env");
+    mkdirSync(repoRoot, { recursive: true });
+    writeFileSync(join(repoRoot, ".env"), "FROM_REPO=secret-repo\nEMPTYVAL=\n");
+    writeFileSync(homeEnvPath, "FROM_HOME=secret-home\n");
+    const rows = listPresent(
+      ["FROM_PROC", "FROM_REPO", "FROM_HOME", "EMPTYVAL", "ABSENT"],
+      {
+        processEnv: { FROM_PROC: "secret-proc" },
+        repoRoot,
+        homeEnvPath,
+      },
+    );
+    assert.deepEqual(rows, [
+      { name: "FROM_PROC", present: true, source: "process" },
+      { name: "FROM_REPO", present: true, source: "repo" },
+      { name: "FROM_HOME", present: true, source: "home" },
+      { name: "EMPTYVAL", present: false, source: "repo" },
+      { name: "ABSENT", present: false, source: null },
+    ]);
+    assert.equal(JSON.stringify(rows).includes("secret-"), false);
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+// --- saveEnvVar -----------------------------------------------------------------------
+
+test("writeSecret creates the home file with mode 600 and upserts on re-save", () => {
+  const base = tempDir("env-layers-save-");
+  try {
+    const homeEnvPath = join(base, ".eliza", ".env");
+    const processEnv = {};
+    const first = writeSecret("NEW_TOKEN", "tok-1", {
+      scope: "home",
+      homeEnvPath,
+      processEnv,
+    });
+    assert.deepEqual(first, {
+      key: "NEW_TOKEN",
+      scope: "home",
+      path: homeEnvPath,
+    });
+    assert.equal(readFileSync(homeEnvPath, "utf8"), "NEW_TOKEN=tok-1\n");
+    assert.equal(statSync(homeEnvPath).mode & 0o777, 0o600);
+    assert.equal(processEnv.NEW_TOKEN, "tok-1");
+
+    writeFileSync(homeEnvPath, "# note\nNEW_TOKEN=tok-1\nOTHER=keep\n");
+    writeSecret("NEW_TOKEN", "tok-2", {
+      scope: "home",
+      homeEnvPath,
+      processEnv,
+    });
+    assert.equal(
+      readFileSync(homeEnvPath, "utf8"),
+      "# note\nNEW_TOKEN=tok-2\nOTHER=keep\n",
+    );
+    assert.equal(statSync(homeEnvPath).mode & 0o777, 0o600);
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("writeSecret writes the repo layer when scoped", () => {
+  const base = tempDir("env-layers-save-repo-");
+  try {
+    const processEnv = {};
+    const result = writeSecret("REPO_ONLY", "x", {
+      scope: "repo",
+      repoRoot: base,
+      processEnv,
+    });
+    assert.equal(result.path, join(base, ".env"));
+    assert.equal(readFileSync(join(base, ".env"), "utf8"), "REPO_ONLY=x\n");
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("saveEnvVar remains a compatibility wrapper for existing dashboard callers", () => {
+  const base = tempDir("env-layers-save-wrapper-");
+  try {
+    const processEnv = {};
+    const result = saveEnvVar("WRAPPED", "x", "repo", {
+      repoRoot: base,
+      processEnv,
+    });
+    assert.deepEqual(result, {
+      key: "WRAPPED",
+      target: "repo",
+      path: join(base, ".env"),
+    });
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("writeSecret rejects invalid keys, multi-line values, and bad scopes", () => {
+  const base = tempDir("env-layers-save-bad-");
+  try {
+    const options = { homeEnvPath: join(base, ".env"), processEnv: {} };
+    assert.throws(
+      () => writeSecret("bad key", "v", options),
+      /invalid env key/,
+    );
+    assert.throws(
+      () => writeSecret("GOOD_KEY", "a\nb", options),
+      /single-line/,
+    );
+    assert.throws(
+      () => writeSecret("GOOD_KEY", "v", { ...options, scope: "elsewhere" }),
+      /scope/,
+    );
+  } finally {
+    rmSync(base, { recursive: true, force: true });
+  }
+});
+
+test("HITL dashboard, lane driver, collector, and tests import the shared layered env module", () => {
+  const importers = [
+    "scripts/lifeops/hitl-credential-dashboard.mjs",
+    "scripts/lifeops/run-11632-live-lanes.mjs",
+    "scripts/lifeops/collect-11632-live-validation-status.mjs",
+    "scripts/lifeops/env-layers.test.mjs",
+  ];
+  for (const relativePath of importers) {
+    const text = readFileSync(join(ROOT, relativePath), "utf8");
+    assert.match(
+      text,
+      /from "\.\/env-layers\.mjs"/,
+      `${relativePath} must import scripts/lifeops/env-layers.mjs`,
+    );
+  }
+});

scripts/lifeops/github-device-login.mjs

@@ -0,0 +1,177 @@
+/**
+ * GitHub OAuth device-flow state for the LifeOps credential dashboard.
+ *
+ * The browser receives only the short user code and an opaque local flow id;
+ * the device code stays in server memory until GitHub returns a token or the
+ * ten-minute flow expires. Network access is injectable so protocol behavior
+ * is covered without contacting GitHub in deterministic tests.
+ */
+import { randomBytes } from "node:crypto";
+
+const DEVICE_CODE_URL = "https://github.com/login/device/code";
+const ACCESS_TOKEN_URL = "https://github.com/login/oauth/access_token";
+const DEFAULT_SCOPE = "repo read:user";
+const pendingFlows = new Map();
+
+function protocolError(label, response, payload) {
+  const providerError =
+    typeof payload?.error === "string" ? ` (${payload.error})` : "";
+  return new Error(`${label} failed: HTTP ${response.status}${providerError}`);
+}
+
+async function jsonResponse(response, label) {
+  let payload;
+  try {
+    payload = await response.json();
+  } catch (error) {
+    throw new Error(`${label} returned invalid JSON`, { cause: error });
+  }
+  if (!response.ok) throw protocolError(label, response, payload);
+  return payload;
+}
+
+function requireString(payload, key, label) {
+  const value = payload?.[key];
+  if (typeof value !== "string" || value.trim().length === 0) {
+    throw new Error(`${label} response is missing ${key}`);
+  }
+  return value.trim();
+}
+
+function positiveNumber(value, fallback) {
+  return typeof value === "number" && Number.isFinite(value) && value > 0
+    ? value
+    : fallback;
+}
+
+function sweepExpired(nowMs) {
+  for (const [flowId, flow] of pendingFlows) {
+    if (nowMs >= flow.expiresAtMs) pendingFlows.delete(flowId);
+  }
+}
+
+export async function startGitHubDeviceLogin({
+  clientId,
+  target,
+  fetchFn = fetch,
+  now = Date.now,
+  randomBytesFn = randomBytes,
+}) {
+  if (typeof clientId !== "string" || clientId.trim().length === 0) {
+    throw new Error(
+      "GitHub device login needs owner setup: GITHUB_OAUTH_CLIENT_ID is absent",
+    );
+  }
+  if (target !== "home" && target !== "repo") {
+    throw new Error('GitHub device login target must be "home" or "repo"');
+  }
+  const response = await fetchFn(DEVICE_CODE_URL, {
+    method: "POST",
+    headers: {
+      Accept: "application/json",
+      "Content-Type": "application/x-www-form-urlencoded",
+    },
+    body: new URLSearchParams({
+      client_id: clientId.trim(),
+      scope: DEFAULT_SCOPE,
+    }).toString(),
+  });
+  const payload = await jsonResponse(response, "GitHub device-code request");
+  const deviceCode = requireString(
+    payload,
+    "device_code",
+    "GitHub device-code",
+  );
+  const userCode = requireString(payload, "user_code", "GitHub device-code");
+  const verificationUri = requireString(
+    payload,
+    "verification_uri",
+    "GitHub device-code",
+  );
+  const intervalSeconds = positiveNumber(payload.interval, 5);
+  const expiresInSeconds = positiveNumber(payload.expires_in, 900);
+  const nowMs = now();
+  sweepExpired(nowMs);
+  const flowId = randomBytesFn(24).toString("base64url");
+  pendingFlows.set(flowId, {
+    clientId: clientId.trim(),
+    deviceCode,
+    target,
+    intervalSeconds,
+    nextPollAtMs: nowMs,
+    expiresAtMs: nowMs + expiresInSeconds * 1_000,
+  });
+  return {
+    flowId,
+    userCode,
+    verificationUri,
+    intervalSeconds,
+    expiresInSeconds,
+  };
+}
+
+export async function pollGitHubDeviceLogin({
+  flowId,
+  fetchFn = fetch,
+  now = Date.now,
+}) {
+  const nowMs = now();
+  sweepExpired(nowMs);
+  const flow = pendingFlows.get(flowId);
+  if (!flow) {
+    throw new Error("GitHub device login is unknown or expired");
+  }
+  if (nowMs < flow.nextPollAtMs) {
+    return {
+      status: "pending",
+      retryAfterSeconds: Math.max(
+        1,
+        Math.ceil((flow.nextPollAtMs - nowMs) / 1_000),
+      ),
+    };
+  }
+  flow.nextPollAtMs = nowMs + flow.intervalSeconds * 1_000;
+  const response = await fetchFn(ACCESS_TOKEN_URL, {
+    method: "POST",
+    headers: {
+      Accept: "application/json",
+      "Content-Type": "application/x-www-form-urlencoded",
+    },
+    body: new URLSearchParams({
+      client_id: flow.clientId,
+      device_code: flow.deviceCode,
+      grant_type: "urn:ietf:params:oauth:grant-type:device_code",
+    }).toString(),
+  });
+  const payload = await jsonResponse(response, "GitHub device-token request");
+  if (
+    typeof payload.access_token === "string" &&
+    payload.access_token.length > 0
+  ) {
+    pendingFlows.delete(flowId);
+    return {
+      status: "complete",
+      token: payload.access_token,
+      target: flow.target,
+      tokenType:
+        typeof payload.token_type === "string" ? payload.token_type : "bearer",
+      scope: typeof payload.scope === "string" ? payload.scope : "",
+    };
+  }
+  if (payload.error === "authorization_pending") {
+    return { status: "pending", retryAfterSeconds: flow.intervalSeconds };
+  }
+  if (payload.error === "slow_down") {
+    flow.intervalSeconds += 5;
+    flow.nextPollAtMs = nowMs + flow.intervalSeconds * 1_000;
+    return { status: "pending", retryAfterSeconds: flow.intervalSeconds };
+  }
+  pendingFlows.delete(flowId);
+  const code =
+    typeof payload.error === "string" ? payload.error : "unknown_error";
+  throw new Error(`GitHub device login failed: ${code}`);
+}
+
+export function clearGitHubDeviceLoginsForTest() {
+  pendingFlows.clear();
+}

scripts/lifeops/github-device-login.test.mjs

@@ -0,0 +1,135 @@
+/** GitHub device-flow protocol tests use injected responses and no network. */
+import assert from "node:assert/strict";
+import test from "node:test";
+import {
+  clearGitHubDeviceLoginsForTest,
+  pollGitHubDeviceLogin,
+  startGitHubDeviceLogin,
+} from "./github-device-login.mjs";
+
+function response(payload, status = 200) {
+  return new Response(JSON.stringify(payload), {
+    status,
+    headers: { "Content-Type": "application/json" },
+  });
+}
+
+test.afterEach(() => clearGitHubDeviceLoginsForTest());
+
+test("device flow keeps the device code server-side and returns the token once", async () => {
+  let nowMs = 1_000;
+  const requests = [];
+  const fetchFn = async (url, init) => {
+    requests.push({ url, body: String(init.body) });
+    if (requests.length === 1) {
+      return response({
+        device_code: "secret-device-code",
+        user_code: "ABCD-EFGH",
+        verification_uri: "https://github.com/login/device",
+        expires_in: 900,
+        interval: 5,
+      });
+    }
+    if (requests.length === 2)
+      return response({ error: "authorization_pending" });
+    return response({
+      access_token: "gho_live_token_value",
+      token_type: "bearer",
+      scope: "repo,read:user",
+    });
+  };
+  const started = await startGitHubDeviceLogin({
+    clientId: "client-1",
+    target: "home",
+    fetchFn,
+    now: () => nowMs,
+    randomBytesFn: () => Buffer.from("opaque-flow-id"),
+  });
+  assert.deepEqual(started, {
+    flowId: Buffer.from("opaque-flow-id").toString("base64url"),
+    userCode: "ABCD-EFGH",
+    verificationUri: "https://github.com/login/device",
+    intervalSeconds: 5,
+    expiresInSeconds: 900,
+  });
+  assert.equal(JSON.stringify(started).includes("secret-device-code"), false);
+
+  assert.deepEqual(
+    await pollGitHubDeviceLogin({
+      flowId: started.flowId,
+      fetchFn,
+      now: () => nowMs,
+    }),
+    { status: "pending", retryAfterSeconds: 5 },
+  );
+  nowMs += 5_000;
+  const completed = await pollGitHubDeviceLogin({
+    flowId: started.flowId,
+    fetchFn,
+    now: () => nowMs,
+  });
+  assert.deepEqual(completed, {
+    status: "complete",
+    token: "gho_live_token_value",
+    target: "home",
+    tokenType: "bearer",
+    scope: "repo,read:user",
+  });
+  await assert.rejects(
+    pollGitHubDeviceLogin({
+      flowId: started.flowId,
+      fetchFn,
+      now: () => nowMs,
+    }),
+    /unknown or expired/,
+  );
+  assert.match(requests[1].body, /device_code=secret-device-code/);
+});
+
+test("slow_down raises the server-owned polling interval", async () => {
+  let nowMs = 10_000;
+  let calls = 0;
+  const fetchFn = async () => {
+    calls += 1;
+    return calls === 1
+      ? response({
+          device_code: "device",
+          user_code: "CODE",
+          verification_uri: "https://github.com/login/device",
+          expires_in: 60,
+          interval: 2,
+        })
+      : response({ error: "slow_down" });
+  };
+  const started = await startGitHubDeviceLogin({
+    clientId: "client",
+    target: "repo",
+    fetchFn,
+    now: () => nowMs,
+  });
+  assert.deepEqual(
+    await pollGitHubDeviceLogin({
+      flowId: started.flowId,
+      fetchFn,
+      now: () => nowMs,
+    }),
+    { status: "pending", retryAfterSeconds: 7 },
+  );
+  nowMs += 1_000;
+  assert.deepEqual(
+    await pollGitHubDeviceLogin({
+      flowId: started.flowId,
+      fetchFn,
+      now: () => nowMs,
+    }),
+    { status: "pending", retryAfterSeconds: 6 },
+  );
+  assert.equal(calls, 2);
+});
+
+test("missing client registration is an explicit owner-setup state", async () => {
+  await assert.rejects(
+    startGitHubDeviceLogin({ clientId: "", target: "home" }),
+    /needs owner setup/,
+  );
+});

scripts/lifeops/hitl-credential-dashboard.test.mjs

@@ -0,0 +1,142 @@
+/**
+ * Loopback integration coverage for the HITL credential dashboard's write
+ * boundary. The test starts the real dashboard process with an isolated HOME so
+ * forged browser-style POSTs can prove they do not mutate any operator env file.
+ */
+import assert from "node:assert/strict";
+import { spawn } from "node:child_process";
+import {
+  existsSync,
+  mkdtempSync,
+  readFileSync,
+  realpathSync,
+  rmSync,
+} from "node:fs";
+import { tmpdir } from "node:os";
+import { join, resolve } from "node:path";
+import test from "node:test";
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+
+function tempDir(prefix) {
+  return realpathSync(mkdtempSync(join(tmpdir(), prefix)));
+}
+
+function waitForDashboard(child) {
+  return new Promise((resolvePromise, reject) => {
+    let output = "";
+    const timer = setTimeout(() => {
+      reject(new Error(`dashboard did not start:\n${output}`));
+    }, 20_000);
+    const onData = (chunk) => {
+      output += chunk.toString();
+      const match = /listening on (http:\/\/127\.0\.0\.1:\d+\/)/.exec(output);
+      if (!match) return;
+      clearTimeout(timer);
+      child.stdout.off("data", onData);
+      child.stderr.off("data", onData);
+      resolvePromise(match[1]);
+    };
+    child.stdout.on("data", onData);
+    child.stderr.on("data", onData);
+    child.on("exit", (status, signal) => {
+      clearTimeout(timer);
+      reject(
+        new Error(
+          `dashboard exited before listening: status=${status} signal=${signal}\n${output}`,
+        ),
+      );
+    });
+  });
+}
+
+async function postEnv(baseUrl, headers, value) {
+  return fetch(`${baseUrl}api/env`, {
+    method: "POST",
+    headers: {
+      "Content-Type": "application/json",
+      ...headers,
+    },
+    body: JSON.stringify({
+      key: "TELEGRAM_BOT_TOKEN",
+      value,
+      target: "home",
+    }),
+  });
+}
+
+test("credential dashboard rejects cross-site writes and requires its page session", async () => {
+  const home = tempDir("hitl-dashboard-home-");
+  const child = spawn(
+    "node",
+    ["scripts/lifeops/hitl-credential-dashboard.mjs"],
+    {
+      cwd: ROOT,
+      env: {
+        ...process.env,
+        HOME: home,
+        XDG_CONFIG_HOME: join(home, ".config"),
+        XDG_CACHE_HOME: join(home, ".cache"),
+      },
+      stdio: ["ignore", "pipe", "pipe"],
+    },
+  );
+  const envPath = join(home, ".eliza", ".env");
+  try {
+    const baseUrl = await waitForDashboard(child);
+    const sameOrigin = new URL(baseUrl).origin;
+
+    const forged = await postEnv(
+      baseUrl,
+      { Origin: "http://attacker.invalid" },
+      "attacker-token",
+    );
+    assert.equal(forged.status, 403);
+    assert.equal(existsSync(envPath), false);
+
+    const pageResponse = await fetch(baseUrl);
+    assert.equal(pageResponse.status, 200);
+    const pageHtml = await pageResponse.text();
+    const tokenMatch = /var SESSION_TOKEN = "([^"]+)";/.exec(pageHtml);
+    assert.ok(tokenMatch, "dashboard page exposes a per-process session token");
+
+    const missingToken = await postEnv(
+      baseUrl,
+      { Origin: sameOrigin },
+      "missing-session-token",
+    );
+    assert.equal(missingToken.status, 403);
+    assert.equal(existsSync(envPath), false);
+
+    const wrongContentType = await fetch(`${baseUrl}api/env`, {
+      method: "POST",
+      headers: {
+        Origin: sameOrigin,
+        "Content-Type": "text/plain",
+        "X-HITL-Session": tokenMatch[1],
+      },
+      body: JSON.stringify({
+        key: "TELEGRAM_BOT_TOKEN",
+        value: "wrong-content-type",
+        target: "home",
+      }),
+    });
+    assert.equal(wrongContentType.status, 415);
+    assert.equal(existsSync(envPath), false);
+
+    const saved = await postEnv(
+      baseUrl,
+      { Origin: sameOrigin, "X-HITL-Session": tokenMatch[1] },
+      "operator-token-value",
+    );
+    assert.equal(saved.status, 200);
+    assert.match(
+      readFileSync(envPath, "utf8"),
+      /TELEGRAM_BOT_TOKEN=operator-token-value/,
+    );
+  } finally {
+    child.kill("SIGTERM");
+    await new Promise((resolvePromise) => child.once("exit", resolvePromise));
+    rmSync(home, { recursive: true, force: true });
+  }
+});

scripts/lifeops/hitl-ledger.mjs

@@ -0,0 +1,223 @@
+#!/usr/bin/env node
+/**
+ * Committed evidence ledger for HITL connector auth-path validation (#11632).
+ * docs/testing/hitl-ledger.json records, per connector auth path (a
+ * CONNECTOR_PATHS id), when that path last ran and last succeeded against the
+ * real provider: { pathId, lastSuccessAt, lastRunAt, lane, commit, counts }.
+ * `counts` mirrors the run-11632-live-lanes summary shape
+ * { passed, failed, skipped }. The HITL dashboard writes one entry per probe
+ * (lane "dashboard-probe"); lane drivers can record with their own lane ids.
+ * The file is committed evidence, so it must never contain secrets — only
+ * path ids, timestamps, lane names, commit shas, and counts — and entries are
+ * kept sorted by pathId for stable diffs.
+ *
+ * Freshness policy (locked): green when lastSuccessAt is within 7 days,
+ * yellow beyond 7, red beyond 30 or never proven.
+ */
+import {
+  existsSync,
+  mkdirSync,
+  readFileSync,
+  renameSync,
+  writeFileSync,
+} from "node:fs";
+import { dirname, join, resolve } from "node:path";
+
+const ROOT = resolve(new URL("../..", import.meta.url).pathname);
+
+export const LEDGER_PATH = join(ROOT, "docs/testing/hitl-ledger.json");
+
+export const FRESH_DAYS = 7;
+export const STALE_DAYS = 30;
+const DAY_MS = 86_400_000;
+const LEDGER_KEYS = new Set(["version", "updatedAt", "entries"]);
+const ENTRY_KEYS = new Set([
+  "pathId",
+  "lastSuccessAt",
+  "lastRunAt",
+  "lane",
+  "commit",
+  "counts",
+]);
+const OUTCOME_KEYS = new Set([
+  "pathId",
+  "ok",
+  "at",
+  "lane",
+  "commit",
+  "counts",
+]);
+const COUNT_KEYS = new Set(["passed", "failed", "skipped"]);
+const SECRET_SHAPED_KEY_PATTERN =
+  /(token|secret|password|credential|private[_-]?key|api[_-]?key|auth)/i;
+
+function assertNoUnknownKeys(label, object, allowed) {
+  for (const key of Object.keys(object)) {
+    if (SECRET_SHAPED_KEY_PATTERN.test(key)) {
+      throw new Error(
+        `hitl-ledger(${label}): secret-shaped field '${key}' is forbidden`,
+      );
+    }
+    if (!allowed.has(key)) {
+      throw new Error(`hitl-ledger(${label}): unknown field '${key}'`);
+    }
+  }
+}
+
+function assertCounts(counts, label) {
+  if (!counts || typeof counts !== "object" || Array.isArray(counts)) {
+    throw new Error(`hitl-ledger(${label}): counts must be an object`);
+  }
+  assertNoUnknownKeys(`${label}.counts`, counts, COUNT_KEYS);
+  for (const key of COUNT_KEYS) {
+    if (!Number.isInteger(counts[key]) || counts[key] < 0) {
+      throw new Error(
+        `hitl-ledger(${label}): counts.${key} must be a non-negative integer`,
+      );
+    }
+  }
+}
+
+function assertIsoOrNull(value, label) {
+  if (value === null) return;
+  if (typeof value !== "string" || Number.isNaN(Date.parse(value))) {
+    throw new Error(`hitl-ledger(${label}): timestamp must be ISO or null`);
+  }
+}
+
+function assertLedgerEntry(pathId, entry) {
+  if (!entry || typeof entry !== "object" || Array.isArray(entry)) {
+    throw new Error(`hitl-ledger(${pathId}): entry must be an object`);
+  }
+  assertNoUnknownKeys(pathId, entry, ENTRY_KEYS);
+  if (entry.pathId !== pathId) {
+    throw new Error(
+      `hitl-ledger(${pathId}): entry.pathId must match its ledger key`,
+    );
+  }
+  assertIsoOrNull(entry.lastSuccessAt, `${pathId}.lastSuccessAt`);
+  assertIsoOrNull(entry.lastRunAt, `${pathId}.lastRunAt`);
+  if (typeof entry.lane !== "string" || typeof entry.commit !== "string") {
+    throw new Error(`hitl-ledger(${pathId}): lane and commit are required`);
+  }
+  assertCounts(entry.counts, pathId);
+}
+
+/**
+ * Read the ledger, or the empty shape when the file does not exist yet. A
+ * malformed file throws: this is committed evidence, not a regenerable cache,
+ * so silent discard would erase proof history.
+ */
+export function readLedger(path = LEDGER_PATH) {
+  if (!existsSync(path)) return { version: 1, updatedAt: null, entries: {} };
+  const parsed = JSON.parse(readFileSync(path, "utf8"));
+  if (
+    !parsed ||
+    typeof parsed !== "object" ||
+    Array.isArray(parsed) ||
+    typeof parsed.entries !== "object" ||
+    parsed.entries === null
+  ) {
+    throw new Error(
+      `hitl-ledger: malformed ledger at ${path} — restore it from git history rather than regenerating`,
+    );
+  }
+  assertNoUnknownKeys("root", parsed, LEDGER_KEYS);
+  for (const [pathId, entry] of Object.entries(parsed.entries)) {
+    assertLedgerEntry(pathId, entry);
+  }
+  return {
+    version: parsed.version ?? 1,
+    updatedAt: parsed.updatedAt ?? null,
+    entries: parsed.entries,
+  };
+}
+
+/**
+ * Locked freshness bands from lastSuccessAt: green ≤7d, yellow >7d,
+ * red >30d-or-never. Returns { state, ageDays, label } for direct rendering.
+ */
+export function freshness(lastSuccessAt, now = Date.now()) {
+  if (typeof lastSuccessAt !== "string" || lastSuccessAt.length === 0) {
+    return { state: "red", ageDays: null, label: "never proven" };
+  }
+  const successMs = Date.parse(lastSuccessAt);
+  if (Number.isNaN(successMs)) {
+    throw new Error(`hitl-ledger: unparseable lastSuccessAt ${lastSuccessAt}`);
+  }
+  const ageDays = Math.max(0, Math.floor((now - successMs) / DAY_MS));
+  if (ageDays <= FRESH_DAYS) {
+    return { state: "green", ageDays, label: `proven ${ageDays}d ago` };
+  }
+  if (ageDays <= STALE_DAYS) {
+    return { state: "yellow", ageDays, label: `stale — ${ageDays}d ago` };
+  }
+  return { state: "red", ageDays, label: `stale — ${ageDays}d ago` };
+}
+
+function assertOutcome(outcome) {
+  const { pathId, ok, at, lane, commit, counts } = outcome;
+  assertNoUnknownKeys("outcome", outcome, OUTCOME_KEYS);
+  if (typeof pathId !== "string" || pathId.length === 0) {
+    throw new Error("hitl-ledger: outcome.pathId must be a non-empty string");
+  }
+  if (ok !== true && ok !== false && ok !== null) {
+    throw new Error(`hitl-ledger(${pathId}): ok must be true, false, or null`);
+  }
+  if (typeof at !== "string" || Number.isNaN(Date.parse(at))) {
+    throw new Error(`hitl-ledger(${pathId}): at must be an ISO timestamp`);
+  }
+  if (typeof lane !== "string" || typeof commit !== "string") {
+    throw new Error(`hitl-ledger(${pathId}): lane and commit are required`);
+  }
+  assertCounts(counts, pathId);
+}
+
+function atomicWrite(path, content) {
+  mkdirSync(dirname(path), { recursive: true });
+  const tmp = `${path}.${process.pid}.${Date.now()}.tmp`;
+  writeFileSync(tmp, content, "utf8");
+  renameSync(tmp, path);
+}
+
+/**
+ * Merge a batch of outcomes into the ledger with a single read+write, so
+ * probe-all's parallel results cannot lose updates to read-modify-write
+ * interleaving. `lastSuccessAt` only advances on ok === true; a failed or
+ * skipped run updates lastRunAt but preserves the last proof.
+ */
+export function recordOutcomes(outcomes, path = LEDGER_PATH) {
+  for (const outcome of outcomes) assertOutcome(outcome);
+  const ledger = readLedger(path);
+  for (const { pathId, ok, at, lane, commit, counts } of outcomes) {
+    const prior = ledger.entries[pathId];
+    ledger.entries[pathId] = {
+      pathId,
+      lastSuccessAt: ok === true ? at : (prior?.lastSuccessAt ?? null),
+      lastRunAt: at,
+      lane,
+      commit,
+      counts: {
+        passed: counts.passed,
+        failed: counts.failed,
+        skipped: counts.skipped,
+      },
+    };
+  }
+  const entries = Object.fromEntries(
+    Object.keys(ledger.entries)
+      .sort()
+      .map((key) => [key, ledger.entries[key]]),
+  );
+  const next = {
+    version: 1,
+    updatedAt: new Date().toISOString(),
+    entries,
+  };
+  atomicWrite(path, `${JSON.stringify(next, null, 2)}\n`);
+  return next;
+}
+
+export function recordOutcome(outcome, path = LEDGER_PATH) {
+  return recordOutcomes([outcome], path).entries[outcome.pathId];
+}

scripts/lifeops/hitl-ledger.test.mjs

@@ -0,0 +1,245 @@
+/**
+ * Unit tests for the committed HITL evidence ledger: locked freshness bands
+ * (green ≤7d, yellow >7d, red >30d-or-never), upsert semantics (lastSuccessAt
+ * only advances on success), stable serialization for clean diffs, and outcome
+ * validation — all against real temp-dir files, no mocks.
+ */
+import assert from "node:assert/strict";
+import { mkdtempSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import test from "node:test";
+import {
+  FRESH_DAYS,
+  freshness,
+  readLedger,
+  recordOutcome,
+  recordOutcomes,
+  STALE_DAYS,
+} from "./hitl-ledger.mjs";
+
+const DAY_MS = 86_400_000;
+const NOW = Date.parse("2026-07-06T00:00:00.000Z");
+
+function daysAgo(days) {
+  return new Date(NOW - days * DAY_MS).toISOString();
+}
+
+function outcome(overrides = {}) {
+  return {
+    pathId: "telegram.bot",
+    ok: true,
+    at: "2026-07-06T00:00:00.000Z",
+    lane: "live-lane-9",
+    commit: "abc1234",
+    counts: { passed: 3, failed: 0, skipped: 0 },
+    ...overrides,
+  };
+}
+
+// --- freshness ----------------------------------------------------------------
+
+test("freshness: never proven reads red", () => {
+  assert.deepEqual(freshness(null, NOW), {
+    state: "red",
+    ageDays: null,
+    label: "never proven",
+  });
+  assert.equal(freshness("", NOW).state, "red");
+  assert.equal(freshness(undefined, NOW).state, "red");
+});
+
+test("freshness: locked band boundaries (green ≤7d, yellow >7d, red >30d)", () => {
+  assert.equal(freshness(daysAgo(0), NOW).state, "green");
+  assert.equal(freshness(daysAgo(FRESH_DAYS), NOW).state, "green");
+  assert.equal(freshness(daysAgo(FRESH_DAYS + 1), NOW).state, "yellow");
+  assert.equal(freshness(daysAgo(STALE_DAYS), NOW).state, "yellow");
+  assert.equal(freshness(daysAgo(STALE_DAYS + 1), NOW).state, "red");
+});
+
+test("freshness: reports age in whole days and clamps future timestamps", () => {
+  assert.equal(freshness(daysAgo(3), NOW).ageDays, 3);
+  assert.equal(freshness(daysAgo(3.9), NOW).ageDays, 3);
+  const future = freshness(daysAgo(-2), NOW);
+  assert.equal(future.state, "green");
+  assert.equal(future.ageDays, 0);
+});
+
+test("freshness: unparseable timestamp throws instead of masking", () => {
+  assert.throws(() => freshness("not-a-date", NOW), /unparseable/);
+});
+
+// --- readLedger ----------------------------------------------------------------
+
+test("readLedger: missing file yields the empty shape", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  assert.deepEqual(readLedger(path), {
+    version: 1,
+    updatedAt: null,
+    entries: {},
+  });
+});
+
+test("readLedger: malformed ledger throws (committed evidence, not a cache)", () => {
+  const dir = mkdtempSync(join(tmpdir(), "hitl-ledger-"));
+  const path = join(dir, "ledger.json");
+  writeFileSync(path, JSON.stringify({ entries: null }), "utf8");
+  assert.throws(() => readLedger(path), /malformed ledger/);
+  writeFileSync(path, JSON.stringify([1, 2]), "utf8");
+  assert.throws(() => readLedger(path), /malformed ledger/);
+});
+
+test("readLedger: committed entries are schema-checked and secret-shaped fields are rejected", () => {
+  const dir = mkdtempSync(join(tmpdir(), "hitl-ledger-"));
+  const path = join(dir, "ledger.json");
+  recordOutcome(outcome(), path);
+  const parsed = JSON.parse(readFileSync(path, "utf8"));
+
+  parsed.entries["telegram.bot"].token = "xoxb-secret-value";
+  writeFileSync(path, JSON.stringify(parsed), "utf8");
+  assert.throws(() => readLedger(path), /secret-shaped field 'token'/);
+
+  delete parsed.entries["telegram.bot"].token;
+  parsed.entries["telegram.bot"].operatorEmail = "owner@example.com";
+  writeFileSync(path, JSON.stringify(parsed), "utf8");
+  assert.throws(() => readLedger(path), /unknown field 'operatorEmail'/);
+
+  delete parsed.entries["telegram.bot"].operatorEmail;
+  parsed.entries["telegram.bot"].counts.retried = 1;
+  writeFileSync(path, JSON.stringify(parsed), "utf8");
+  assert.throws(() => readLedger(path), /unknown field 'retried'/);
+});
+
+// --- recordOutcome upsert semantics ----------------------------------------------
+
+test("recordOutcome: creates the ledger and stamps the success entry", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  const entry = recordOutcome(outcome(), path);
+  assert.deepEqual(entry, {
+    pathId: "telegram.bot",
+    lastSuccessAt: "2026-07-06T00:00:00.000Z",
+    lastRunAt: "2026-07-06T00:00:00.000Z",
+    lane: "live-lane-9",
+    commit: "abc1234",
+    counts: { passed: 3, failed: 0, skipped: 0 },
+  });
+  const onDisk = readLedger(path);
+  assert.equal(onDisk.version, 1);
+  assert.ok(typeof onDisk.updatedAt === "string");
+  assert.deepEqual(onDisk.entries["telegram.bot"], entry);
+});
+
+test("recordOutcome: failed and skipped runs advance lastRunAt but preserve proof", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  recordOutcome(outcome(), path);
+  const afterFail = recordOutcome(
+    outcome({
+      ok: false,
+      at: "2026-07-07T00:00:00.000Z",
+      commit: "def5678",
+      counts: { passed: 1, failed: 2, skipped: 0 },
+    }),
+    path,
+  );
+  assert.equal(afterFail.lastSuccessAt, "2026-07-06T00:00:00.000Z");
+  assert.equal(afterFail.lastRunAt, "2026-07-07T00:00:00.000Z");
+  assert.equal(afterFail.commit, "def5678");
+  assert.deepEqual(afterFail.counts, { passed: 1, failed: 2, skipped: 0 });
+
+  const afterSkip = recordOutcome(
+    outcome({
+      ok: null,
+      at: "2026-07-08T00:00:00.000Z",
+      counts: { passed: 0, failed: 0, skipped: 0 },
+    }),
+    path,
+  );
+  assert.equal(afterSkip.lastSuccessAt, "2026-07-06T00:00:00.000Z");
+  assert.equal(afterSkip.lastRunAt, "2026-07-08T00:00:00.000Z");
+});
+
+test("recordOutcome: a path that never succeeded keeps lastSuccessAt null", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  const entry = recordOutcome(
+    outcome({ pathId: "x.bearer-app", ok: false }),
+    path,
+  );
+  assert.equal(entry.lastSuccessAt, null);
+  assert.equal(freshness(entry.lastSuccessAt, NOW).state, "red");
+});
+
+test("recordOutcomes: batch upsert lands every outcome in one write", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  const next = recordOutcomes(
+    [
+      outcome({ pathId: "slack.bot" }),
+      outcome({ pathId: "discord.bot", ok: null }),
+    ],
+    path,
+  );
+  assert.deepEqual(Object.keys(next.entries), ["discord.bot", "slack.bot"]);
+  assert.deepEqual(readLedger(path).entries, next.entries);
+});
+
+test("recordOutcomes: entries serialize sorted by pathId for stable diffs", () => {
+  const dir = mkdtempSync(join(tmpdir(), "hitl-ledger-"));
+  const path = join(dir, "ledger.json");
+  recordOutcome(outcome({ pathId: "zz.last" }), path);
+  recordOutcome(outcome({ pathId: "aa.first" }), path);
+  const raw = readFileSync(path, "utf8");
+  assert.ok(raw.indexOf('"aa.first"') < raw.indexOf('"zz.last"'));
+  assert.ok(raw.endsWith("}\n"), "file ends with a single trailing newline");
+  // Atomic write: the tmp file is renamed away, never left behind.
+  assert.deepEqual(readdirSync(dir), ["ledger.json"]);
+});
+
+test("recordOutcome: rewriting identical outcomes keeps key order byte-stable", () => {
+  const path = join(mkdtempSync(join(tmpdir(), "hitl-ledger-")), "ledger.json");
+  recordOutcome(outcome({ pathId: "b.path" }), path);
+  recordOutcome(outcome({ pathId: "a.path" }), path);
+  const first = readFileSync(path, "utf8");
+  recordOutcome(outcome({ pathId: "a.path" }), path);
+  const second = readFileSync(path, "utf8");
+  const stripUpdatedAt = (text) => text.replace(/"updatedAt": "[^"]+"/, "");
+  assert.equal(stripUpdatedAt(first), stripUpdatedAt(second));
+});
+
+// --- outcome validation -----------------------------------------------------------
+
+test("recordOutcome: rejects malformed outcomes before touching the file", () => {
+  const dir = mkdtempSync(join(tmpdir(), "hitl-ledger-"));
+  const path = join(dir, "ledger.json");
+  assert.throws(() => recordOutcome(outcome({ pathId: "" }), path), /pathId/);
+  assert.throws(() => recordOutcome(outcome({ ok: "yes" }), path), /ok must/);
+  assert.throws(() => recordOutcome(outcome({ at: "garbage" }), path), /ISO/);
+  assert.throws(() => recordOutcome(outcome({ lane: 7 }), path), /lane/);
+  assert.throws(
+    () => recordOutcome(outcome({ counts: { passed: 1, failed: 0 } }), path),
+    /counts\.skipped/,
+  );
+  assert.throws(
+    () =>
+      recordOutcome(
+        outcome({ counts: { passed: -1, failed: 0, skipped: 0 } }),
+        path,
+      ),
+    /counts\.passed/,
+  );
+  assert.throws(
+    () => recordOutcome(outcome({ token: "xoxb-secret-value" }), path),
+    /secret-shaped field 'token'/,
+  );
+  assert.throws(
+    () => recordOutcome(outcome({ notes: "operator said it passed" }), path),
+    /unknown field 'notes'/,
+  );
+  assert.throws(
+    () =>
+      recordOutcome(
+        outcome({ counts: { passed: 1, failed: 0, skipped: 0, retried: 1 } }),
+        path,
+      ),
+    /unknown field 'retried'/,
+  );
+  assert.deepEqual(readdirSync(dir), [], "no file created on rejected input");
+});

scripts/lifeops/run-11632-live-lanes.mjs

@@ -0,0 +1,553 @@
+#!/usr/bin/env node
+/**
+ * Per-auth-path live-lane driver for the LifeOps HITL validation run (#11632).
+ *
+ * Env resolution is the layered load shared with the HITL dashboard
+ * (env-layers.mjs: process.env > repo .env > ~/.eliza/.env), hydrated into
+ * process.env once at startup so readiness
+ * checks here and the spawned suites see exactly what the dashboard rows show.
+ * Env var NAMES are printed for readiness; env var VALUES are never logged.
+ *
+ * Each lane is one connector auth path (a CONNECTOR_PATHS id, or a lifeops.*
+ * pseudo-path for the keyless/model-gated LifeOps suites) bound to the live
+ * vitest suite that proves it. Gating is per path: the path's own
+ * requiredAll/requiredAny env names plus its declarative availability spec
+ * (checkAvailability). An unsatisfied path prints `SKIP <pathId>: <reason>`
+ * and is recorded — it is never a failure. `--dry-run` extends the readiness
+ * view to every registry path, so suiteless paths (probe-only, covered by the
+ * dashboard) are visible instead of silently absent.
+ *
+ * Every lane completion (any status) upserts the committed evidence ledger
+ * docs/testing/hitl-ledger.json ({pathId, lastRunAt, lastSuccessAt, lane,
+ * commit, counts} — hitl-ledger.mjs writes atomically with stable key order)
+ * and then commits the ledger: `chore(hitl): ledger — <pathId> <status>`.
+ * `lastSuccessAt` only advances on live-proven lanes: exit 0 with zero
+ * skipped tests in the vitest summary — skipped live describes downgrade to
+ * `ran-with-skips`, a nonzero exit records `failed`. `--status` renders the
+ * ledger freshness table (green ≤7d, yellow >7d, red >30d or never).
+ *
+ * Lane stdout+stderr tee to the exact filenames the collector's
+ * `existingEvidence` parsers grep — owner-agent-permission-matrix.txt,
+ * plugin-google-live.txt, plugin-x-live.txt under
+ * reports/lifeops-live-validation/11632-status/ — and the remaining lanes log
+ * into a datestamped session dir next to them, alongside summary.json.
+ *
+ * Usage:
+ *   node scripts/lifeops/run-11632-live-lanes.mjs            # all ready lanes
+ *   node scripts/lifeops/run-11632-live-lanes.mjs --dry-run  # per-path readiness only
+ *   node scripts/lifeops/run-11632-live-lanes.mjs --lane=1   # one lane
+ *   node scripts/lifeops/run-11632-live-lanes.mjs --status   # ledger freshness table
+ */
+import { spawn, spawnSync } from "node:child_process";
+import {
+  createWriteStream,
+  mkdirSync,
+  readFileSync,
+  writeFileSync,
+} from "node:fs";
+import { dirname, join, relative, resolve } from "node:path";
+import process from "node:process";
+import { fileURLToPath } from "node:url";
+import { CONNECTOR_GROUPS } from "./collect-11632-live-validation-status.mjs";
+import { CONNECTOR_PATHS, checkAvailability } from "./connector-paths.mjs";
+import { applyLayeredEnvToProcess } from "./env-layers.mjs";
+import {
+  freshness,
+  LEDGER_PATH,
+  readLedger,
+  recordOutcome,
+} from "./hitl-ledger.mjs";
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+const ROOT = resolve(__dirname, "../..");
+const STATUS_DIR = join(ROOT, "reports/lifeops-live-validation/11632-status");
+const COLLECTOR = join(__dirname, "collect-11632-live-validation-status.mjs");
+
+const PA_LIVE_E2E_FILES = [
+  "plugins/plugin-personal-assistant/test/assistant-user-journeys.live.e2e.test.ts",
+  "plugins/plugin-personal-assistant/test/lifeops-chat.live.e2e.test.ts",
+  "plugins/plugin-personal-assistant/test/lifeops-gmail-chat.live.e2e.test.ts",
+  "plugins/plugin-personal-assistant/test/lifeops-memory.live.e2e.test.ts",
+];
+
+const MODEL_REQUIRED_ANY = CONNECTOR_GROUPS.find(
+  (group) => group.id === "model",
+).requiredAny;
+
+/**
+ * LifeOps suites are auth-path rows too, but they are not connector paths:
+ * the permission matrix is deliberately credential-free, and the two live
+ * model lanes gate on "any live model provider" rather than one key. They
+ * live in the ledger under the lifeops.* family with the same entry shape.
+ */
+const LIFEOPS_PSEUDO_PATHS = [
+  {
+    id: "lifeops.permission-matrix",
+    label: "Owner/agent permission matrix (keyless)",
+    requiredAll: [],
+    requiredAny: [],
+    availability: { type: "always" },
+  },
+  {
+    id: "lifeops.pa-background-real",
+    label: "Personal-assistant background loop (live model)",
+    requiredAll: [],
+    requiredAny: MODEL_REQUIRED_ANY,
+    availability: { type: "always" },
+  },
+  {
+    id: "lifeops.pa-live-e2e",
+    label: "Personal-assistant live e2e journeys (live model)",
+    requiredAll: [],
+    requiredAny: MODEL_REQUIRED_ANY,
+    availability: { type: "always" },
+  },
+];
+
+const PATH_SPECS = new Map(
+  [...CONNECTOR_PATHS, ...LIFEOPS_PSEUDO_PATHS].map((path) => [path.id, path]),
+);
+
+/**
+ * Lane order mirrors the collector's `nextCommands`: keyless proof first, then
+ * connector-scoped live suites, then the model-gated LifeOps lanes. Each lane
+ * names the auth path it proves; gates come from that path's registry entry,
+ * so telegram.bot-style additions are one suite mapping here, not a new gate
+ * system. Lane 3's plugin-x tests call dotenv.config() against the plugin cwd
+ * with override:false, so the env injected here wins over any plugin-local
+ * .env. Lane 5 runs the shared live-e2e vitest lane; its include globs are
+ * derived from the resolved eliza workspace root
+ * (packages/test/vitest/e2e.config.ts), so the same command collects the PA
+ * live e2e files from both the flat elizaOS checkout and the nested `eliza/`
+ * consumer layout.
+ */
+const LANES = [
+  {
+    n: 1,
+    pathId: "lifeops.permission-matrix",
+    env: { LIFEOPS_PERMISSION_MATRIX: "1" },
+    command: [
+      "bunx",
+      "vitest",
+      "run",
+      "--config",
+      "packages/test/vitest/integration.config.ts",
+      "plugins/plugin-personal-assistant/test/owner-agent-permission-matrix.integration.test.ts",
+    ],
+    logPath: () => join(STATUS_DIR, "owner-agent-permission-matrix.txt"),
+  },
+  {
+    n: 2,
+    pathId: "google.oauth-owner",
+    env: { TEST_LANE: "post-merge", ELIZA_LIVE_TEST: "1" },
+    command: ["bun", "run", "--cwd", "plugins/plugin-google", "test"],
+    logPath: () => join(STATUS_DIR, "plugin-google-live.txt"),
+  },
+  {
+    n: 3,
+    pathId: "x.oauth1-user",
+    env: { TEST_LANE: "post-merge", ELIZA_LIVE_TEST: "1" },
+    command: ["bun", "run", "--cwd", "plugins/plugin-x", "test"],
+    logPath: () => join(STATUS_DIR, "plugin-x-live.txt"),
+  },
+  {
+    n: 4,
+    pathId: "lifeops.pa-background-real",
+    env: { ELIZA_LIVE_TEST: "1", LIFEOPS_PERMISSION_MATRIX: "1" },
+    command: [
+      "bun",
+      "run",
+      "--cwd",
+      "plugins/plugin-personal-assistant",
+      "test:background-real",
+    ],
+    logPath: (sessionDir) => join(sessionDir, "pa-background-real.txt"),
+  },
+  {
+    n: 5,
+    pathId: "lifeops.pa-live-e2e",
+    env: { ELIZA_LIVE_TEST: "1" },
+    command: [
+      "bunx",
+      "vitest",
+      "run",
+      "--config",
+      "packages/test/vitest/live-e2e.config.ts",
+      ...PA_LIVE_E2E_FILES,
+    ],
+    logPath: (sessionDir) => join(sessionDir, "pa-live-e2e.txt"),
+  },
+];
+
+for (const lane of LANES) {
+  if (!PATH_SPECS.has(lane.pathId)) {
+    throw new Error(`lane ${lane.n}: unknown auth path '${lane.pathId}'`);
+  }
+}
+
+function parseArgs(argv) {
+  const usage =
+    "Usage: node scripts/lifeops/run-11632-live-lanes.mjs [--dry-run] [--lane=<n>] [--status]";
+  const args = { dryRun: false, lane: null, status: false };
+  for (const arg of argv) {
+    if (arg === "--dry-run") {
+      args.dryRun = true;
+    } else if (arg === "--status") {
+      args.status = true;
+    } else if (/^--lane=\d+$/.test(arg)) {
+      args.lane = Number(arg.slice("--lane=".length));
+    } else if (arg === "--help") {
+      console.log(usage);
+      process.exit(0);
+    } else {
+      console.error(`Unknown argument: ${arg}`);
+      console.error(usage);
+      process.exit(2);
+    }
+  }
+  if (args.lane !== null && !LANES.some((lane) => lane.n === args.lane)) {
+    console.error(`--lane must be one of 1..${LANES.length}`);
+    process.exit(2);
+  }
+  return args;
+}
+
+function hasEnvName(name) {
+  const value = process.env[name];
+  return typeof value === "string" && value.trim().length > 0;
+}
+
+/**
+ * Per-path gate: availability spec first (machine state), then the path's own
+ * env names against the hydrated layered env. Reasons carry env var NAMES
+ * only, never values.
+ */
+function pathReadiness(spec) {
+  const availability = checkAvailability(spec.availability);
+  if (!availability.available) {
+    return { ready: false, reason: availability.reason };
+  }
+  const requiredAll = spec.requiredAll ?? [];
+  const requiredAny = spec.requiredAny ?? [];
+  const missing = requiredAll.filter((name) => !hasEnvName(name));
+  const anySatisfied = requiredAny.length === 0 || requiredAny.some(hasEnvName);
+  if (missing.length === 0 && anySatisfied)
+    return { ready: true, reason: null };
+  const parts = [];
+  if (missing.length > 0) parts.push(`missing ${missing.join(", ")}`);
+  if (!anySatisfied) parts.push(`one of: ${requiredAny.join("|")}`);
+  return { ready: false, reason: parts.join("; ") };
+}
+
+/** Spawns a command from the repo root, tee-ing stdout+stderr to `logPath`. */
+function runCommand(command, extraEnv, logPath) {
+  mkdirSync(dirname(logPath), { recursive: true });
+  return new Promise((resolvePromise, rejectPromise) => {
+    const logStream = createWriteStream(logPath);
+    const child = spawn(command[0], command.slice(1), {
+      cwd: ROOT,
+      env: { ...process.env, ...extraEnv },
+      stdio: ["ignore", "pipe", "pipe"],
+    });
+    const tee = (chunk) => {
+      process.stdout.write(chunk);
+      logStream.write(chunk);
+    };
+    child.stdout.on("data", tee);
+    child.stderr.on("data", tee);
+    child.on("error", rejectPromise);
+    child.on("close", (code, signal) => {
+      // Flush the log before callers re-read it for skip-count parsing.
+      logStream.end(() => resolvePromise({ code, signal }));
+    });
+  });
+}
+
+function stripAnsi(text) {
+  // biome-ignore lint/suspicious/noControlCharactersInRegex: strips vitest's color codes before count parsing.
+  return text.replace(/\x1b\[[0-9;]*m/g, "");
+}
+
+/**
+ * Sums the `Tests  N passed | M skipped (T)` summary lines of every vitest
+ * invocation in the tee'd log. The skip count is the live-proof gate: a green
+ * exit with skipped live describes is not live proof.
+ */
+function parseVitestCounts(logText) {
+  const counts = { passed: 0, failed: 0, skipped: 0 };
+  const summaryLines =
+    stripAnsi(logText).match(/^[ \t]*Tests[ \t]+[^\n]*$/gm) ?? [];
+  for (const line of summaryLines) {
+    for (const key of Object.keys(counts)) {
+      const match = new RegExp(`(\\d+)\\s+${key}`).exec(line);
+      if (match) counts[key] += Number(match[1]);
+    }
+  }
+  return { ...counts, summaryLines: summaryLines.map((line) => line.trim()) };
+}
+
+function laneOutcome(result, counts) {
+  if (result.code !== 0) {
+    return {
+      status: "failed",
+      skipReason: `exit ${result.code ?? `signal ${result.signal}`}`,
+    };
+  }
+  if (counts.skipped > 0) {
+    return {
+      status: "ran-with-skips",
+      skipReason: `${counts.skipped} skipped test(s) in log — not live proof`,
+    };
+  }
+  return { status: "live-proven", skipReason: null };
+}
+
+// --- committed evidence ledger -------------------------------------------------
+
+function git(args) {
+  return spawnSync("git", args, { cwd: ROOT, encoding: "utf8" });
+}
+
+function shortHead() {
+  const result = git(["rev-parse", "--short", "HEAD"]);
+  if (result.status !== 0) {
+    throw new Error(`git rev-parse --short HEAD failed: ${result.stderr}`);
+  }
+  return result.stdout.trim();
+}
+
+/** lastSuccessAt only advances on live-proven; a skip preserves prior proof. */
+function ledgerOk(status) {
+  if (status === "live-proven") return true;
+  if (status === "skipped") return null;
+  return false;
+}
+
+/**
+ * Stage and commit only the ledger file (pathspec commit), so concurrent
+ * working-tree edits from other agents are never swept into an evidence
+ * commit. Committing evidence-of-run to the repo is the owner's explicit ask.
+ */
+function commitLedger(pathId, status) {
+  const rel = relative(ROOT, LEDGER_PATH);
+  const dirty = git(["status", "--porcelain", "--", rel]);
+  if (dirty.status !== 0) {
+    throw new Error(`git status for ${rel} failed: ${dirty.stderr}`);
+  }
+  if (dirty.stdout.trim().length === 0) return false;
+  const add = git(["add", "--", rel]);
+  if (add.status !== 0) {
+    throw new Error(`git add ${rel} failed: ${add.stderr}`);
+  }
+  const commit = git([
+    "commit",
+    "-m",
+    `chore(hitl): ledger — ${pathId} ${status}`,
+    "--",
+    rel,
+  ]);
+  if (commit.status !== 0) {
+    throw new Error(
+      `git commit of ${rel} failed: ${commit.stdout}${commit.stderr}`,
+    );
+  }
+  return true;
+}
+
+function recordLane(lane, status, counts, headShort) {
+  recordOutcome({
+    pathId: lane.pathId,
+    ok: ledgerOk(status),
+    at: new Date().toISOString(),
+    lane: `live-lane-${lane.n}`,
+    commit: headShort,
+    counts: {
+      passed: counts.passed,
+      failed: counts.failed,
+      skipped: counts.skipped,
+    },
+  });
+  const committed = commitLedger(lane.pathId, status);
+  console.log(
+    `[11632-lanes] ledger ${committed ? "committed" : "unchanged"} — ${lane.pathId} ${status}`,
+  );
+}
+
+// --- terminal freshness table ----------------------------------------------------
+
+const STATE_COLORS = { green: "\x1b[32m", yellow: "\x1b[33m", red: "\x1b[31m" };
+
+function printStatusTable() {
+  const ledger = readLedger();
+  const ids = [
+    ...new Set([
+      ...LANES.map((lane) => lane.pathId),
+      ...CONNECTOR_PATHS.map((path) => path.id),
+      ...Object.keys(ledger.entries),
+    ]),
+  ].sort();
+  const paint = (state, text) =>
+    process.stdout.isTTY ? `${STATE_COLORS[state]}${text}\x1b[0m` : text;
+  console.log(
+    "[11632-lanes] HITL freshness from ledger lastSuccessAt — green ≤7d, yellow >7d, red >30d or never",
+  );
+  console.log(
+    `[11632-lanes] ledger ${relative(ROOT, LEDGER_PATH)} (updated ${ledger.updatedAt ?? "never"})`,
+  );
+  for (const id of ids) {
+    const entry = ledger.entries[id];
+    const fresh = freshness(entry ? entry.lastSuccessAt : null);
+    const counts = entry
+      ? `${entry.counts.passed}p/${entry.counts.failed}f/${entry.counts.skipped}s`
+      : "-";
+    const detail = entry
+      ? `lastRun ${entry.lastRunAt}  ${entry.lane}@${entry.commit}  ${counts}`
+      : "no ledger entry";
+    console.log(
+      `${paint(fresh.state, fresh.state.padEnd(6))} ${id.padEnd(28)} ${fresh.label.padEnd(20)} ${detail}`,
+    );
+  }
+}
+
+// --- entrypoint --------------------------------------------------------------------
+
+const args = parseArgs(process.argv.slice(2));
+
+if (args.status) {
+  printStatusTable();
+  process.exit(0);
+}
+
+const layered = applyLayeredEnvToProcess();
+for (const layer of layered.layers) {
+  console.log(
+    `[11632-lanes] env layer ${layer.source.padEnd(8)} ${layer.path ?? "(process.env)"}${layer.exists ? "" : " (absent)"}`,
+  );
+}
+
+if (args.dryRun) {
+  console.log(
+    "[11632-lanes] dry run — per-auth-path readiness from the layered env (nothing executed):",
+  );
+  for (const lane of LANES) {
+    const { ready, reason } = pathReadiness(PATH_SPECS.get(lane.pathId));
+    if (ready) {
+      console.log(`READY lane ${lane.n} ${lane.pathId}`);
+    } else {
+      console.log(`SKIP ${lane.pathId}: ${reason}`);
+    }
+  }
+  const lanePathIds = new Set(LANES.map((lane) => lane.pathId));
+  console.log(
+    "[11632-lanes] registry paths without a wired live suite (probe via the HITL dashboard):",
+  );
+  for (const path of CONNECTOR_PATHS) {
+    if (lanePathIds.has(path.id)) continue;
+    const { ready, reason } = pathReadiness(path);
+    console.log(
+      `SKIP ${path.id}: ${ready ? "env satisfied — no live suite wired for this path" : reason}`,
+    );
+  }
+  process.exit(0);
+}
+
+const selected =
+  args.lane === null ? LANES : LANES.filter((lane) => lane.n === args.lane);
+const sessionStamp = new Date()
+  .toISOString()
+  .replace(/:/g, "-")
+  .replace(/\..+$/, "Z");
+const sessionDir = join(STATUS_DIR, `session-${sessionStamp}`);
+mkdirSync(sessionDir, { recursive: true });
+
+// Captured once, before any ledger commits advance HEAD: ledger entries must
+// point at the code the lane ran against, not at a prior lane's evidence commit.
+const headShort = shortHead();
+
+console.log("[11632-lanes] collector pre-pass");
+const collectorResult = await runCommand(
+  ["node", COLLECTOR],
+  {},
+  join(sessionDir, "collector-prepass.txt"),
+);
+if (collectorResult.code !== 0) {
+  console.error(
+    `[11632-lanes] collector pre-pass failed (exit ${collectorResult.code}); aborting`,
+  );
+  process.exit(1);
+}
+
+const summary = [];
+let anyFailed = false;
+for (const lane of selected) {
+  const { ready, reason } = pathReadiness(PATH_SPECS.get(lane.pathId));
+  if (!ready) {
+    console.log(`SKIP ${lane.pathId}: ${reason}`);
+    recordLane(
+      lane,
+      "skipped",
+      { passed: 0, failed: 0, skipped: 0 },
+      headShort,
+    );
+    summary.push({
+      pathId: lane.pathId,
+      laneNumber: lane.n,
+      status: "skipped",
+      skipReason: reason,
+      logPath: null,
+    });
+    continue;
+  }
+
+  const logPath = lane.logPath(sessionDir);
+  console.log(
+    `[11632-lanes] lane ${lane.n} ${lane.pathId}: ${lane.command.join(" ")} → ${relative(ROOT, logPath)}`,
+  );
+  const result = await runCommand(lane.command, lane.env, logPath);
+  const counts = parseVitestCounts(readFileSync(logPath, "utf8"));
+  const { status, skipReason } = laneOutcome(result, counts);
+  if (status === "failed") anyFailed = true;
+  console.log(
+    `[11632-lanes] lane ${lane.n} ${lane.pathId}: ${status}${skipReason ? ` (${skipReason})` : ""} — ${counts.passed} passed | ${counts.failed} failed | ${counts.skipped} skipped`,
+  );
+  recordLane(lane, status, counts, headShort);
+  summary.push({
+    pathId: lane.pathId,
+    laneNumber: lane.n,
+    status,
+    skipReason,
+    exitCode: result.code,
+    counts: {
+      passed: counts.passed,
+      failed: counts.failed,
+      skipped: counts.skipped,
+    },
+    vitestSummaryLines: counts.summaryLines,
+    logPath: relative(ROOT, logPath),
+  });
+}
+
+const summaryPath = join(sessionDir, "summary.json");
+writeFileSync(
+  summaryPath,
+  `${JSON.stringify(
+    {
+      issue: 11632,
+      generatedAt: new Date().toISOString(),
+      sessionDir: relative(ROOT, sessionDir),
+      commit: headShort,
+      lanes: summary,
+    },
+    null,
+    2,
+  )}\n`,
+  "utf8",
+);
+
+console.log(`[11632-lanes] summary → ${relative(ROOT, summaryPath)}`);
+for (const entry of summary) {
+  console.log(
+    `  ${entry.status.padEnd(14)} lane ${entry.laneNumber} ${entry.pathId}${entry.skipReason ? ` — ${entry.skipReason}` : ""}`,
+  );
+}
+process.exit(anyFailed ? 1 : 0);

scripts/personality-bench-bridge.d.mts

@@ -0,0 +1,25 @@
+export interface PersonalityScenarioLike {
+  personalityExpect?: {
+    bucket?: string;
+    directiveTurn?: number;
+    checkTurns?: number[];
+    options?: Record<string, unknown>;
+    judgeKwargs?: Record<string, unknown>;
+  };
+}
+
+export interface BridgedPersonalityExpect {
+  bucket: unknown;
+  directiveTurn: number;
+  checkTurns: number[];
+  options: Record<string, unknown>;
+}
+
+export const STYLE_KEY_TO_STYLE: Readonly<Record<string, string>>;
+export const TRAIT_KEY_TO_OPTIONS: Readonly<
+  Record<string, Record<string, unknown>>
+>;
+export const SCOPE_VARIANT_TO_MODE: Readonly<Record<string, string>>;
+export function bridgePersonalityExpect(
+  scenario: PersonalityScenarioLike,
+): BridgedPersonalityExpect;

scripts/personality-bench-bridge.mjs

@@ -0,0 +1,146 @@
+/**
+ * Translation tables and turn-index helpers bridging personality-bench scenario
+ * keys to the runtime's style / trait / scope-mode configuration. Maps each
+ * scenario's style key, trait key, and scope-variant name onto the concrete
+ * style names, trait options, and enforcement modes the bench applies and then
+ * scores against assistant turns.
+ */
+export const STYLE_KEY_TO_STYLE = Object.freeze({
+  no_hedging: "no-hedging",
+  haiku: "haiku",
+  pirate: "pirate",
+  terse_one_sentence: "terse",
+  all_lowercase: "all_lowercase",
+  limerick: "limerick",
+  shakespearean: "shakespearean",
+  second_person_only: "second_person_only",
+});
+
+export const TRAIT_KEY_TO_OPTIONS = Object.freeze({
+  no_emojis: { trait: "no-emojis" },
+  no_buddy_friend: {
+    trait: "no-buddy",
+    forbiddenPhrases: ["buddy", "friend"],
+  },
+  code_blocks_only: { trait: "wants-code-blocks" },
+  first_name_only: { trait: "first_name_only" },
+  metric_units: { trait: "metric_units" },
+  prefers_short: { trait: "prefers_short" },
+});
+
+export const SCOPE_VARIANT_TO_MODE = Object.freeze({
+  user_tries_global_should_refuse: "refuse",
+  user_overrides_global: "user_overrides_global",
+  user_wins_conflict: "user_wins_conflict",
+  global_applies_to_admin_only: "global_applies_to_admin_only",
+  global_applies_to_all: "global_applies_to_all",
+  persistence: "persistence",
+  isolation: "isolation",
+});
+
+function assistantTurnFor(userTurnIndex) {
+  return userTurnIndex * 2 + 2;
+}
+
+function arrayOfAssistantTurns(values) {
+  return Array.isArray(values)
+    ? values.filter(Number.isInteger).map(assistantTurnFor)
+    : [];
+}
+
+function objectValue(value) {
+  return value && typeof value === "object" ? value : {};
+}
+
+function copyString(source, target, sourceKey, targetKey = sourceKey) {
+  const value = source[sourceKey];
+  if (typeof value === "string" && value.length > 0) {
+    target[targetKey] = value;
+  }
+}
+
+export function bridgePersonalityExpect(scenario) {
+  const sourceExpect = objectValue(scenario?.personalityExpect);
+  const judgeKwargs = objectValue(sourceExpect.judgeKwargs);
+  const bucket = sourceExpect.bucket;
+  const options = { ...objectValue(sourceExpect.options) };
+  const bridged = {
+    bucket,
+    directiveTurn:
+      Number.isInteger(sourceExpect.directiveTurn) &&
+      sourceExpect.directiveTurn > 0
+        ? sourceExpect.directiveTurn
+        : 1,
+    checkTurns: Array.isArray(sourceExpect.checkTurns)
+      ? [...sourceExpect.checkTurns]
+      : [],
+    options,
+  };
+
+  switch (bucket) {
+    case "hold_style": {
+      const style = STYLE_KEY_TO_STYLE[judgeKwargs.styleKey];
+      if (style) {
+        options.style = style;
+      }
+      if (style === "terse") {
+        options.maxTokens = options.maxTokens ?? 80;
+      }
+      if (Number.isInteger(judgeKwargs.instructionTurnIndex)) {
+        bridged.directiveTurn = assistantTurnFor(
+          judgeKwargs.instructionTurnIndex,
+        );
+      }
+      bridged.checkTurns = arrayOfAssistantTurns(judgeKwargs.probeTurnIndices);
+      break;
+    }
+
+    case "note_trait_unrelated": {
+      Object.assign(options, TRAIT_KEY_TO_OPTIONS[judgeKwargs.traitKey] ?? {});
+      copyString(judgeKwargs, options, "lastName");
+      copyString(judgeKwargs, options, "last_name", "lastName");
+      if (Number.isInteger(judgeKwargs.traitMentionTurnIndex)) {
+        bridged.directiveTurn = assistantTurnFor(
+          judgeKwargs.traitMentionTurnIndex,
+        );
+      }
+      bridged.checkTurns = arrayOfAssistantTurns(
+        judgeKwargs.traitCheckTurnIndices,
+      );
+      break;
+    }
+
+    case "shut_up": {
+      if (Number.isInteger(judgeKwargs.instructionTurnIndex)) {
+        bridged.directiveTurn = assistantTurnFor(
+          judgeKwargs.instructionTurnIndex,
+        );
+      }
+      const silentTurns = arrayOfAssistantTurns(judgeKwargs.silentTurnIndices);
+      if (
+        silentTurns.length === 0 &&
+        judgeKwargs.allowOneLineAcknowledgmentOnInstructionTurn === true
+      ) {
+        options.len1AckMode = true;
+        bridged.checkTurns = [bridged.directiveTurn];
+      } else {
+        bridged.checkTurns = silentTurns;
+      }
+      break;
+    }
+
+    case "scope_global_vs_user": {
+      const mode =
+        SCOPE_VARIANT_TO_MODE[judgeKwargs.scopeVariant] ??
+        SCOPE_VARIANT_TO_MODE[judgeKwargs.variantKey];
+      if (mode) {
+        options.mode = mode;
+      }
+      copyString(judgeKwargs, options, "scopeVariant");
+      bridged.checkTurns = arrayOfAssistantTurns(judgeKwargs.checkTurnIndices);
+      break;
+    }
+  }
+
+  return bridged;
+}

scripts/pr-evidence.mjs

@@ -0,0 +1,273 @@
+#!/usr/bin/env node
+/**
+ * One-command PR evidence attach for agents and humans: uploads media/log
+ * artifacts to the shared `pr-evidence` release (the CLI attachment path for
+ * headless agents that cannot drag-and-drop), rewrites the PR body's
+ * `evidence-row:*` rows to reference the uploaded assets, and re-runs the
+ * local evidence gate against the resulting body so the author knows the CI
+ * check will pass BEFORE pushing the edit. Exists because the manual loop
+ * (upload → copy URLs → hand-edit eight rows → wait for CI) is exactly the
+ * friction that made agents skip evidence.
+ *
+ *   node scripts/pr-evidence.mjs attach <pr> <files...>        upload + print URLs
+ *   node scripts/pr-evidence.mjs rows <pr> --row id=<file|url|"N/A - reason"> …
+ *                                                              patch body rows + verify
+ *   node scripts/pr-evidence.mjs verify <pr>                   run the gate locally
+ *
+ * `attach` prefixes every asset `<pr>-` so one release serves all PRs, and
+ * skips re-uploading an asset that already exists with identical bytes.
+ * `rows` accepts a local file (uploaded automatically), an existing URL, or an
+ * `N/A - reason` string per row; rows not named are left untouched. Every
+ * mutation is previewed and the gate verdict printed; `--dry-run` stops before
+ * editing the PR.
+ */
+
+import { execFileSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import { existsSync, readFileSync, writeFileSync } from "node:fs";
+import { basename } from "node:path";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { pathToFileURL } from "node:url";
+
+const RELEASE_TAG = "pr-evidence";
+const ASSET_BASE = `https://github.com/elizaOS/eliza/releases/download/${RELEASE_TAG}`;
+const ROW_IDS = [
+  "before-screenshots",
+  "after-screenshots",
+  "walkthrough-video",
+  "backend-logs",
+  "frontend-logs",
+  "llm-trajectory",
+  "domain-artifacts",
+  "ocr-review",
+];
+
+function gh(args, opts = {}) {
+  return execFileSync("gh", args, { encoding: "utf8", ...opts });
+}
+
+function fail(message) {
+  console.error(`pr-evidence: ${message}`);
+  process.exit(1);
+}
+
+function usage() {
+  console.log(`Usage:
+  node scripts/pr-evidence.mjs attach <pr> <files...>
+      Upload files to the '${RELEASE_TAG}' release as <pr>-<name> and print
+      the asset URLs ready to paste into evidence rows.
+
+  node scripts/pr-evidence.mjs rows <pr> [--dry-run] --row <id>=<value> [...]
+      Patch the PR body's evidence rows and verify the gate locally.
+      <id>    one of: ${ROW_IDS.join(", ")}
+      <value> a local file path (auto-uploaded), an https URL, or an
+              'N/A - <reason>' string.
+
+  node scripts/pr-evidence.mjs verify <pr>
+      Fetch the PR body/labels/files and run the local evidence gate exactly
+      as CI does.
+
+Assets land on: ${ASSET_BASE}/<pr>-<filename>
+Worked example of a fully evidenced PR: https://github.com/elizaOS/eliza/pull/15171`);
+}
+
+/** Existing release assets, name → { sha? } (sha lazily unavailable via API — name match only). */
+function existingAssetNames() {
+  const out = gh([
+    "release",
+    "view",
+    RELEASE_TAG,
+    "--json",
+    "assets",
+    "-q",
+    ".assets[].name",
+  ]);
+  return new Set(out.split("\n").filter(Boolean));
+}
+
+/**
+ * Upload local files as `<pr>-<basename>`; returns name → URL. A name
+ * collision uploads with `--clobber` only when the local bytes differ from a
+ * previous upload this run cannot verify — so collide loudly instead:
+ * existing names are reused as-is (assets referenced by open PRs are
+ * immutable by policy), and a genuinely different file must be renamed.
+ */
+function attach(pr, files) {
+  if (files.length === 0) fail("attach needs at least one file");
+  const existing = existingAssetNames();
+  const urls = new Map();
+  const toUpload = [];
+  const staged = [];
+  for (const file of files) {
+    if (!existsSync(file)) fail(`no such file: ${file}`);
+    // GitHub rejects zero-byte release assets with an opaque 400
+    // (Bad Content-Length) that aborts the whole batch — fail per-file with
+    // the actual reason instead. An empty artifact is never real evidence.
+    if (readFileSync(file).length === 0) {
+      fail(`refusing to upload empty file (0 bytes): ${file}`);
+    }
+    const name = `${pr}-${basename(file).replace(new RegExp(`^${pr}-`), "")}`;
+    const url = `${ASSET_BASE}/${name}`;
+    urls.set(name, url);
+    if (existing.has(name)) {
+      console.log(`  = ${name} (already uploaded, reusing)`);
+      continue;
+    }
+    // gh names the asset after the file, so stage a correctly-named copy.
+    const stagedPath = join(tmpdir(), name);
+    writeFileSync(stagedPath, readFileSync(file));
+    staged.push(stagedPath);
+    toUpload.push(stagedPath);
+  }
+  if (toUpload.length > 0) {
+    gh(["release", "upload", RELEASE_TAG, ...toUpload], { stdio: "inherit" });
+  }
+  for (const [name, url] of urls) console.log(`  ${url}`);
+  return urls;
+}
+
+function isMediaName(name) {
+  return /\.(png|jpe?g|gif|webp|mp4|mov|webm)$/i.test(name);
+}
+
+/** Render the replacement row line for an id + resolved value. */
+function renderRow(id, value) {
+  if (/^N\/?A\s*[-:]/i.test(value)) return `- [ ] ${value}`;
+  if (isMediaName(value) && /^https?:/i.test(value)) {
+    // Embed images inline; videos/GIFs render from the bare URL on GitHub.
+    return /\.(png|jpe?g|gif|webp)$/i.test(value)
+      ? `- [x] ![${id}](${value})`
+      : `- [x] ${value}`;
+  }
+  if (/^https?:/i.test(value)) return `- [x] [${basename(value)}](${value})`;
+  fail(`row ${id}: value must be a file, URL, or 'N/A - <reason>' (got: ${value})`);
+}
+
+/** Replace the block after `<!-- evidence-row:<id> -->` with `line`. */
+function patchRow(body, id, line) {
+  const marker = `<!-- evidence-row:${id} -->`;
+  const at = body.indexOf(marker);
+  if (at === -1) {
+    // Row marker absent (old template) — append a fresh marker + row.
+    return `${body.trimEnd()}\n\n${marker}\n${line}\n`;
+  }
+  const afterMarker = at + marker.length;
+  const rest = body.slice(afterMarker);
+  // The row block ends at the next blank line, heading, or marker.
+  const end = rest.search(/\n\s*\n|\n#|\n<!-- evidence-row:/);
+  const blockEnd = end === -1 ? rest.length : end;
+  return body.slice(0, afterMarker) + "\n" + line + body.slice(afterMarker + blockEnd);
+}
+
+async function runGate(pr, body) {
+  const { evaluatePrEvidence } = await import(
+    pathToFileURL(join(import.meta.dirname, "check-pr-evidence.mjs")).href
+  );
+  const labels = gh([
+    "pr",
+    "view",
+    String(pr),
+    "--json",
+    "labels",
+    "-q",
+    "[.labels[].name]|join(\",\")",
+  ]).trim();
+  // `gh pr diff` 406s past GitHub's 300-file diff cap (hit live on the 350-file
+  // #15291); the paginated files API has no such limit.
+  const changedFiles = gh([
+    "api",
+    `repos/{owner}/{repo}/pulls/${pr}/files`,
+    "--paginate",
+    "-q",
+    ".[].filename",
+  ])
+    .split("\n")
+    .filter(Boolean);
+  const addedFiles = gh([
+    "api",
+    `repos/{owner}/{repo}/pulls/${pr}/files`,
+    "--paginate",
+    "-q",
+    '.[] | select(.status=="added") | .filename',
+  ])
+    .split("\n")
+    .filter(Boolean);
+  const { ok, findings } = evaluatePrEvidence(body, undefined, {
+    labels,
+    changedFiles,
+    addedFiles,
+  });
+  for (const f of findings) {
+    console.log(`  [${f.status === "ok" ? "ok  " : "FAIL"}] ${f.id}: ${f.status}`);
+  }
+  return ok;
+}
+
+async function rows(pr, args) {
+  const dryRun = args.includes("--dry-run");
+  const rowArgs = [];
+  for (let i = 0; i < args.length; i += 1) {
+    if (args[i] === "--row") {
+      const spec = args[i + 1] ?? "";
+      const eq = spec.indexOf("=");
+      if (eq === -1) fail(`--row needs <id>=<value>, got: ${spec}`);
+      const id = spec.slice(0, eq);
+      if (!ROW_IDS.includes(id)) fail(`unknown row id '${id}' (valid: ${ROW_IDS.join(", ")})`);
+      rowArgs.push({ id, value: spec.slice(eq + 1) });
+      i += 1;
+    }
+  }
+  if (rowArgs.length === 0) fail("rows needs at least one --row <id>=<value>");
+
+  // Resolve local files to uploaded asset URLs first.
+  const localFiles = rowArgs.filter((r) => !/^https?:/i.test(r.value) && !/^N\/?A\s*[-:]/i.test(r.value));
+  if (localFiles.length > 0) {
+    console.log(`Uploading ${localFiles.length} local file(s) to '${RELEASE_TAG}':`);
+    const urls = attach(pr, localFiles.map((r) => r.value));
+    for (const r of localFiles) {
+      const name = `${pr}-${basename(r.value).replace(new RegExp(`^${pr}-`), "")}`;
+      r.value = urls.get(name);
+    }
+  }
+
+  let body = gh(["pr", "view", String(pr), "--json", "body", "-q", ".body"]);
+  for (const { id, value } of rowArgs) {
+    body = patchRow(body, id, renderRow(id, value));
+  }
+
+  console.log("\nLocal gate verdict on the new body:");
+  const ok = await runGate(pr, body);
+  if (dryRun) {
+    console.log(`\n--dry-run: PR #${pr} not edited. Gate ${ok ? "would PASS" : "would FAIL"}.`);
+    return;
+  }
+  const bodyFile = join(tmpdir(), `pr-${pr}-body.md`);
+  writeFileSync(bodyFile, body);
+  gh(["pr", "edit", String(pr), "--body-file", bodyFile], { stdio: "inherit" });
+  console.log(`\nPR #${pr} updated. Gate ${ok ? "PASSES" : "still FAILS — fix the rows above"}.`);
+  if (!ok) process.exit(1);
+}
+
+async function verify(pr) {
+  const body = gh(["pr", "view", String(pr), "--json", "body", "-q", ".body"]);
+  const ok = await runGate(pr, body);
+  console.log(ok ? "\nEvidence gate PASSES." : "\nEvidence gate FAILS.");
+  if (!ok) process.exit(1);
+}
+
+async function main() {
+  const [cmd, prArg, ...rest] = process.argv.slice(2);
+  if (!cmd || cmd === "--help" || cmd === "-h") {
+    usage();
+    return;
+  }
+  const pr = Number(prArg);
+  if (!Number.isInteger(pr) || pr <= 0) fail(`invalid PR number: ${prArg}`);
+  if (cmd === "attach") attach(pr, rest);
+  else if (cmd === "rows") await rows(pr, rest);
+  else if (cmd === "verify") await verify(pr);
+  else fail(`unknown command: ${cmd}`);
+}
+
+await main();

scripts/release-set-public-access.mjs

@@ -0,0 +1,37 @@
+/**
+ * Release helper that stamps publishConfig.access="public" onto public @elizaos packages.
+ *
+ * Scoped packages default to restricted access, which fails on the free @elizaos
+ * npm organization during `lerna publish from-package`. Lerna publishes through
+ * libnpmpublish and honors each package's publishConfig.access rather than the
+ * npmrc access key, so this script sets the per-package field and leaves private
+ * packages untouched.
+ */
+
+import { execSync } from "node:child_process";
+import { readFileSync, writeFileSync } from "node:fs";
+
+const files = execSync("git ls-files '*package.json'")
+  .toString()
+  .trim()
+  .split("\n")
+  .filter(Boolean);
+
+let changed = 0;
+for (const file of files) {
+  if (file.includes("/node_modules/")) continue;
+  let pkg;
+  try {
+    pkg = JSON.parse(readFileSync(file, "utf8"));
+  } catch {
+    continue;
+  }
+  if (!pkg.name || !pkg.name.startsWith("@elizaos/") || pkg.private) continue;
+  if (pkg.publishConfig?.access === "public") continue;
+  pkg.publishConfig = { ...(pkg.publishConfig ?? {}), access: "public" };
+  writeFileSync(file, `${JSON.stringify(pkg, null, 2)}\n`);
+  changed += 1;
+}
+console.log(
+  `[release] set publishConfig.access=public on ${changed} package(s)`,
+);

scripts/rm-path-recursive.mjs

@@ -0,0 +1,34 @@
+#!/usr/bin/env node
+/**
+ * Remove a path recursively. Uses fs.rmSync for reliable deletion on
+ * macOS/APFS under parallel builds (shell rm -rf can sporadically fail with
+ * "Directory not empty" when the tree is huge or files are busy).
+ */
+import { rmSync } from "node:fs";
+import path from "node:path";
+import { setTimeout as delay } from "node:timers/promises";
+
+const rel = process.argv[2];
+if (!rel) {
+  console.error("usage: node packages/scripts/rm-path-recursive.mjs <path>");
+  process.exit(1);
+}
+const target = path.resolve(process.cwd(), rel);
+const retryableCodes = new Set(["EBUSY", "ENOTEMPTY", "EPERM"]);
+
+for (let attempt = 0; attempt < 5; attempt += 1) {
+  try {
+    rmSync(target, { recursive: true, force: true });
+    process.exit(0);
+  } catch (e) {
+    const code = e && typeof e === "object" && "code" in e ? e.code : undefined;
+    if (code === "ENOENT") {
+      process.exit(0);
+    }
+    if (typeof code === "string" && retryableCodes.has(code) && attempt < 4) {
+      await delay(50 * (attempt + 1));
+      continue;
+    }
+    throw e;
+  }
+}

scripts/security/apply-branch-protection.sh

@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+# apply-branch-protection.sh
+#
+# Idempotently applies branch-protection rules to `main` and `develop` on the
+# elizaOS repositories. Required for SOC2 CC8.1 (change management)
+# and CC6.1 (logical access).
+#
+# Required token scopes:
+#   - `admin:org`  (to read team membership for required reviewers)
+#   - `repo`       (to write branch protection)
+# Or a fine-grained PAT with "Administration: write" on the target repos.
+#
+# Usage:
+#   GH_TOKEN=ghp_xxx ./scripts/security/apply-branch-protection.sh
+#   ./scripts/security/apply-branch-protection.sh --repo elizaos/eliza --branches main,develop
+#
+# Pre-flight checks: requires `gh` CLI authenticated with sufficient scopes.
+
+set -euo pipefail
+
+REPOS=("elizaos/eliza")
+BRANCHES=("main" "develop")
+
+# Required status checks. These must match job NAMES (not workflow names) that
+# run on PRs to the protected branch. Update as workflows evolve.
+REQUIRED_CHECKS=(
+  "typecheck"
+  "test"
+  "gitleaks"
+)
+# Optional checks — added if known to exist; CI publish/container scans live
+# in different workflows on different repos, so we keep this list short and
+# tighten per-repo via overrides below if needed.
+OPTIONAL_CHECKS=(
+  "container-scan"
+)
+
+usage() {
+  cat <<EOF
+Usage: $0 [--repo OWNER/NAME] [--branches BR1,BR2] [--dry-run]
+
+Applies branch protection rules to the configured repos and branches.
+Without --repo / --branches, defaults are: ${REPOS[*]} / ${BRANCHES[*]}.
+EOF
+}
+
+DRY_RUN=0
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --repo)     REPOS=("$2"); shift 2 ;;
+    --branches) IFS=',' read -ra BRANCHES <<< "$2"; shift 2 ;;
+    --dry-run)  DRY_RUN=1; shift ;;
+    -h|--help)  usage; exit 0 ;;
+    *) echo "Unknown arg: $1" >&2; usage; exit 2 ;;
+  esac
+done
+
+if ! command -v gh >/dev/null 2>&1; then
+  echo "error: gh CLI not found. Install from https://cli.github.com/" >&2
+  exit 1
+fi
+if ! gh auth status >/dev/null 2>&1; then
+  echo "error: gh not authenticated. Run 'gh auth login' or set GH_TOKEN." >&2
+  exit 1
+fi
+
+# Build the JSON payload. We require code-owner review and at least one
+# approving review. Status checks are required and must be up-to-date. Force
+# pushes and deletions are denied. Signed commits and linear history are
+# required. Conversation resolution required.
+build_payload() {
+  local contexts_json
+  contexts_json=$(printf '"%s",' "${REQUIRED_CHECKS[@]}")
+  contexts_json="[${contexts_json%,}]"
+
+  cat <<JSON
+{
+  "required_status_checks": {
+    "strict": true,
+    "contexts": ${contexts_json}
+  },
+  "enforce_admins": true,
+  "required_pull_request_reviews": {
+    "dismiss_stale_reviews": true,
+    "require_code_owner_reviews": true,
+    "required_approving_review_count": 1,
+    "require_last_push_approval": true
+  },
+  "restrictions": null,
+  "required_linear_history": true,
+  "allow_force_pushes": false,
+  "allow_deletions": false,
+  "required_conversation_resolution": true,
+  "lock_branch": false,
+  "allow_fork_syncing": false,
+  "required_signatures": true
+}
+JSON
+}
+
+apply_one() {
+  local repo="$1" branch="$2"
+  echo "==> ${repo}@${branch}"
+  local payload
+  payload=$(build_payload)
+  if [[ "$DRY_RUN" -eq 1 ]]; then
+    echo "$payload"
+    return 0
+  fi
+  # PUT /repos/{owner}/{repo}/branches/{branch}/protection
+  echo "$payload" \
+    | gh api -X PUT "repos/${repo}/branches/${branch}/protection" \
+        -H "Accept: application/vnd.github+json" \
+        --input - \
+    > /dev/null
+  # Signed commits is a separate endpoint historically; the unified payload
+  # above already includes required_signatures, but enforce explicitly too
+  # so older gh API versions still set it.
+  gh api -X POST "repos/${repo}/branches/${branch}/protection/required_signatures" \
+        -H "Accept: application/vnd.github+json" \
+    > /dev/null 2>&1 || true
+  echo "    ok"
+}
+
+main() {
+  for repo in "${REPOS[@]}"; do
+    for branch in "${BRANCHES[@]}"; do
+      if gh api "repos/${repo}/branches/${branch}" >/dev/null 2>&1; then
+        apply_one "$repo" "$branch"
+      else
+        echo "==> ${repo}@${branch} — branch not found, skipping"
+      fi
+    done
+  done
+
+  cat <<EOF
+
+Done. Verify in the GitHub UI:
+  https://github.com/<owner>/<repo>/settings/branches
+
+Optional checks (not required by this script but recommended once they exist):
+$(printf '  - %s\n' "${OPTIONAL_CHECKS[@]}")
+EOF
+}
+
+main "$@"

scripts/security/audit-workflow-permissions.sh

@@ -0,0 +1,83 @@
+#!/usr/bin/env bash
+# audit-workflow-permissions.sh
+#
+# Adds a top-level `permissions: contents: read` block to every workflow in
+# .github/workflows that lacks one. Per-job permissions overrides remain
+# untouched. This implements the OpenSSF Scorecard "Token-Permissions" check
+# baseline (SOC2 CC6.1 / CC6.8).
+#
+# Usage:
+#   scripts/security/audit-workflow-permissions.sh         # apply
+#   scripts/security/audit-workflow-permissions.sh --check # report only
+#
+# Safe to re-run: skips files that already have a top-level `permissions:`.
+
+set -euo pipefail
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+WORKFLOW_DIR="${REPO_ROOT}/.github/workflows"
+MODE="apply"
+[[ "${1:-}" == "--check" ]] && MODE="check"
+
+[[ -d "$WORKFLOW_DIR" ]] || { echo "No workflow dir at $WORKFLOW_DIR" >&2; exit 1; }
+
+needs_permissions() {
+  local f="$1"
+  # has a top-level `permissions:` line (no leading whitespace)?
+  ! grep -qE '^permissions:' "$f"
+}
+
+inject_permissions() {
+  local f="$1"
+  python3 - "$f" <<'PY'
+import sys, pathlib, re
+p = pathlib.Path(sys.argv[1])
+text = p.read_text()
+lines = text.splitlines(keepends=True)
+# Insert permissions block immediately before the first top-level `jobs:` key.
+# Falling back to the first blank line after `on:` if `jobs:` is not found.
+out = []
+inserted = False
+for i, line in enumerate(lines):
+    if not inserted and re.match(r'^jobs:\s*$', line):
+        out.append("# Default to least privilege. Override per-job where needed.\n")
+        out.append("permissions:\n")
+        out.append("  contents: read\n")
+        out.append("\n")
+        inserted = True
+    out.append(line)
+if not inserted:
+    sys.stderr.write(f"warn: could not find `jobs:` in {p}, skipping\n")
+    sys.exit(0)
+p.write_text("".join(out))
+PY
+}
+
+count_missing=0
+count_fixed=0
+shopt -s nullglob
+for f in "$WORKFLOW_DIR"/*.yml "$WORKFLOW_DIR"/*.yaml; do
+  [[ -f "$f" ]] || continue
+  if needs_permissions "$f"; then
+    count_missing=$((count_missing + 1))
+    if [[ "$MODE" == "check" ]]; then
+      echo "missing: $f"
+    else
+      inject_permissions "$f"
+      count_fixed=$((count_fixed + 1))
+      echo "fixed:   $f"
+    fi
+  fi
+done
+
+if [[ "$MODE" == "check" ]]; then
+  if [[ $count_missing -gt 0 ]]; then
+    echo ""
+    echo "$count_missing workflow(s) missing top-level permissions block."
+    exit 1
+  fi
+  echo "All workflows have a top-level permissions block."
+else
+  echo ""
+  echo "Done. $count_fixed workflow(s) updated."
+fi

scripts/security/coverage-changed-files.self-test.mjs

@@ -0,0 +1,380 @@
+#!/usr/bin/env node
+/** Exercises changed-source/test classification against a real throwaway Git history. */
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { dirname, join } from "node:path";
+
+const root = new URL("../..", import.meta.url).pathname;
+const script = join(root, "scripts/security/coverage-changed-files.sh");
+
+function git(cwd, ...args) {
+  const result = spawnSync("git", args, { cwd, encoding: "utf8" });
+  assert.equal(
+    result.status,
+    0,
+    `git ${args.join(" ")} failed: ${result.stderr}`,
+  );
+  return result.stdout.trim();
+}
+
+function write(cwd, relPath, contents) {
+  const full = join(cwd, relPath);
+  mkdirSync(dirname(full), { recursive: true });
+  writeFileSync(full, contents);
+}
+
+// Parse the `name<<EOF\n...\nEOF` heredoc blocks the script emits so assertions
+// read against the same shape GitHub Actions stores into step outputs.
+function parseOutput(stdout) {
+  const sections = {};
+  const lines = stdout.split("\n");
+  let key = null;
+  let buf = [];
+  for (const line of lines) {
+    if (key === null) {
+      const m = line.match(/^(\w+)<<EOF$/);
+      if (m) {
+        key = m[1];
+        buf = [];
+      }
+      continue;
+    }
+    if (line === "EOF") {
+      sections[key] = buf.filter((l) => l !== "");
+      key = null;
+      continue;
+    }
+    buf.push(line);
+  }
+  return sections;
+}
+
+function runScript(cwd, base, head) {
+  const result = spawnSync("bash", [script, base, head], {
+    cwd,
+    encoding: "utf8",
+  });
+  assert.equal(result.status, 0, result.stderr || result.stdout);
+  return parseOutput(result.stdout);
+}
+
+function assertCase(name, fn) {
+  try {
+    fn();
+    console.log(`ok - ${name}`);
+  } catch (error) {
+    console.error(`not ok - ${name}`);
+    throw error;
+  }
+}
+
+const dir = mkdtempSync(join(tmpdir(), "coverage-changed-"));
+try {
+  git(dir, "init", "-q");
+  git(dir, "config", "user.email", "test@example.com");
+  git(dir, "config", "user.name", "test");
+  git(dir, "checkout", "-q", "-b", "develop");
+
+  // Merge-base commit: the point the feature branch forks from.
+  write(dir, "packages/demo/src/base.ts", "export const base = 1;\n");
+  write(
+    dir,
+    "packages/demo/src/runtime-equivalent.ts",
+    "export const stable: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/runtime-changed.ts",
+    "export const changed: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/vite-directive.ts",
+    'export const load = () => import("./module");\n',
+  );
+  write(
+    dir,
+    "packages/demo/src/coverage-directive.ts",
+    "export const covered: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/decorator-metadata.ts",
+    "class Example { @field value: string; }\n",
+  );
+  write(dir, "packages/demo/src/deleted.ts", "export const removed = 1;\n");
+  git(dir, "add", "-A");
+  git(dir, "commit", "-q", "-m", "base");
+  const mergeBase = git(dir, "rev-parse", "HEAD");
+
+  // develop advances with a file the feature branch never touches.
+  write(dir, "packages/demo/src/develop-only.ts", "export const d = 1;\n");
+  git(dir, "add", "-A");
+  git(dir, "commit", "-q", "-m", "develop advances");
+  const developTip = git(dir, "rev-parse", "HEAD"); // BASE (event-time develop tip)
+
+  // Feature branch forks from the merge-base and adds its own source + tests.
+  git(dir, "checkout", "-q", "-b", "feature", mergeBase);
+  rmSync(join(dir, "packages/demo/src/deleted.ts"));
+  write(
+    dir,
+    "packages/demo/src/runtime-equivalent.ts",
+    "export   const stable : number | string = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/runtime-changed.ts",
+    "export const changed: number = 2;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/vite-directive.ts",
+    'export const load = () => import(/* @vite-ignore */ "./module");\n',
+  );
+  write(
+    dir,
+    "packages/demo/src/coverage-directive.ts",
+    "/* c8 ignore next */\nexport const covered: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/decorator-metadata.ts",
+    "class Example { @field value: number; }\n",
+  );
+  write(dir, "packages/demo/src/feature.ts", "export const f = 1;\n");
+  write(
+    dir,
+    "packages/demo/src/types.ts",
+    "export interface RuntimeFree { id: string }\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/public.d.ts",
+    "export interface PublicType { id: string }\n",
+  );
+  write(dir, "packages/demo/src/runtime.mjs", "export const mjs = 1;\n");
+  write(dir, "packages/demo/src/runtime.cjs", "exports.cjs = 1;\n");
+  write(
+    dir,
+    "packages/demo/src/_router.generated.ts",
+    "export const generatedRoute = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/generated/schema.ts",
+    "export const generatedSchema = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/runtime.mts",
+    "export const mts: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/runtime.cts",
+    "export const cts: number = 1;\n",
+  );
+  write(
+    dir,
+    "packages/app/scripts/walkthrough-e2e.mjs",
+    "export async function runWalkthrough() {}\n",
+  );
+  write(
+    dir,
+    "scripts/security/tool.self-test.mjs",
+    "throw new Error('self-test only');\n",
+  );
+  write(
+    dir,
+    "scripts/security/coverage-gate.self-test.mjs",
+    "process.stdout.write('registered self-test ran');\n",
+  );
+  write(
+    dir,
+    "plugins/plugin-demo/vitest.config.ts",
+    "export default { test: { include: ['scripts/**/*.test.mjs'] } };\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/feature.test.ts",
+    "import { test } from 'vitest';\ntest('f', () => {});\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/native.test.ts",
+    "import { test } from 'bun:test';\ntest('n', () => {});\n",
+  );
+  write(
+    dir,
+    "packages/demo/test/e2e/flow.test.ts",
+    "import { test } from 'bun:test';\ntest('e2e', () => {});\n",
+  );
+  write(
+    dir,
+    "packages/test/cloud-e2e/tests/live-deploy.spec.ts",
+    "import { test } from '../src/helpers/test-fixtures';\ntest('live', () => {});\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/__e2e__/fixture.tsx",
+    "export const Fixture = () => null;\n",
+  );
+  write(
+    dir,
+    "packages/demo/src/feature.stories.tsx",
+    "export default { title: 'Feature' };\n",
+  );
+  git(dir, "add", "-A");
+  git(dir, "commit", "-q", "-m", "feature work");
+  const featureTip = git(dir, "rev-parse", "HEAD"); // HEAD
+
+  const out = runScript(dir, developTip, featureTip);
+
+  assertCase(
+    "three-dot diff excludes develop-side files (issue #15845)",
+    () => {
+      // Prove the hazard is real: a plain two-dot diff would drag the develop-only
+      // file into the changed set.
+      const twoDot = git(
+        dir,
+        "diff",
+        "--name-only",
+        developTip,
+        featureTip,
+      ).split("\n");
+      assert.ok(
+        twoDot.includes("packages/demo/src/develop-only.ts"),
+        "expected two-dot diff to include the develop-side file",
+      );
+      // The script uses three-dot, so it must not.
+      assert.ok(
+        !out.files.includes("packages/demo/src/develop-only.ts"),
+        `develop-side file leaked into changed source: ${out.files.join(",")}`,
+      );
+      assert.ok(
+        out.files.includes("packages/demo/src/feature.ts"),
+        `feature source missing from changed set: ${out.files.join(",")}`,
+      );
+    },
+  );
+
+  assertCase(
+    "e2e directory tests excluded from unit lanes (issue #15845)",
+    () => {
+      assert.ok(
+        !out.bun_tests.includes("packages/demo/test/e2e/flow.test.ts"),
+        `e2e-dir test leaked into bun lane: ${out.bun_tests.join(",")}`,
+      );
+      assert.ok(
+        !out.vitest_tests.includes("packages/demo/test/e2e/flow.test.ts"),
+        `e2e-dir test leaked into vitest lane: ${out.vitest_tests.join(",")}`,
+      );
+    },
+  );
+
+  assertCase("cloud Playwright specs stay in their dedicated lane", () => {
+    const liveSpec = "packages/test/cloud-e2e/tests/live-deploy.spec.ts";
+    assert.ok(
+      !out.bun_tests.includes(liveSpec),
+      `cloud Playwright spec leaked into bun lane: ${out.bun_tests.join(",")}`,
+    );
+    assert.ok(
+      !out.vitest_tests.includes(liveSpec),
+      `cloud Playwright spec leaked into vitest lane: ${out.vitest_tests.join(",")}`,
+    );
+  });
+
+  assertCase(
+    "e2e fixtures and stories are not product coverage targets",
+    () => {
+      assert.ok(!out.files.includes("packages/demo/src/__e2e__/fixture.tsx"));
+      assert.ok(!out.files.includes("packages/demo/src/feature.stories.tsx"));
+      assert.ok(
+        !out.files.includes("packages/app/scripts/walkthrough-e2e.mjs"),
+      );
+    },
+  );
+
+  assertCase("unit tests bucket by imported runner", () => {
+    assert.ok(
+      out.bun_tests.includes("packages/demo/src/native.test.ts"),
+      `bun-native test missing: ${out.bun_tests.join(",")}`,
+    );
+    assert.ok(
+      !out.bun_tests.includes("packages/demo/src/feature.test.ts"),
+      "vitest test wrongly in bun lane",
+    );
+    assert.ok(
+      out.vitest_tests.includes("packages/demo/src/feature.test.ts"),
+      `vitest test missing: ${out.vitest_tests.join(",")}`,
+    );
+  });
+
+  assertCase("vitest config changes are not LCOV-enforced source", () => {
+    assert.ok(
+      !out.files.includes("plugins/plugin-demo/vitest.config.ts"),
+      `vitest config leaked into changed source: ${out.files.join(",")}`,
+    );
+  });
+
+  assertCase("generated modules are not LCOV-enforced source", () => {
+    assert.ok(!out.files.includes("packages/demo/src/_router.generated.ts"));
+    assert.ok(!out.files.includes("packages/demo/src/generated/schema.ts"));
+  });
+
+  assertCase(
+    "only registered standalone self-tests leave source enforcement",
+    () => {
+      assert.ok(out.files.includes("scripts/security/tool.self-test.mjs"));
+      assert.ok(
+        !out.files.includes("scripts/security/coverage-gate.self-test.mjs"),
+      );
+      assert.ok(
+        out.node_tests.includes("scripts/security/coverage-gate.self-test.mjs"),
+      );
+    },
+  );
+
+  assertCase(
+    "deleted, declaration, and type-only sources are not LCOV-enforced",
+    () => {
+      assert.ok(!out.files.includes("packages/demo/src/deleted.ts"));
+      assert.ok(!out.files.includes("packages/demo/src/public.d.ts"));
+      assert.ok(!out.files.includes("packages/demo/src/types.ts"));
+    },
+  );
+
+  assertCase("runtime-equivalent source changes are not LCOV-enforced", () => {
+    assert.ok(!out.files.includes("packages/demo/src/runtime-equivalent.ts"));
+    assert.ok(
+      out.files.includes("packages/demo/src/runtime-changed.ts"),
+      `runtime change missing from changed source: ${out.files.join(",")}`,
+    );
+    assert.ok(
+      out.files.includes("packages/demo/src/feature.ts"),
+      `added runtime source missing from changed source: ${out.files.join(",")}`,
+    );
+  });
+
+  assertCase("semantic comment changes remain LCOV-enforced", () => {
+    assert.ok(out.files.includes("packages/demo/src/vite-directive.ts"));
+    assert.ok(out.files.includes("packages/demo/src/coverage-directive.ts"));
+  });
+
+  assertCase("decorator metadata type changes remain LCOV-enforced", () => {
+    assert.ok(out.files.includes("packages/demo/src/decorator-metadata.ts"));
+  });
+
+  assertCase("all executable module extensions are LCOV-enforced", () => {
+    for (const extension of ["mjs", "cjs", "mts", "cts"]) {
+      assert.ok(
+        out.files.includes(`packages/demo/src/runtime.${extension}`),
+        `${extension} runtime module missing: ${out.files.join(",")}`,
+      );
+    }
+  });
+} finally {
+  rmSync(dir, { recursive: true, force: true });
+}

scripts/security/coverage-changed-files.sh

@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# Enumerates the source and unit-test files a pull request actually changed, in
+# the GITHUB_OUTPUT heredoc format the coverage-gate workflow consumes. It is
+# invoked by .github/workflows/coverage-gate.yml as
+# `coverage-changed-files.sh "$BASE" "$HEAD" >> "$GITHUB_OUTPUT"`, and lives as a
+# standalone script (rather than inline YAML) so its path/diff boundary rules are
+# regression-tested by coverage-changed-files.self-test.mjs against a real git
+# repo.
+#
+# The diff is three-dot: it walks from `git merge-base BASE HEAD` to HEAD, so
+# only files the branch itself touched enter the lane. A plain two-dot
+# `BASE..HEAD` diff would count develop-side files the branch never touched as
+# "changed" whenever the branch trails develop, dragging unrelated tests into the
+# gate (issue #15845). Test files are bucketed into a Bun-native lane and a
+# Vitest lane by which runner they import; e2e/live suites and Android specs are
+# excluded by both filename and directory so a `test/e2e/` path cannot slip into
+# the fast unit lane.
+set -euo pipefail
+
+BASE=$1
+HEAD=$2
+SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
+NODE_SELF_TEST_MANIFEST="$SCRIPT_DIR/coverage-node-self-tests.txt"
+
+# Fail fast: an empty merge-base means the two commits share no history (bad
+# fetch depth / wrong refs), which would otherwise silently diff the entire tree.
+MERGE_BASE=$(git merge-base "$BASE" "$HEAD")
+if [ -z "$MERGE_BASE" ]; then
+  echo "coverage-changed-files: no merge-base between $BASE and $HEAD" >&2
+  exit 1
+fi
+
+# Excluded from both unit lanes: e2e/live suites (by filename *and* by a
+# `test/e2e/` directory segment) and Android specs. These run in dedicated lanes
+# and pull in heavy harnesses that the changed-file coverage gate must not.
+is_excluded_test() {
+  case "$1" in
+    *.e2e.test.*|*.live.test.*|packages/app/test/android/*.android.spec.*) return 0 ;;
+    packages/test/cloud-e2e/tests/*.spec.*) return 0 ;;
+    */test/e2e/*|test/e2e/*|*/e2e/*.test.*|e2e/*.test.*) return 0 ;;
+  esac
+  return 1
+}
+
+changed_source() {
+  {
+    git diff --name-only --diff-filter=ACMRT "$MERGE_BASE" "$HEAD" -- \
+      '*.ts' '*.tsx' '*.js' '*.jsx' '*.mjs' '*.cjs' '*.mts' '*.cts' \
+      | grep -vE '(^|/)(__tests__|__e2e__|test|tests|generated)/|([.-]e2e|[.]generated|[.]test|[.]spec|[.]stories)[.](ts|tsx|js|jsx|mjs|cjs|mts|cts)$|(^|/)vitest[.]config[.](ts|js|mts|mjs|cts|cjs)$' || true
+  } \
+    | while IFS= read -r file; do
+        [ -f "$file" ] || continue
+        grep -Fxq "$file" "$NODE_SELF_TEST_MANIFEST" && continue
+        echo "$file"
+      done \
+    | node --no-warnings "$SCRIPT_DIR/coverage-source-classifier.mjs" --base "$MERGE_BASE"
+}
+
+changed_tests() {
+  git diff --name-only "$MERGE_BASE" "$HEAD" -- \
+    '*.test.ts' '*.test.tsx' '*.test.js' '*.test.jsx' '*.test.mjs' \
+    '*.test.cjs' '*.test.mts' '*.test.cts' \
+    '*.spec.ts' '*.spec.tsx' '*.spec.js' '*.spec.jsx' '*.spec.mjs' \
+    '*.spec.cjs' '*.spec.mts' '*.spec.cts'
+}
+
+changed_node_self_tests() {
+  git diff --name-only --diff-filter=ACMRT "$MERGE_BASE" "$HEAD" \
+    | while IFS= read -r file; do
+        [ -f "$file" ] || continue
+        if grep -Fxq "$file" "$NODE_SELF_TEST_MANIFEST"; then
+          echo "$file"
+        fi
+      done
+}
+
+echo 'files<<EOF'
+changed_source
+echo 'EOF'
+
+echo 'node_tests<<EOF'
+changed_node_self_tests
+echo 'EOF'
+
+echo 'bun_tests<<EOF'
+changed_tests | while IFS= read -r file; do
+  [ -f "$file" ] || continue
+  is_excluded_test "$file" && continue
+  if grep -Eq "from ['\"]vitest['\"]|require\\(['\"]vitest['\"]\\)" "$file"; then
+    continue
+  fi
+  if grep -Eq "from ['\"]@playwright/test['\"]|require\\(['\"]@playwright/test['\"]\\)" "$file"; then
+    continue
+  fi
+  echo "$file"
+done
+echo 'EOF'
+
+echo 'vitest_tests<<EOF'
+changed_tests | while IFS= read -r file; do
+  [ -f "$file" ] || continue
+  is_excluded_test "$file" && continue
+  if grep -Eq "from ['\"]@playwright/test['\"]|require\\(['\"]@playwright/test['\"]\\)" "$file"; then
+    continue
+  fi
+  if grep -Eq "from ['\"]vitest['\"]|require\\(['\"]vitest['\"]\\)" "$file"; then
+    echo "$file"
+  fi
+done
+echo 'EOF'

scripts/security/coverage-gate.awk

@@ -0,0 +1,127 @@
+# coverage-gate.awk
+#
+# Parses LCOV output and computes per-file line coverage. Compares against a
+# changed-files list and prints a summary. Exit code is non-zero only when
+# enforcement is enabled (COVERAGE_GATE_ENFORCE=1) and any changed file is
+# missing from the LCOV report or below the threshold (default 70%).
+#
+# Usage:
+#   awk -v changed="$CHANGED_FILES" -v threshold=70 \
+#       -f scripts/security/coverage-gate.awk coverage/lcov.info
+
+BEGIN {
+  if (threshold == "") threshold = 70
+  if (changed == "") changed = ""
+  # Split changed files into a lookup table.
+  n = split(changed, parts, "\n")
+  for (i = 1; i <= n; i++) {
+    if (parts[i] != "") {
+      gsub(/\\/, "/", parts[i])
+      changed_map[parts[i]] = 1
+    }
+  }
+}
+
+function path_matches_lcov(current_path, changed_path,    current_len, changed_len, prefix_len) {
+  gsub(/\\/, "/", current_path)
+  gsub(/\\/, "/", changed_path)
+  if (current_path == changed_path) return 1
+
+  current_len = length(current_path)
+  changed_len = length(changed_path)
+  if (current_len <= changed_len) return 0
+
+  prefix_len = current_len - changed_len
+  return \
+    substr(current_path, prefix_len + 1) == changed_path && \
+    substr(current_path, prefix_len, 1) == "/"
+}
+
+/^SF:/ {
+  sub(/^SF:/, "", $0)
+  current = $0
+  lines_found = 0
+  lines_hit = 0
+}
+
+/^LF:/ {
+  sub(/^LF:/, "", $0)
+  lines_found = $0 + 0
+}
+
+/^LH:/ {
+  sub(/^LH:/, "", $0)
+  lines_hit = $0 + 0
+}
+
+/^end_of_record/ {
+  if (lines_found > 0) {
+    pct = (lines_hit / lines_found) * 100
+    # LCOV paths may be absolute while the changed list is repo-relative. Pick
+    # the longest exact path-segment suffix so overlaps are deterministic.
+    matched = ""
+    matched_len = 0
+    for (cf in changed_map) {
+      if (path_matches_lcov(current, cf) && length(cf) > matched_len) {
+        matched = cf
+        matched_len = length(cf)
+      }
+    }
+    if (matched != "") {
+      # Aggregate per file across ALL lcov inputs BEFORE thresholding. The same
+      # source can appear in several lane reports — its own focused lane at real
+      # coverage plus another package's lane that merely imports it at ~2%. Keep
+      # the file's BEST-lane percentage so an incidental low record cannot fail a
+      # file whose real coverage clears the floor (#16043). Counting, the mean,
+      # and the below[] verdict are all computed from file_pct in END.
+      if (!(matched in file_pct) || pct + 0 > file_pct[matched] + 0) {
+        file_pct[matched] = pct
+      }
+    }
+  }
+  current = ""; lines_found = 0; lines_hit = 0
+}
+
+END {
+  missing_count = 0
+  changed_count = 0
+  changed_sum = 0
+  for (f in changed_map) {
+    if (!(f in file_pct)) {
+      printf "  MISSING: %s\n", f
+      missing_count++
+    } else {
+      changed_count++
+      changed_sum += file_pct[f]
+      printf "  %6.2f%% %s\n", file_pct[f], f
+      if (file_pct[f] + 0 < threshold + 0) below[f] = file_pct[f]
+    }
+  }
+
+  if (changed_count == 0) {
+    print "no changed files matched the LCOV report"
+  } else {
+    avg = changed_sum / changed_count
+    printf "\nchanged files: %d, mean coverage: %.2f%%, threshold: %d%%\n", \
+      changed_count, avg, threshold
+  }
+
+  fail = missing_count > 0
+  for (f in below) {
+    printf "  BELOW: %s (%.2f%%)\n", f, below[f]
+    fail = 1
+  }
+  if (fail && ENVIRON["COVERAGE_GATE_ENFORCE"] == "1") {
+    if (missing_count > 0) {
+      print "coverage gate FAILED (changed source missing from LCOV)"
+    } else {
+      print "coverage gate FAILED (enforcement enabled)"
+    }
+    exit 1
+  }
+  if (fail) {
+    print "coverage gate ADVISORY (set COVERAGE_GATE_ENFORCE=1 to require)"
+  } else {
+    print "coverage gate OK"
+  }
+}

scripts/security/coverage-gate.self-test.mjs

@@ -0,0 +1,192 @@
+#!/usr/bin/env node
+/** Verifies exact changed-path attribution and fail-closed missing-source handling. */
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+const root = new URL("../..", import.meta.url).pathname;
+const awkScript = join(root, "scripts/security/coverage-gate.awk");
+
+function writeLcov(dir, sourcePath, found = 2, hit = 2) {
+  const file = join(dir, "lcov.info");
+  writeFileSync(
+    file,
+    [`SF:${sourcePath}`, `LF:${found}`, `LH:${hit}`, "end_of_record", ""].join(
+      "\n",
+    ),
+  );
+  return file;
+}
+
+function writeLcovRecords(dir, sourcePaths) {
+  const file = join(dir, "lcov.info");
+  writeFileSync(
+    file,
+    sourcePaths
+      .flatMap((sourcePath) => [
+        `SF:${sourcePath}`,
+        "LF:2",
+        "LH:2",
+        "end_of_record",
+      ])
+      .concat("")
+      .join("\n"),
+  );
+  return file;
+}
+
+// Write a single-record lcov under an explicit filename so a test can hand the
+// awk MULTIPLE lcov inputs (one per lane), exactly as the CI does with the
+// per-nearest-config lcov.info files.
+function writeLcovAs(dir, name, sourcePath, found, hit) {
+  const file = join(dir, name);
+  writeFileSync(
+    file,
+    [`SF:${sourcePath}`, `LF:${found}`, `LH:${hit}`, "end_of_record", ""].join(
+      "\n",
+    ),
+  );
+  return file;
+}
+
+function runGate({ changed, lcov, enforce = true, threshold = 50 }) {
+  const changedArgument = changed
+    .replaceAll("\\", "\\\\")
+    .replaceAll("\n", "\\n");
+  const lcovArgs = Array.isArray(lcov) ? lcov : [lcov];
+  return spawnSync(
+    "awk",
+    [
+      "-v",
+      `changed=${changedArgument}`,
+      "-v",
+      `threshold=${threshold}`,
+      "-f",
+      awkScript,
+      ...lcovArgs,
+    ],
+    {
+      cwd: root,
+      env: { ...process.env, COVERAGE_GATE_ENFORCE: enforce ? "1" : "" },
+      encoding: "utf8",
+    },
+  );
+}
+
+function assertGate(name, fn) {
+  try {
+    fn();
+    console.log(`ok - ${name}`);
+  } catch (error) {
+    console.error(`not ok - ${name}`);
+    throw error;
+  }
+}
+
+const dir = mkdtempSync(join(tmpdir(), "coverage-gate-"));
+try {
+  assertGate("matches identical repo-relative path", () => {
+    const lcov = writeLcov(dir, "packages/demo/src/foo.ts");
+    const result = runGate({ changed: "packages/demo/src/foo.ts", lcov });
+    assert.equal(result.status, 0, result.stderr || result.stdout);
+    assert.match(result.stdout, /coverage gate OK/);
+  });
+
+  assertGate("matches absolute LCOV path at path boundary", () => {
+    const lcov = writeLcov(dir, "/workspace/eliza/packages/demo/src/foo.ts");
+    const result = runGate({ changed: "packages/demo/src/foo.ts", lcov });
+    assert.equal(result.status, 0, result.stderr || result.stdout);
+    assert.match(result.stdout, /coverage gate OK/);
+  });
+
+  assertGate("does not match similar filename substring", () => {
+    const lcov = writeLcov(dir, "/workspace/eliza/packages/demo/src/foo.tsx");
+    const result = runGate({ changed: "packages/demo/src/foo.ts", lcov });
+    assert.equal(result.status, 1, result.stdout);
+    assert.match(result.stdout, /changed source missing from LCOV/);
+  });
+
+  assertGate("does not match longer path segment prefix", () => {
+    const lcov = writeLcov(dir, "/workspace/eliza/packages/demo/src/notfoo.ts");
+    const result = runGate({ changed: "packages/demo/src/foo.ts", lcov });
+    assert.equal(result.status, 1, result.stdout);
+    assert.match(result.stdout, /changed source missing from LCOV/);
+  });
+
+  assertGate("fails when any changed source is absent from LCOV", () => {
+    const covered = "packages/demo/src/covered.ts";
+    const missing = "packages/demo/src/missing.ts";
+    const lcov = writeLcov(dir, covered);
+    const result = runGate({ changed: `${covered}\n${missing}`, lcov });
+
+    assert.equal(result.status, 1, result.stdout);
+    assert.match(result.stdout, /100\.00% packages\/demo\/src\/covered\.ts/);
+    assert.match(result.stdout, /MISSING: packages\/demo\/src\/missing\.ts/);
+    assert.match(result.stdout, /changed source missing from LCOV/);
+  });
+
+  assertGate("prefers the longest matching changed path", () => {
+    const rootPath = "src/foo.ts";
+    const nestedPath = "packages/demo/src/foo.ts";
+    const lcov = writeLcovRecords(dir, [
+      `/workspace/eliza/${rootPath}`,
+      `/workspace/eliza/${nestedPath}`,
+    ]);
+    const result = runGate({ changed: `${rootPath}\n${nestedPath}`, lcov });
+
+    assert.equal(result.status, 0, result.stdout);
+    assert.match(result.stdout, /100\.00% src\/foo\.ts/);
+    assert.match(result.stdout, /100\.00% packages\/demo\/src\/foo\.ts/);
+    assert.doesNotMatch(result.stdout, /MISSING:/);
+  });
+
+  assertGate("rejects an executable source reported with LF zero", () => {
+    const source = "packages/demo/src/runtime.ts";
+    const lcov = writeLcov(dir, source, 0, 0);
+    const result = runGate({ changed: source, lcov });
+
+    assert.equal(result.status, 1, result.stdout);
+    assert.match(result.stdout, /MISSING: packages\/demo\/src\/runtime[.]ts/);
+    assert.match(result.stdout, /changed source missing from LCOV/);
+  });
+  assertGate(
+    "aggregates a file across lanes: an incidental low record does not fail a file whose real lane clears the floor (#16043)",
+    () => {
+      const src = "packages/core/src/features/documents/service.ts";
+      // Its own focused lane covers it well (8/10 = 80%); another package's lane
+      // merely imports it (1/50 = 2%). Pre-fix, the 2% record alone failed the gate.
+      const ownLane = writeLcovAs(dir, "core.lcov", src, 10, 8);
+      const importLane = writeLcovAs(dir, "meetings.lcov", src, 50, 1);
+      const result = runGate({ changed: src, lcov: [ownLane, importLane] });
+      assert.equal(result.status, 0, result.stderr || result.stdout);
+      assert.match(result.stdout, /coverage gate OK/);
+      // Counted once, at the best lane's percentage — not the 2% record.
+      assert.match(result.stdout, /changed files: 1,/);
+      assert.match(
+        result.stdout,
+        /80\.00% packages\/core\/src\/features\/documents\/service\.ts/,
+      );
+    },
+  );
+
+  assertGate(
+    "still fails a file that is genuinely below the floor in every lane (#16043)",
+    () => {
+      const src = "packages/core/src/features/documents/service.ts";
+      // Both lanes under 50% (19.90% and 2%). The aggregated best is still below,
+      // so the gate must fail — aggregation must not blanket-pass shared files.
+      const laneA = writeLcovAs(dir, "a.lcov", src, 1000, 199);
+      const laneB = writeLcovAs(dir, "b.lcov", src, 50, 1);
+      const result = runGate({ changed: src, lcov: [laneA, laneB] });
+      assert.equal(result.status, 1, result.stdout);
+      assert.match(
+        result.stdout,
+        /BELOW: packages\/core\/src\/features\/documents\/service\.ts/,
+      );
+    },
+  );
+} finally {
+  rmSync(dir, { recursive: true, force: true });
+}

scripts/security/coverage-node-self-tests.txt

@@ -0,0 +1,4 @@
+packages/scripts/ci-path-gate.self-test.mjs
+packages/scripts/test-cloud-run-flush.self-test.mjs
+scripts/security/coverage-changed-files.self-test.mjs
+scripts/security/coverage-gate.self-test.mjs

scripts/security/coverage-source-classifier.mjs

@@ -0,0 +1,185 @@
+#!/usr/bin/env node
+/** Emits changed modules that retain runtime code after Node strips TypeScript-only syntax. */
+
+import { execFileSync } from "node:child_process";
+import { readFileSync } from "node:fs";
+import * as nodeModule from "node:module";
+import { fileURLToPath } from "node:url";
+
+const bunTypeScriptTranspiler = globalThis.Bun
+  ? new globalThis.Bun.Transpiler({ loader: "ts" })
+  : undefined;
+
+function eraseTypeScriptSyntax(source) {
+  if (typeof nodeModule.stripTypeScriptTypes === "function") {
+    return nodeModule.stripTypeScriptTypes(source, {
+      mode: "transform",
+      sourceMap: false,
+    });
+  }
+  if (bunTypeScriptTranspiler) {
+    return bunTypeScriptTranspiler.transformSync(source);
+  }
+  throw new Error("TypeScript syntax erasure is unavailable in this runtime");
+}
+
+function runtimeSource(source) {
+  return eraseTypeScriptSyntax(source).trim();
+}
+
+function stripTypeScriptSyntaxPreservingComments(source) {
+  if (typeof nodeModule.stripTypeScriptTypes !== "function") {
+    throw new Error("comment-preserving TypeScript erasure is unavailable");
+  }
+  return nodeModule.stripTypeScriptTypes(source, {
+    mode: "strip",
+    sourceMap: false,
+  });
+}
+
+/**
+ * Captures every comment-like delimiter with a whitespace-insensitive source
+ * anchor. False positives inside literals only widen coverage enforcement;
+ * importantly, tooling directives can never disappear from the proof.
+ */
+function commentSensitiveRecords(source) {
+  const stripped = stripTypeScriptSyntaxPreservingComments(source);
+  const commentLike = /\/\/[^\r\n]*|\/\*[\s\S]*?\*\//g;
+  const records = [];
+  let cursor = 0;
+  let anchor = 0;
+  for (const match of stripped.matchAll(commentLike)) {
+    const index = match.index ?? 0;
+    anchor += stripped.slice(cursor, index).replace(/\s/g, "").length;
+    records.push([anchor, match[0]]);
+    anchor += match[0].replace(/\s/g, "").length;
+    cursor = index + match[0].length;
+  }
+  return records;
+}
+
+/** V8 emits no line records for a module made entirely of re-export facades. */
+function isPureReExportFacade(source) {
+  const reExport =
+    /export\s+(?:\*\s*(?:as\s+[A-Za-z_$][\w$]*\s*)?|\{[^}]*\})\s+from\s+["'][^"']+["']\s*;?/gs;
+  return source.trim().length > 0 && source.replace(reExport, "").trim() === "";
+}
+
+/** Returns true only when type erasure leaves coverable runtime statements. */
+export function sourceRetainsRuntimeCode(source) {
+  const emittedSource = runtimeSource(source);
+  return emittedSource.length > 0 && !isPureReExportFacade(emittedSource);
+}
+
+/** Returns true unless emitted JavaScript and comment-sensitive tokens match. */
+export function sourceChangesRuntimeCode(baseSource, headSource) {
+  // Type annotations can alter emitted decorator metadata under repository
+  // tsconfigs even when Node's transform output is otherwise identical.
+  if (
+    baseSource !== headSource &&
+    (baseSource.includes("@") || headSource.includes("@"))
+  ) {
+    return true;
+  }
+  const emittedHead = runtimeSource(headSource);
+  if (emittedHead.length === 0 || isPureReExportFacade(emittedHead)) {
+    return false;
+  }
+  if (runtimeSource(baseSource) !== emittedHead) return true;
+  return (
+    JSON.stringify(commentSensitiveRecords(baseSource)) !==
+    JSON.stringify(commentSensitiveRecords(headSource))
+  );
+}
+
+/** Classifier failures retain the file as a coverage target. */
+export function pathRetainsRuntimeCode(
+  path,
+  warn = (message) => process.stderr.write(message),
+) {
+  try {
+    const source = readFileSync(path, "utf8");
+    return sourceRetainsRuntimeCode(source);
+  } catch (error) {
+    // A classifier failure must widen enforcement, never hide the source.
+    warn(
+      `[coverage-source-classifier] treating unclassifiable module as executable: ${path} (${error instanceof Error ? error.message : String(error)})\n`,
+    );
+    return true;
+  }
+}
+
+/** Writes coverable paths and explains independently proven exclusions. */
+export function classifyPaths(
+  paths,
+  writeOutput = (message) => process.stdout.write(message),
+  writeError = (message) => process.stderr.write(message),
+  { readBaseSource } = {},
+) {
+  for (const path of paths) {
+    if (!pathRetainsRuntimeCode(path, writeError)) {
+      writeError(
+        `[coverage-source-classifier] excluding module without coverable runtime statements: ${path}\n`,
+      );
+      continue;
+    }
+
+    if (readBaseSource) {
+      try {
+        const baseSource = readBaseSource(path);
+        if (baseSource !== undefined) {
+          const headSource = readFileSync(path, "utf8");
+          if (!sourceChangesRuntimeCode(baseSource, headSource)) {
+            writeError(
+              `[coverage-source-classifier] excluding runtime-equivalent source change: ${path}\n`,
+            );
+            continue;
+          }
+        }
+      } catch (error) {
+        // A failed comparison must retain the module just like a failed parse.
+        writeError(
+          `[coverage-source-classifier] treating unclassifiable source change as executable: ${path} (${error instanceof Error ? error.message : String(error)})\n`,
+        );
+      }
+    }
+
+    writeOutput(`${path}\n`);
+  }
+}
+
+function parseBaseRef(args) {
+  if (args.length === 0) return undefined;
+  if (args.length === 2 && args[0] === "--base" && args[1]) return args[1];
+  throw new Error(
+    "usage: coverage-source-classifier.mjs [--base <git-revision>]",
+  );
+}
+
+function readSourceAtRevision(revision, path) {
+  try {
+    return execFileSync("git", ["cat-file", "blob", `${revision}:${path}`], {
+      encoding: "utf8",
+      maxBuffer: 16 * 1024 * 1024,
+      stdio: ["ignore", "pipe", "ignore"],
+    });
+  } catch {
+    // New and renamed paths have no blob at the merge base. Retaining them is
+    // conservative, and also fail-widens unexpected Git lookup failures.
+    return undefined;
+  }
+}
+
+if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
+  const baseRef = parseBaseRef(process.argv.slice(2));
+  classifyPaths(
+    readFileSync(0, "utf8").split("\n").filter(Boolean),
+    undefined,
+    undefined,
+    {
+      readBaseSource: baseRef
+        ? (path) => readSourceAtRevision(baseRef, path)
+        : undefined,
+    },
+  );
+}

scripts/security/coverage-source-classifier.test.mjs

@@ -0,0 +1,69 @@
+/** Verifies runtime-source classification in Bun, including TypeScript erasure and conservative failures. */
+
+import { describe, expect, test } from "bun:test";
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import {
+  classifyPaths,
+  pathRetainsRuntimeCode,
+  sourceRetainsRuntimeCode,
+} from "./coverage-source-classifier.mjs";
+
+describe("coverage source classifier", () => {
+  test("erases TypeScript-only declarations but retains executable modules", () => {
+    expect(
+      sourceRetainsRuntimeCode("export interface Record { id: string }\n"),
+    ).toBe(false);
+    expect(
+      sourceRetainsRuntimeCode("export type Identifier = string;\n"),
+    ).toBe(false);
+    expect(
+      sourceRetainsRuntimeCode("export const value: number = 1;\n"),
+    ).toBe(true);
+  });
+
+  test("excludes pure re-export facades", () => {
+    expect(sourceRetainsRuntimeCode('export * from "./runtime.js";\n')).toBe(false);
+    expect(sourceRetainsRuntimeCode('export { value } from "./runtime.js";\n')).toBe(false);
+  });
+
+  test("classifies paths and reports exclusions", () => {
+    const directory = mkdtempSync(join(tmpdir(), "coverage-source-classifier-"));
+    const runtimePath = join(directory, "runtime.ts");
+    const typesPath = join(directory, "types.ts");
+    writeFileSync(runtimePath, "export const value: number = 1;\n");
+    writeFileSync(typesPath, "export interface Record { id: string }\n");
+
+    try {
+      let output = "";
+      let errors = "";
+      classifyPaths(
+        [runtimePath, typesPath],
+        (message) => {
+          output += message;
+        },
+        (message) => {
+          errors += message;
+        },
+      );
+
+      expect(output).toBe(`${runtimePath}\n`);
+      expect(errors).toContain(typesPath);
+    } finally {
+      rmSync(directory, { recursive: true, force: true });
+    }
+  });
+
+  test("retains unreadable paths so classifier failures cannot weaken enforcement", () => {
+    const warnings = [];
+    expect(
+      pathRetainsRuntimeCode("/definitely/missing.ts", (message) =>
+        warnings.push(message),
+      ),
+    ).toBe(true);
+    expect(warnings.join("")).toContain(
+      "treating unclassifiable module as executable",
+    );
+  });
+});

scripts/security/install-git-hooks.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+# install-git-hooks.sh — one-shot installer for repo-local git hooks.
+#
+# Wires git-hooks/pre-commit-gitleaks.sh into .git/hooks/pre-commit. If a
+# pre-commit hook already exists, the gitleaks check is appended (idempotent).
+#
+# Run manually. Not invoked from postinstall.
+
+set -euo pipefail
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+HOOK_SRC="${REPO_ROOT}/scripts/git-hooks/pre-commit-gitleaks.sh"
+HOOK_DST="${REPO_ROOT}/.git/hooks/pre-commit"
+MARKER="# >>> elizaos gitleaks pre-commit >>>"
+END_MARKER="# <<< elizaos gitleaks pre-commit <<<"
+
+if [[ ! -f "$HOOK_SRC" ]]; then
+  echo "error: $HOOK_SRC not found" >&2
+  exit 1
+fi
+chmod +x "$HOOK_SRC"
+
+mkdir -p "$(dirname "$HOOK_DST")"
+
+if [[ -f "$HOOK_DST" ]] && grep -qF "$MARKER" "$HOOK_DST"; then
+  echo "pre-commit hook already wired up (marker present)."
+  exit 0
+fi
+
+if [[ ! -f "$HOOK_DST" ]]; then
+  cat > "$HOOK_DST" <<'EOF'
+#!/usr/bin/env bash
+set -e
+EOF
+fi
+
+cat >> "$HOOK_DST" <<EOF
+
+${MARKER}
+"${HOOK_SRC}" "\$@"
+${END_MARKER}
+EOF
+
+chmod +x "$HOOK_DST"
+echo "installed gitleaks pre-commit hook at $HOOK_DST"

scripts/security/pin-action-shas.sh

@@ -0,0 +1,144 @@
+#!/usr/bin/env bash
+# pin-action-shas.sh
+#
+# Rewrites `uses: owner/repo@vTAG` references in .github/workflows to use the
+# resolved commit SHA.
+# This implements the OpenSSF Scorecard "Pinned-Dependencies" check (SOC2
+# CC9.2 supply-chain integrity).
+#
+# Resolves SHAs via `gh api repos/<owner>/<repo>/git/refs/tags/<tag>`,
+# following annotated tags to their commit. Requires `gh` authenticated.
+#
+# Usage:
+#   scripts/security/pin-action-shas.sh              # dry-run (default)
+#   scripts/security/pin-action-shas.sh --apply      # rewrite files
+#   scripts/security/pin-action-shas.sh --apply path/to/workflow.yml
+#
+# Skipped:
+#   - References that already use a 40-char SHA (`@<sha>`).
+#   - Local (`uses: ./.github/actions/...`) and docker (`uses: docker://...`).
+#   - Local reusable workflow references (`uses: ./.github/workflows/...`).
+#
+# Idempotent and safe to re-run.
+
+set -euo pipefail
+
+MODE="dry-run"
+TARGETS=()
+for arg in "$@"; do
+  case "$arg" in
+    --apply) MODE="apply" ;;
+    -h|--help)
+      sed -n '2,25p' "$0"; exit 0 ;;
+    *) TARGETS+=("$arg") ;;
+  esac
+done
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+RM_PATH_RECURSIVE=(node "${REPO_ROOT}/packages/scripts/rm-path-recursive.mjs")
+if [[ ${#TARGETS[@]} -eq 0 ]]; then
+  shopt -s nullglob
+  TARGETS=("${REPO_ROOT}/.github/workflows/"*.yml "${REPO_ROOT}/.github/workflows/"*.yaml)
+fi
+
+command -v gh >/dev/null || { echo "gh CLI required" >&2; exit 1; }
+gh auth status >/dev/null 2>&1 || { echo "gh not authenticated" >&2; exit 1; }
+
+SHA_CACHE_DIR=$(mktemp -d)
+cleanup_sha_cache() {
+  "${RM_PATH_RECURSIVE[@]}" "$SHA_CACHE_DIR"
+}
+trap cleanup_sha_cache EXIT
+
+resolve_sha() {
+  local owner_repo="$1" ref="$2" key cache_key cache_file sha obj_type obj_sha
+  key="${owner_repo}@${ref}"
+  cache_key=$(printf '%s' "$key" | shasum -a 256 | awk '{print $1}')
+  cache_file="${SHA_CACHE_DIR}/${cache_key}"
+  if [[ -f "$cache_file" ]]; then
+    cat "$cache_file"
+    return 0
+  fi
+  # Try tag first.
+  local json
+  json=$(gh api "repos/${owner_repo}/git/ref/tags/${ref}" 2>/dev/null) || \
+    json=$(gh api "repos/${owner_repo}/git/ref/heads/${ref}" 2>/dev/null) || {
+      echo ""
+      return 1
+    }
+  obj_type=$(printf '%s' "$json" | python3 -c "import sys,json;print(json.load(sys.stdin)['object']['type'])")
+  obj_sha=$(printf '%s' "$json" | python3 -c "import sys,json;print(json.load(sys.stdin)['object']['sha'])")
+  if [[ "$obj_type" == "tag" ]]; then
+    # Annotated tag — dereference to commit.
+    sha=$(gh api "repos/${owner_repo}/git/tags/${obj_sha}" 2>/dev/null \
+      | python3 -c "import sys,json;print(json.load(sys.stdin)['object']['sha'])")
+  else
+    sha="$obj_sha"
+  fi
+  printf '%s' "$sha" > "$cache_file"
+  echo "$sha"
+}
+
+process_file() {
+  local f="$1"
+  local tmp
+  tmp=$(mktemp)
+  local changed=0
+  while IFS= read -r line; do
+    # Match: optional indent + `- uses: ` or `  uses: ` + owner/repo@ref
+    if [[ "$line" =~ ^([[:space:]]*-?[[:space:]]*uses:[[:space:]]+)([^[:space:]@]+)@([^[:space:]#]+)(.*)$ ]]; then
+      local prefix="${BASH_REMATCH[1]}"
+      local action_path="${BASH_REMATCH[2]}"
+      local ref="${BASH_REMATCH[3]}"
+      local rest="${BASH_REMATCH[4]}"
+      # Skip only local actions / reusable workflows. External reusable
+      # workflows execute third-party code and must be pinned by SHA.
+      if [[ "$action_path" == .* || "$action_path" == docker:* ]]; then
+        echo "$line" >> "$tmp"; continue
+      fi
+      # Already a 40-char SHA?
+      if [[ "$ref" =~ ^[0-9a-f]{40}$ ]]; then
+        echo "$line" >> "$tmp"; continue
+      fi
+      local owner_repo
+      if [[ "$action_path" =~ ^([^/]+/[^/]+)(/.*)?$ ]]; then
+        owner_repo="${BASH_REMATCH[1]}"
+      else
+        echo "$line" >> "$tmp"; continue
+      fi
+      local sha
+      sha=$(resolve_sha "$owner_repo" "$ref" || true)
+      if [[ -z "$sha" ]]; then
+        echo "  ! could not resolve ${owner_repo}@${ref}" >&2
+        echo "$line" >> "$tmp"; continue
+      fi
+      local new_line="${prefix}${action_path}@${sha}"
+      if [[ -n "$rest" ]] && [[ ! "$rest" =~ ^[[:space:]]*# ]] && [[ ! "$rest" =~ ^[[:space:]]*$ ]]; then
+        new_line="${new_line}${rest}"
+      fi
+      echo "$new_line" >> "$tmp"
+      changed=1
+      echo "  ${owner_repo}@${ref} -> ${sha:0:12}"
+    else
+      echo "$line" >> "$tmp"
+    fi
+  done < "$f"
+
+  if [[ $changed -eq 1 ]]; then
+    if [[ "$MODE" == "apply" ]]; then
+      mv "$tmp" "$f"
+    else
+      rm -f "$tmp"
+    fi
+    return 0
+  fi
+  rm -f "$tmp"
+  return 1
+}
+
+echo "Mode: $MODE (use --apply to rewrite files)"
+for f in "${TARGETS[@]}"; do
+  [[ -f "$f" ]] || continue
+  echo "==> $f"
+  process_file "$f" || true
+done

scripts/security/verify-patches.sh

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# verify-patches.sh
+#
+# Verifies the integrity of patches/ against patches/CHECKSUMS.sha256.
+# Any patch change requires the CHECKSUMS file to be regenerated and reviewed
+# as part of the same PR (enforced via CI). SOC2 CC6.8 / CC8.1.
+#
+# Usage:
+#   scripts/security/verify-patches.sh             # verify
+#   scripts/security/verify-patches.sh --generate  # regenerate CHECKSUMS
+#
+# Exit codes:
+#   0  all checksums match
+#   1  mismatch or missing patches
+#   2  CHECKSUMS file missing
+
+set -euo pipefail
+
+REPO_ROOT=$(git rev-parse --show-toplevel)
+PATCHES_DIR="${REPO_ROOT}/patches"
+CHECKSUMS="${PATCHES_DIR}/CHECKSUMS.sha256"
+
+if [[ ! -d "$PATCHES_DIR" ]]; then
+  echo "no patches/ directory at ${PATCHES_DIR}; nothing to verify"
+  exit 0
+fi
+
+# Pick a sha256 tool. macOS has shasum, Linux has sha256sum.
+if command -v sha256sum >/dev/null 2>&1; then
+  SHA_CMD="sha256sum"
+  SHA_CHECK="sha256sum --check --strict"
+elif command -v shasum >/dev/null 2>&1; then
+  SHA_CMD="shasum -a 256"
+  SHA_CHECK="shasum -a 256 --check --strict"
+else
+  echo "error: neither sha256sum nor shasum found" >&2
+  exit 1
+fi
+
+generate() {
+  ( cd "$PATCHES_DIR" && \
+    find . -type f \( -name '*.patch' -o -name '*.diff' \) ! -name 'CHECKSUMS.sha256' \
+      -print0 | sort -z | xargs -0 $SHA_CMD ) > "$CHECKSUMS"
+  echo "wrote $(wc -l < "$CHECKSUMS") entries to $CHECKSUMS"
+}
+
+verify() {
+  if [[ ! -f "$CHECKSUMS" ]]; then
+    echo "error: $CHECKSUMS missing. Run with --generate to create the baseline." >&2
+    exit 2
+  fi
+  # Reject any patch on disk not listed in CHECKSUMS.
+  local known_count actual_count
+  known_count=$(grep -cE '\.(patch|diff)$' "$CHECKSUMS" 2>/dev/null || true)
+  known_count=${known_count:-0}
+  actual_count=$(find "$PATCHES_DIR" -type f \( -name '*.patch' -o -name '*.diff' \) | wc -l | tr -d ' ')
+  if [[ "$known_count" -ne "$actual_count" ]]; then
+    echo "error: patch count mismatch — CHECKSUMS lists $known_count files, disk has $actual_count" >&2
+    echo "Run with --generate after reviewing the change." >&2
+    exit 1
+  fi
+  ( cd "$PATCHES_DIR" && $SHA_CHECK CHECKSUMS.sha256 ) >/dev/null
+  echo "patches/: all $known_count checksums verified"
+}
+
+case "${1:-verify}" in
+  --generate|-g) generate ;;
+  --verify|verify|"") verify ;;
+  -h|--help) sed -n '2,20p' "$0"; exit 0 ;;
+  *) echo "unknown arg: $1" >&2; exit 2 ;;
+esac

scripts/testing-coverage-matrix.mjs

@@ -0,0 +1,208 @@
+#!/usr/bin/env node
+/**
+ * Per-package testing coverage matrix generator for issue #9943.
+ *
+ * Walks every workspace package using tracked files from `git ls-files`, counts
+ * test/spec files and skipped tests, and records whether the package exposes the
+ * `test` script used by the root workspace runner. Emits
+ * docs/TESTING_COVERAGE_MATRIX.md.
+ *
+ * "In CI" is a heuristic: a package is swept by the root runner when it is a
+ * workspace member with a `test` script and is not excluded from the root
+ * workspace. packages/feed is excluded from the root workspace and runs through
+ * its own path-gated feed-test.yml lane instead.
+ *
+ * Usage:
+ *   node scripts/testing-coverage-matrix.mjs
+ *   node scripts/testing-coverage-matrix.mjs --check
+ */
+
+import { execFileSync } from "node:child_process";
+import { existsSync, readFileSync, writeFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const REPO_ROOT = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "..",
+);
+const OUTPUT = path.join(REPO_ROOT, "docs", "TESTING_COVERAGE_MATRIX.md");
+
+const TEST_FILE_RE = /\.(test|spec)\.(ts|tsx|js|jsx|mjs|cjs)$/;
+// it.skip / test.skip / describe.skip / xit / xdescribe / .todo / .skipIf(...)
+const SKIP_RE =
+  /\b(?:it|test|describe)\.(?:skip|todo)\b|\bx(?:it|describe)\s*\(|\.skipIf\s*\(|\btest\.skip\b/g;
+
+function gitFiles() {
+  return execFileSync("git", ["ls-files"], {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    maxBuffer: 256 * 1024 * 1024,
+  })
+    .split("\n")
+    .filter(Boolean);
+}
+
+function readJson(rel) {
+  try {
+    return JSON.parse(readFileSync(path.join(REPO_ROOT, rel), "utf8"));
+  } catch {
+    return null;
+  }
+}
+
+function main() {
+  const files = gitFiles();
+
+  // Every directory that owns a package.json (excluding the repo root) is a
+  // package node. Tracked files never include node_modules, so this is clean.
+  const packageDirs = files
+    .filter((f) => f.endsWith("/package.json") || f === "package.json")
+    .map((f) => path.posix.dirname(f))
+    .filter((dir) => dir !== ".")
+    .sort();
+  const packageDirSet = new Set(packageDirs);
+
+  /** Deepest package dir that is an ancestor of `file` (or null). */
+  function ownerPackage(file) {
+    let dir = path.posix.dirname(file);
+    while (dir !== "." && dir.length > 0) {
+      if (packageDirSet.has(dir)) return dir;
+      const next = path.posix.dirname(dir);
+      if (next === dir) break;
+      dir = next;
+    }
+    return null;
+  }
+
+  const rootPkg = readJson("package.json");
+  const workspaceNegations = (rootPkg?.workspaces ?? [])
+    .filter((w) => typeof w === "string" && w.startsWith("!"))
+    .map((w) => w.slice(1));
+  const excludedFromRoot = (dir) =>
+    workspaceNegations.some((neg) => dir === neg || dir.startsWith(`${neg}/`));
+
+  const stats = new Map(
+    packageDirs.map((dir) => [
+      dir,
+      { dir, name: null, testFiles: 0, skips: 0, hasTestScript: false },
+    ]),
+  );
+
+  // Per-package package.json metadata.
+  for (const dir of packageDirs) {
+    const pkg = readJson(`${dir}/package.json`);
+    const entry = stats.get(dir);
+    entry.name = pkg?.name ?? "(unnamed)";
+    entry.hasTestScript = Boolean(pkg?.scripts?.test);
+  }
+
+  // Assign test files to their owning package and count skips.
+  for (const file of files) {
+    if (!TEST_FILE_RE.test(file)) continue;
+    const owner = ownerPackage(file);
+    if (!owner) continue;
+    const entry = stats.get(owner);
+    entry.testFiles += 1;
+    try {
+      const body = readFileSync(path.join(REPO_ROOT, file), "utf8");
+      const matches = body.match(SKIP_RE);
+      if (matches) entry.skips += matches.length;
+    } catch {
+      // Unreadable tracked file (symlink/binary) — skip silently.
+    }
+  }
+
+  const rows = [...stats.values()]
+    .filter((e) => e.testFiles > 0 || e.hasTestScript)
+    .sort((a, b) => a.dir.localeCompare(b.dir));
+
+  const totals = rows.reduce(
+    (acc, r) => {
+      acc.testFiles += r.testFiles;
+      acc.skips += r.skips;
+      acc.withTests += r.testFiles > 0 ? 1 : 0;
+      acc.inCi += r.hasTestScript && !excludedFromRoot(r.dir) ? 1 : 0;
+      acc.zeroTestWithScript += r.testFiles === 0 && r.hasTestScript ? 1 : 0;
+      return acc;
+    },
+    { testFiles: 0, skips: 0, withTests: 0, inCi: 0, zeroTestWithScript: 0 },
+  );
+
+  const lines = [];
+  lines.push("# Testing coverage matrix");
+  lines.push("");
+  lines.push(
+    "Per-package view of test presence, CI wiring, and skip debt. Generated by " +
+      "`node scripts/testing-coverage-matrix.mjs` — do not hand-edit; re-run to refresh.",
+  );
+  lines.push("");
+  lines.push("## Columns");
+  lines.push("");
+  lines.push(
+    "- **Test files** — count of `*.{test,spec}.{ts,tsx,js,jsx,mjs,cjs}` owned by the package (assigned to the deepest enclosing package).",
+  );
+  lines.push(
+    "- **Skips** — count of `it/test/describe.skip`, `.todo`, `x(it|describe)`, and `.skipIf(...)` markers across those files.",
+  );
+  lines.push(
+    "- **`test` script** — package exposes a `test` script (the signal the root runner sweeps on).",
+  );
+  lines.push(
+    "- **In CI (heuristic)** — has a `test` script AND is not excluded from the root workspace. Not a proof of execution; lane-level gating still applies.",
+  );
+  lines.push("");
+  lines.push("## Summary");
+  lines.push("");
+  lines.push(`- Packages with tests: **${totals.withTests}**`);
+  lines.push(
+    `- Packages swept by the root runner (heuristic): **${totals.inCi}**`,
+  );
+  lines.push(`- Total test files: **${totals.testFiles}**`);
+  lines.push(`- Total skipped tests: **${totals.skips}**`);
+  lines.push(
+    `- Packages with a \`test\` script but zero test files: **${totals.zeroTestWithScript}**`,
+  );
+  lines.push("");
+  lines.push(
+    "> Known dark corner: `packages/feed` is excluded from the root workspace " +
+      "(`!packages/feed`) and runs via its own path-gated `feed-test.yml` lane " +
+      "(`test:unit`). Its ~227 skips still need triage (#9943 follow-up).",
+  );
+  lines.push("");
+  lines.push("## Matrix");
+  lines.push("");
+  lines.push(
+    "| Package | Path | Test files | Skips | `test` script | In CI (heuristic) |",
+  );
+  lines.push("| --- | --- | ---: | ---: | :---: | :---: |");
+  for (const r of rows) {
+    const inCi = r.hasTestScript && !excludedFromRoot(r.dir);
+    lines.push(
+      `| ${r.name} | \`${r.dir}\` | ${r.testFiles} | ${r.skips} | ${r.hasTestScript ? "yes" : "no"} | ${inCi ? "yes" : "no"} |`,
+    );
+  }
+  lines.push("");
+
+  const content = `${lines.join("\n")}`;
+
+  if (process.argv.includes("--check")) {
+    const current = existsSync(OUTPUT) ? readFileSync(OUTPUT, "utf8") : "";
+    if (current !== content) {
+      console.error(
+        "docs/TESTING_COVERAGE_MATRIX.md is stale. Run `node scripts/testing-coverage-matrix.mjs`.",
+      );
+      process.exit(1);
+    }
+    console.log("TESTING_COVERAGE_MATRIX.md is up to date.");
+    return;
+  }
+
+  writeFileSync(OUTPUT, content);
+  console.log(
+    `Wrote ${path.relative(REPO_ROOT, OUTPUT)}: ${rows.length} packages, ` +
+      `${totals.testFiles} test files, ${totals.skips} skips.`,
+  );
+}
+
+main();

scripts/training-harvest/README.md

@@ -0,0 +1,80 @@
+# gpt-5.5 trajectory-training harvest pipeline
+
+Stage 2 corpus harvest for the gpt-5.5 → eliza-1 training pipeline. Runs the
+elizaOS test/eval corpus through gpt-5.5 (Codex subscription), captures every
+run as an `eliza_native_v1` trajectory + pass/fail verdict, so Stage 3 can
+GEPA-repair the failures and Stage 4 can extract the passes into training data.
+
+**Stage 1 status:** inventory + wiring + driver proven on a small slice. The
+full corpus is NOT run here.
+
+## Files
+
+| File | Role |
+|---|---|
+| `build-manifest.mjs` | Enumerates the whole corpus → `manifest.json` (families, items, run commands, trajectory landing). Discover-only; runs nothing. |
+| `manifest.json` | Machine-readable corpus manifest (generated). |
+| `harvest-runner.mjs` | Stage-2 driver. Consumes the Stage-1 provider incantation, iterates the manifest, runs each item, captures trajectory + verdict into `harvest/`. |
+| `s1-provider.example.json` | Example provider-env the driver consumes (`{ELIZA_CHAT_VIA_CLI:"codex", ELIZA_CLI_CODEX_MODEL:"gpt-5.5"}`). Stage-1 leg S1 emits the real one. |
+
+## Provider seam (how gpt-5.5-via-Codex slots in)
+
+The scenario runner already has a first-class CLI-subscription provider
+(`packages/core/src/testing/live-provider.ts` → `selectCliProvider`). Setting
+`ELIZA_CHAT_VIA_CLI=codex` selects provider `"cli"`, model `gpt-5.5`, plugin
+`@elizaos/plugin-cli-inference`, which reads `~/.codex/auth.json` itself — no API
+key ever passes through eliza. The driver injects this env per spawn; it is
+never hard-coded. Precedence: `--provider-env <s1.json>` → `$HARVEST_PROVIDER_ENV_FILE`
+→ inherited `ELIZA_CHAT_VIA_CLI`/API key → `--deterministic` (offline self-test).
+
+## Families & trajectory emission
+
+- **scenario** — `@elizaos/scenario-runner` drives a real `AgentRuntime`+PGLite.
+  Emits `eliza_native_v1` **natively** via `--export-native`. Verdict per
+  scenario id (`report.scenarios[].status`, mirrored to `native row.scenarioStatus`).
+  978 base scenario ids across 7 dirs (~10,716 with persona expansion).
+- **benchmark** — `packages/benchmarks` orchestrator, 53 adapters. Emits
+  per-benchmark result JSON today. Native trajectories require wiring: the ~25
+  `eliza-adapter`-routed benchmarks boot a real runtime; set
+  `ELIZA_SAVE_TRAJECTORIES=1 ELIZA_TRAJECTORY_DIR=<dir>` then convert `<dir>`
+  with scenario-runner `native-export`. Python-only/`standard/*` benchmarks
+  don't boot the runtime and can't emit native trajectories.
+- **e2e** — 51 `*.live.e2e.test.ts` / `*.real.e2e.test.ts` vitest lanes that
+  drive a real runtime. Same trajectory wiring as benchmarks; verdict is coarse
+  (vitest pass/fail per file).
+
+Only the **scenario** family emits `eliza_native_v1` today, so the driver
+executes that family in Stage 1. Benchmark/e2e are enumerated with their wiring
+notes for Stage 2.
+
+## Harvest layout (what Stage 2 writes)
+
+```
+<repo>/reports/training-harvest/gpt55/harvest/
+  <family>/<dir-slug>/<item-slug>/
+      report.json                 scenario aggregate report (per-scenario status)
+      native.jsonl                eliza_native_v1 rows (scenarioStatus + judgeScore)
+      native.jsonl.manifest.json
+      verdict.json                { item, status, rows, judgeScore, exitCode }
+      run/trajectories/**         RecordedTrajectory JSON (source for native export)
+      stdout.log / stderr.log
+  harvest-summary-<run|dryrun>-<ts>.json
+```
+
+## Usage
+
+```bash
+# rebuild the manifest
+node scripts/training-harvest/build-manifest.mjs
+
+# driver self-test (offline, deterministic proxy)
+node scripts/training-harvest/harvest-runner.mjs --deterministic --limit 3 --dry-run
+
+# STAGE 2 real harvest (consumes S1's proven provider env)
+node scripts/training-harvest/harvest-runner.mjs \
+  --provider-env <s1-output.json> --family scenario
+```
+
+Prereq in a fresh worktree: generate the i18n keyword data once —
+`node packages/shared/scripts/generate-keywords.mjs` (gitignored
+build artifact; the CLI imports `packages/core/src/i18n/generated/`).

scripts/training-harvest/STAGE34_README.md

@@ -0,0 +1,220 @@
+# Stage 3 (GEPA sweep) + Stage 4 (extract & push) — driver reference
+
+Two drivers that consume the gpt-5.5 harvest tree
+(`reports/training-harvest/gpt55/harvest/{scenario,benchmark}/`)
+built by `harvest-runner.mjs` / `bench-e2e-harvest-runner.mjs`.
+
+- **Stage 3** turns the harvest's *real agent failures* into GEPA-optimized
+  prompts, re-runs the failing scenarios to confirm the flip, and feeds the
+  now-passing trajectories back into the harvest.
+- **Stage 4** collects every *passing* trajectory (originals + Stage-3 flips),
+  converts them to `eliza_native_v1` training JSONL, splits train/val/test, and
+  (guarded) pushes an additive subdir to HuggingFace.
+
+Neither driver commits, and neither echoes any secret (HF token, Cerebras key,
+codex tokens). They only write under the evidence tree.
+
+---
+
+## Stage 3 — `stage3-gepa-sweep.mjs`
+
+### What it does
+
+1. **Select GEPA-optimizable failures** from `report.json` (authoritative — never
+   stderr). Each failed scenario is classified:
+   - `CONNECTOR_CRED` — env-gated (missing connector / credential / mock; or a
+     `connector.*` / `*.certify` scenario id). **Excluded** — can't train/GEPA a
+     missing-credential failure.
+   - `ACTION_SELECTION` — wrong action / wrong action args
+     (`no selected action in […]`, `expectedActions`, `selectedActionArguments`,
+     `expected N call(s) to X`). GEPA-optimizable.
+   - `JUDGE_BELOW` — response judge score under threshold (`responseJudge`,
+     `score X < Y`). GEPA-optimizable.
+   - `OUTPUT_FORMAT` — `responseIncludes/Excludes`, structured-argument mismatch.
+     GEPA-optimizable.
+   - `OTHER` — real failure, no clean task mapping — not GEPA-targeted here.
+2. **Group by GEPA task.** Each failing scenario's own `native.jsonl` rows carry a
+   stamped `metadata.task_type`; the failure is bucketed into the GEPA `--task`
+   it implicates (ACTION_SELECTION → `action_planner`; else the dominant mapped
+   task present — `should_respond`, a LifeOps task, etc.). `evaluation` rows are
+   evaluator calls, not a GEPA prompt task — dropped from grouping.
+3. **Build the per-task GEPA dataset.** GEPA optimizes a task's **system prompt**
+   against that task's **gold** `(input → expected)` rows. The native backend
+   deliberately **excludes failed-scenario rows as gold** (`native.ts`
+   `isFailedScenarioSignal`, #8795) — a wrong output must never be optimized
+   *toward*. So the dataset for a task = the **passing** scenarios' `native.jsonl`
+   rows for that `task_type`, deduped, written in the exact `eliza_native_v1`
+   shape the native backend parses (`datasets/<task>.gepa.jsonl`). The failing
+   scenarios supply only the *inputs* the optimized prompt must then get right;
+   the confirmation is the re-run.
+4. **Emit the exact `train` command per task**, then (with `--run`) execute it:
+   GEPA → artifact at `<state-dir>/optimized-prompts/<task>/current` → re-run the
+   failing scenarios with `ELIZA_STATE_DIR=<state-dir>` so the runtime substitutes
+   the optimized prompt at boot → capture the flip. Every flip's now-passing
+   trajectory is copied into `harvest/scenario/<family>/<item>__gepa/` (additive;
+   never overwrites the original failing capture) so Stage 4 picks it up.
+
+Scoring model is **Cerebras** (fast, sanctioned — `TRAIN_MODEL_PROVIDER=cerebras`,
+default `gemma-4-31b`). The flip **validation** re-run is **gpt-5.5-via-Codex**
+(the S1 provider env at `/tmp/s1-provider-full.json`).
+
+Resumable: per-task dataset, per-task artifact, and per-scenario rerun verdict are
+all on disk; re-invoking skips completed steps unless `--force`.
+
+### Commands
+
+```bash
+# DRY RUN (default): classify + group + build datasets + PRINT the per-task commands.
+node scripts/training-harvest/stage3-gepa-sweep.mjs --dry-run
+
+# SMALL end-to-end proof: GEPA one task on its real harvested cases + 1 rerun.
+CEREBRAS_API_KEY=<key> \
+  node scripts/training-harvest/stage3-gepa-sweep.mjs --run --only action_planner --max-rerun 1
+
+# FULL SWEEP (expensive — run after the full harvest completes):
+CEREBRAS_API_KEY=<key> \
+  node scripts/training-harvest/stage3-gepa-sweep.mjs --run
+
+# Re-run failing scenarios against an already-produced artifact store (no GEPA):
+node scripts/training-harvest/stage3-gepa-sweep.mjs --rerun-only --state-dir <dir>
+```
+
+Flags: `--only <task>`, `--max-rerun <n>` (cap reruns per task), `--state-dir <dir>`
+(GEPA artifact store; default `s3-gepa/sweep/state`), `--force`, `--timeout-ms`.
+
+Each task's raw `train` command (what `--run` executes):
+
+```bash
+TRAIN_MODEL_PROVIDER=cerebras CEREBRAS_API_KEY=<key> ELIZA_STATE_DIR=<state-dir> \
+  bun run --cwd plugins/plugin-training train -- \
+    --backend native --optimizer gepa --task <task> \
+    --dataset reports/training-harvest/gpt55/s3-gepa/sweep/datasets/<task>.gepa.jsonl
+```
+
+(`--baseline` is inferred from the dataset's system message when omitted; for
+tasks whose native rows carry no `request.system` — e.g. `view_context` — pass
+`--baseline <prompt.txt>`.)
+
+### Task-grouping map (native `task_type` → GEPA `--task`)
+
+| native `task_type` | GEPA `--task`     | scorer                       |
+|--------------------|-------------------|------------------------------|
+| `should_respond`   | `should_respond`  | token-overlap (default)      |
+| `context_routing`  | `context_routing` | token-overlap (default)      |
+| `action_planner`   | `action_planner`  | `scorePlannerAction` (exact) |
+| `response`         | `response`        | token-overlap (default)      |
+| `media_description`| `media_description`| token-overlap (default)     |
+| `view_context`     | `view_context`    | `scoreViewSelection` (exact) |
+| `calendar_extract` … `screentime_recap` (8 LifeOps) | same id | `scoreLifeOpsTask` (exact) |
+| `evaluation`       | *(dropped)*       | not a GEPA prompt task       |
+
+Valid `--task` ids = `ALL_TRAJECTORY_TRAINING_TASKS`
+(`plugins/plugin-training/src/core/trajectory-task-datasets.ts`). Scorers:
+`plugins/plugin-training/src/optimizers/scoring.ts`.
+
+### Known scorer nuance (documented, non-blocking)
+
+`scorePlannerAction` → `extractPlannerAction` reads `record.name ?? action ??
+actionName` first; the harvested AI-SDK toolCalls use `toolName`, so it falls
+through to the `\b[A-Z][A-Z0-9_]{2,}\b` regex fallback. On the harvested
+`action_planner` gold this resolves the correct action for **152/154** rows; 2
+rows where an ALL-CAPS token precedes the toolName mis-extract. GEPA still runs
+and scores over the whole set; the S3 smoke already proved the full 0→1 flip on
+`view_context`. (A one-line fix — teach `extractPlannerAction` to read
+`toolName` — would make it exact; out of scope for these drivers.)
+
+---
+
+## Stage 4 — `stage4-extract-and-push.mjs`
+
+### What it does
+
+1. **Collect all passing trajectories** — every harvest item with
+   `verdict.status === "passed"`, across both families, **including Stage-3
+   GEPA-flips** (`<item>__gepa/`).
+2. **Extract** each item's `native.jsonl` through the proven Python extractor
+   (`packages/training/scripts/extract_trajectory_to_native.py --require-pass`),
+   which validates every row through `validate_native_record` and drops any
+   failed/skipped-scenario row (belt-and-suspenders — passing items shouldn't
+   have any).
+3. **Dedup + clean.** Drop degenerate rows (no user turn OR no non-empty
+   expected). Dedup on the training signal `(system, user, expected)`, not on
+   ids/timestamps.
+4. **Split** train/val/test with a deterministic `hash(seed + callId)` in
+   `[0,1)` (mirrors `prepare_eliza1_trajectory_dataset.py` `stable_unit`; default
+   val-ratio 0.05, test-ratio 0.05). Splits are disjoint by construction.
+5. **Assemble** `assembled/{train,val,test}.jsonl` + `manifest.json`, and
+   schema-self-check the sample row.
+6. **Push (guarded, `--push` only)** to `elizaos/eliza-1-training-data` under a
+   **new additive subdir** `converted/gpt55-scenarios/` via
+   `hf upload … --repo-type dataset`. Default is local-assemble-only; the push
+   never clobbers the existing `converted/{xlam,hermes,glaive,openclaw,merged}`
+   sets.
+
+### Commands
+
+```bash
+# STATS only (extract + classify, no writes):
+node scripts/training-harvest/stage4-extract-and-push.mjs --stats
+
+# LOCAL ASSEMBLE (default — writes assembled/{train,val,test}.jsonl + manifest):
+node scripts/training-harvest/stage4-extract-and-push.mjs
+
+# PUSH to HF (guarded — run after full harvest + Stage-3 flips are in):
+HF_TOKEN=$(cat ~/.cache/huggingface/token) \
+  node scripts/training-harvest/stage4-extract-and-push.mjs --push
+```
+
+Flags: `--out <dir>`, `--seed <s>`, `--val-ratio`, `--test-ratio`.
+
+### Dataset schema (matches `elizaos/eliza-1-training-data` converted rows)
+
+One row = one Vercel AI SDK `generateText` boundary — the `eliza_native_v1`
+shape (`packages/training/scripts/lib/native_record.py`). The HF
+`converted/merged/*.jsonl` records use the core projection
+`{format, boundary, request:{messages,tools}, response:{text,toolCalls},
+metadata, scenarioId, stepIndex}`; the assembled rows are a **strict superset**
+of that (same core keys + richer per-boundary provenance:
+`trajectoryId, callId, scenarioStatus, judgeScore, modelType, provider,
+schemaVersion, …`). Because the push target is a new additive subdir, the
+superset is preferred — no provenance is dropped and every training loader reads
+the same core fields.
+
+```json
+{"format":"eliza_native_v1","schemaVersion":1,"boundary":"vercel_ai_sdk.generateText",
+ "scenarioStatus":"passed",
+ "request":{"messages":[{"role":"user","content":"…"}],"tools":[…]},
+ "response":{"text":"…","toolCalls":[{"toolName":"CALENDAR","input":{…}}],"finishReason":"…"},
+ "trajectoryId":"…","agentId":"…","callId":"…","scenarioId":"…",
+ "metadata":{"task_type":"action_planner","scenario_status":"passed","judge_score":null,…}}
+```
+
+Acceptance gate (`validate_native_record`): `format == eliza_native_v1`,
+`boundary ∈ {generateText, streamText}`, `request` has ≥1 user turn (messages)
+OR a non-empty `prompt`, `response` has non-empty `text` OR non-empty
+`toolCalls`.
+
+### HF push target
+
+- Repo: `elizaos/eliza-1-training-data` (dataset, public; token role = write).
+- Path-in-repo: `converted/gpt55-scenarios/{train,val,test}.jsonl` (additive).
+- Uploader: `hf upload <repo> <local> <path-in-repo> --repo-type dataset`
+  (same mechanism as the runtime cron `trajectory-hf-upload.ts`).
+
+---
+
+## The full-run sequence (once the harvest completes)
+
+```bash
+# 1. Full GEPA sweep — optimize every runnable task, re-run failing scenarios,
+#    capture flips back into harvest/scenario/<family>/<item>__gepa/.
+CEREBRAS_API_KEY=<key> \
+  node scripts/training-harvest/stage3-gepa-sweep.mjs --run
+
+# 2. Extract every passing trajectory (originals + flips) → local splits.
+node scripts/training-harvest/stage4-extract-and-push.mjs
+
+# 3. Publish the additive subdir to HuggingFace.
+HF_TOKEN=$(cat ~/.cache/huggingface/token) \
+  node scripts/training-harvest/stage4-extract-and-push.mjs --push
+```

scripts/training-harvest/bench-e2e-harvest-runner.mjs

@@ -0,0 +1,486 @@
+#!/usr/bin/env node
+/**
+ * gpt-5.5 trajectory-training pipeline — Stage 2 harvest driver for the
+ * BENCHMARK and E2E families (sibling of harvest-runner.mjs, which owns the
+ * scenario family). This driver never touches the scenario code path.
+ *
+ * Both families drive a real AgentRuntime through the CLI-subscription provider
+ * (plugin-cli-inference: `codex exec -m gpt-5.5`), so the runtime's
+ * JsonFileTrajectoryRecorder writes RecordedTrajectory JSON to
+ * ELIZA_TRAJECTORY_DIR. The driver then converts that to eliza_native_v1 JSONL
+ * with the same scenario-runner native-export the scenario family uses.
+ *
+ * Layout written per item (mirrors the scenario family):
+ *   <harvestRoot>/<family>/<suite-or-lane-slug>/<item-slug>/
+ *       run/trajectories/<agentId>/<trajId>.json   RecordedTrajectory (recorder)
+ *       native.jsonl                                eliza_native_v1 rows
+ *       native.manifest.json                        scenario-runner export manifest
+ *       verdict.json                               { item, status, rows, exitCode }
+ *       result.json (benchmark)                    bench.ts scenario result
+ *       stdout.log / stderr.log
+ *
+ * PROVIDER PARAMETERIZATION (consumes Stage-1 output; identical precedence to
+ * harvest-runner.mjs so both drivers take the same --provider-env file):
+ *     1. --provider-env <file.json>   (a JSON object of env vars; what S1 writes)
+ *     2. $HARVEST_PROVIDER_ENV_FILE
+ *     3. inherited ELIZA_CHAT_VIA_CLI / an API key
+ *     4. --deterministic (offline driver self-test — enumerate only, no live run)
+ *
+ * When the resolved provider is a CLI backend (claude or codex), the driver forces
+ * the two env vars the CLI route requires end-to-end but that a bare provider-env
+ * file may omit:
+ *     ELIZA_PLANNER_NATIVE_TOOLS=0   text-planner mode (free-text CLI serves the planner)
+ *     ELIZA_TRAJECTORY_RECORDING=1   keep the recorder on (it is default-on; set explicit)
+ *
+ * FAMILIES
+ *   benchmark  The framework TS benchmark (packages/benchmarks/framework/typescript/src/bench.ts).
+ *              Its --real-llm cli/codex branch (this stage's bench.ts change) drives one real
+ *              gpt-5.5 turn per scenario. Corpus = the model-driving framework scenarios; the
+ *              db/startup scenarios never call the model, so they are excluded. Runs each scenario
+ *              at --iterations=1 --warmup=0 (one real turn = one trajectory), minimal cost.
+ *   e2e        A live app-core vitest lane (packages/app-core/test/live-agent/*). Runs the lane
+ *              through vitest.harvest-live-agent.config.ts with ELIZA_LIVE_TEST=1 + the trajectory
+ *              env; verdict = vitest pass/fail. The app-core selectLiveProvider now recognizes the
+ *              cli backend (this stage) so lanes select gpt-5.5. NOTE: the live-agent lanes eagerly
+ *              import first-party plugins, so a dist-less worktree fails at Vite transform → the
+ *              driver records status "blocked-workspace-build" (run `bun run build` first). Full
+ *              harvestable-vs-env-gated classification + the build precondition: BENCHMARK_E2E_README.md.
+ *
+ * Usage:
+ *   node bench-e2e-harvest-runner.mjs --family benchmark --provider-env s1.json [--limit N]
+ *   node bench-e2e-harvest-runner.mjs --family e2e --provider-env s1.json --lane <substr>
+ *   node bench-e2e-harvest-runner.mjs --family benchmark --deterministic --dry-run   # self-test
+ */
+import { spawnSync } from "node:child_process";
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const TSCONFIG = path.join(REPO_ROOT, "tsconfig.json");
+const NATIVE_EXPORT_TS = path.join(
+  REPO_ROOT,
+  "packages/scenario-runner/src/native-export.ts",
+);
+const BENCH_TS = path.join(
+  REPO_ROOT,
+  "packages/benchmarks/framework/typescript/src/bench.ts",
+);
+
+const CLI_BACKENDS = new Set(["claude", "claude-sdk", "codex", "codex-sdk"]);
+
+function flag(name) {
+  return process.argv.includes(name);
+}
+function opt(name, fallback) {
+  const i = process.argv.indexOf(name);
+  return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
+}
+
+const DRY_RUN = flag("--dry-run");
+const DETERMINISTIC = flag("--deterministic");
+const RESUME = flag("--resume");
+const LIMIT = Number(opt("--limit", "0")) || 0;
+const FAMILY = opt("--family", "benchmark");
+const SHARD = opt("--shard", null);
+const [SHARD_I, SHARD_N] = SHARD ? SHARD.split("/").map(Number) : [0, 1];
+const ITEM_FILTER = opt("--item", null); // benchmark scenario id filter
+const LANE_FILTER = opt("--lane", null); // e2e lane path substring filter
+const ITEM_TIMEOUT_MS = Number(opt("--item-timeout-ms", "900000")) || 900000;
+const HARVEST_ROOT = path.resolve(
+  opt(
+    "--harvest-root",
+    path.join(REPO_ROOT, "reports", "training-harvest", "gpt55", "harvest"),
+  ),
+);
+
+// ── Corpus definitions ──────────────────────────────────────────────────────
+
+/**
+ * The framework benchmark's model-driving scenarios (bench.ts). Ordered
+ * cheapest-first (message count). The db/startup scenarios never call the model
+ * and cannot emit a trajectory, so they are excluded from the harvest corpus.
+ * `single-turn` scenarios send one message; the scaling/burst ones send many —
+ * keep them last (expensive on a live subscription).
+ */
+const BENCHMARK_SUITE = "framework";
+const BENCHMARK_SCENARIOS = [
+  "single-message",
+  "minimal-bootstrap",
+  "with-should-respond",
+  "with-should-respond-no-name",
+  "with-actions",
+  "multi-step",
+  "conversation-10",
+  "history-scaling-100",
+  "provider-scaling-10",
+  "concurrent-10",
+  "conversation-100",
+  "provider-scaling-50",
+  "provider-scaling-100",
+  "history-scaling-1000",
+  "concurrent-50",
+  "burst-100",
+  "burst-1000",
+  "history-scaling-10000",
+];
+
+/**
+ * E2E lanes that exercise the model and need ONLY a live LLM provider (no
+ * connector/cloud/device credentials). These app-core live-agent lanes gate on
+ * the app-core `selectLiveProvider()` helper, which this stage taught to
+ * recognize the cli backend — so they run on gpt-5.5-via-Codex. Ordered
+ * lightest-first (in-process createRealTestRuntime, no subprocess/browser).
+ * Heavier subprocess/Chrome lanes and the lifeops-harness lanes (which use a
+ * separate, still-cli-unaware selector) are documented in BENCHMARK_E2E_README.md.
+ * Env-gated lanes (connectors, cloud, vision, computeruse, shopify, gmail, device)
+ * are intentionally excluded.
+ */
+const E2E_HARVESTABLE_LANES = [
+  // lightest: in-process runtime, single generateText / harness turn
+  "packages/app-core/test/live-agent/cloud-providers.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/real-runtime-helpers.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/experience-extraction.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/action-invocation.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/page-scoped-chat.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/personality-routing.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/runtime-debug.live.e2e.test.ts",
+  // heavier: spawn a full startEliza subprocess (slower, but provider-only)
+  "packages/app-core/test/live-agent/agent-runtime.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/cloud-auth.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/database-conversation.live.e2e.test.ts",
+  "packages/app-core/test/live-agent/plugin-lifecycle.live.e2e.test.ts",
+];
+
+// ── Provider env (shared precedence with harvest-runner.mjs) ─────────────────
+
+function loadProviderEnv() {
+  const file = opt("--provider-env", process.env.HARVEST_PROVIDER_ENV_FILE);
+  if (file && existsSync(file)) {
+    return {
+      source: `file:${file}`,
+      env: JSON.parse(readFileSync(file, "utf8")),
+    };
+  }
+  if (DETERMINISTIC) {
+    return { source: "deterministic-enumerate-only", env: {} };
+  }
+  if (process.env.ELIZA_CHAT_VIA_CLI) {
+    return {
+      source: "inherited:ELIZA_CHAT_VIA_CLI",
+      env: {
+        ELIZA_CHAT_VIA_CLI: process.env.ELIZA_CHAT_VIA_CLI,
+        ...(process.env.ELIZA_CLI_CODEX_MODEL
+          ? { ELIZA_CLI_CODEX_MODEL: process.env.ELIZA_CLI_CODEX_MODEL }
+          : {}),
+      },
+    };
+  }
+  const apiKeys = [
+    "OPENAI_API_KEY",
+    "CEREBRAS_API_KEY",
+    "ANTHROPIC_API_KEY",
+    "GROQ_API_KEY",
+    "OPENROUTER_API_KEY",
+    "GOOGLE_GENERATIVE_AI_API_KEY",
+  ].filter((k) => process.env[k]);
+  if (apiKeys.length > 0) return { source: `inherited:${apiKeys[0]}`, env: {} };
+  throw new Error(
+    "no provider configured: pass --provider-env <s1.json>, --deterministic, or export ELIZA_CHAT_VIA_CLI / an API key",
+  );
+}
+
+/**
+ * Fold in the env vars the CLI route needs end-to-end. A bare provider-env file
+ * ({ELIZA_CHAT_VIA_CLI, ELIZA_CLI_CODEX_MODEL}) omits ELIZA_PLANNER_NATIVE_TOOLS,
+ * without which the free-text CLI cannot serve the planner (text-planner mode).
+ */
+function withCliRunEnv(providerEnv) {
+  const backend = (providerEnv.ELIZA_CHAT_VIA_CLI ?? "").trim().toLowerCase();
+  if (!CLI_BACKENDS.has(backend)) return { ...providerEnv };
+  return {
+    ELIZA_PLANNER_NATIVE_TOOLS: "0",
+    ELIZA_TRAJECTORY_RECORDING: "1",
+    ...providerEnv,
+  };
+}
+
+function slug(s) {
+  return s.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "");
+}
+
+/** Convert RecordedTrajectory JSON under <runDir>/trajectories → eliza_native_v1 JSONL. */
+function nativeExport(runDir, outPath) {
+  const script =
+    `import { exportScenarioNativeJsonl } from ${JSON.stringify(`file://${NATIVE_EXPORT_TS}`)};` +
+    `const rows = exportScenarioNativeJsonl(${JSON.stringify(runDir)}, ${JSON.stringify(outPath)});` +
+    `process.stdout.write(\`NATIVE_ROWS \${rows}\\n\`);`;
+  const res = spawnSync("bun", ["--conditions", "eliza-source", "-e", script], {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: 240000,
+    env: process.env,
+  });
+  const m = /NATIVE_ROWS (\d+)/.exec(res.stdout || "");
+  return m
+    ? Number(m[1])
+    : existsSync(outPath)
+      ? readFileSync(outPath, "utf8")
+          .split("\n")
+          .filter((l) => l.trim()).length
+      : 0;
+}
+
+// ── Benchmark family ─────────────────────────────────────────────────────────
+
+function runBenchmarkItem(scenarioId, runEnv) {
+  const itemDir = path.join(
+    HARVEST_ROOT,
+    "benchmark",
+    BENCHMARK_SUITE,
+    slug(scenarioId),
+  );
+  const trajDir = path.join(itemDir, "run", "trajectories");
+  mkdirSync(trajDir, { recursive: true });
+  const resultPath = path.join(itemDir, "result.json");
+  const args = [
+    "--conditions",
+    "eliza-source",
+    "--tsconfig-override",
+    TSCONFIG,
+    BENCH_TS,
+    "--real-llm",
+    `--scenarios=${scenarioId}`,
+    "--iterations=1",
+    "--warmup=0",
+    `--output=${resultPath}`,
+  ];
+  const res = spawnSync("bun", args, {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: ITEM_TIMEOUT_MS,
+    env: {
+      ...process.env,
+      ...runEnv,
+      LOG_LEVEL: "info",
+      ELIZA_TRAJECTORY_DIR: trajDir,
+    },
+  });
+  writeFileSync(path.join(itemDir, "stdout.log"), res.stdout || "");
+  writeFileSync(path.join(itemDir, "stderr.log"), res.stderr || "");
+
+  const nativePath = path.join(itemDir, "native.jsonl");
+  const rows = nativeExport(path.join(itemDir, "run"), nativePath);
+
+  let status = "error";
+  if (existsSync(resultPath)) {
+    try {
+      const result = JSON.parse(readFileSync(resultPath, "utf8"));
+      const ran = result.scenarios?.[scenarioId];
+      status = ran && rows > 0 ? "passed" : ran ? "no-trajectory" : "error";
+    } catch {
+      status = "error";
+    }
+  }
+  const verdict = {
+    family: "benchmark",
+    suite: BENCHMARK_SUITE,
+    item: scenarioId,
+    provider: runEnv.ELIZA_CHAT_VIA_CLI
+      ? `cli:${runEnv.ELIZA_CHAT_VIA_CLI}`
+      : "api-key",
+    model: runEnv.ELIZA_CLI_CODEX_MODEL ?? null,
+    status,
+    rows,
+    exitCode: res.status,
+    trajectoryFormat: "eliza_native_v1",
+    resultPath: existsSync(resultPath) ? "result.json" : null,
+    nativePath: existsSync(nativePath) ? "native.jsonl" : null,
+  };
+  writeVerdict(itemDir, verdict);
+  return verdict;
+}
+
+// ── E2E family ───────────────────────────────────────────────────────────────
+
+function runE2eItem(laneRel, runEnv) {
+  // These lanes run through the package's own vitest config (which mirrors the
+  // workspace `exports` so @elizaos/* resolves to source with dist absent), via
+  // the shared run-vitest.mjs wrapper (resolves an external Node — the codex
+  // bundled Node cannot run Vitest). The harvestable set is app-core-only.
+  const pkgDir = path.join(REPO_ROOT, laneRel.split("/").slice(0, 2).join("/"));
+  const runVitest = path.join(REPO_ROOT, "packages/scripts/run-vitest.mjs");
+  const itemDir = path.join(HARVEST_ROOT, "e2e", slug(laneRel));
+  const trajDir = path.join(itemDir, "run", "trajectories");
+  mkdirSync(trajDir, { recursive: true });
+  const laneAbs = path.join(REPO_ROOT, laneRel);
+  const res = spawnSync(
+    "node",
+    [
+      runVitest,
+      "run",
+      // The default config excludes *.live.e2e.test.ts wholesale; this harvest
+      // config surfaces the test/live-agent lanes (inherits the default @elizaos
+      // source aliases). See packages/app-core/vitest.harvest-live-agent.config.ts.
+      "--config",
+      "vitest.harvest-live-agent.config.ts",
+      laneAbs,
+      "--reporter=verbose",
+    ],
+    {
+      cwd: pkgDir,
+      encoding: "utf8",
+      timeout: ITEM_TIMEOUT_MS,
+      env: {
+        ...process.env,
+        ...runEnv,
+        ELIZA_LIVE_TEST: "1",
+        // The default vitest.config.ts excludes live-agent e2e lanes unless this
+        // is set (packages/app-core/vitest.config.ts `includeLiveE2e`).
+        ELIZA_INCLUDE_LIVE_E2E: "1",
+        ELIZA_SAVE_TRAJECTORIES: "1",
+        ELIZA_TRAJECTORY_DIR: trajDir,
+        // Harvest wants gpt-5.5 only: blank ambient keys so the cli branch in the
+        // app-core selectLiveProvider is chosen unambiguously.
+        CEREBRAS_API_KEY: "",
+        OPENAI_API_KEY: "",
+      },
+    },
+  );
+  writeFileSync(path.join(itemDir, "stdout.log"), res.stdout || "");
+  writeFileSync(path.join(itemDir, "stderr.log"), res.stderr || "");
+
+  const nativePath = path.join(itemDir, "native.jsonl");
+  const rows = nativeExport(path.join(itemDir, "run"), nativePath);
+
+  const out = `${res.stdout || ""}\n${res.stderr || ""}`;
+  // The live-agent lanes eagerly import first-party plugins; in a dist-less
+  // worktree Vite cannot resolve their entries (build the workspace first —
+  // the nightly "real" lane does). Distinguish that environmental precondition
+  // from a genuine test failure or a clean provider-gated skip.
+  const buildBlocked = /Failed to resolve entry for package/i.test(out);
+  const skipped =
+    /No LLM provider|test\.skip|set ELIZA_LIVE_TEST/i.test(out) && rows === 0;
+  const status = buildBlocked
+    ? "blocked-workspace-build"
+    : res.status === 0 && rows > 0
+      ? "passed"
+      : res.status === 0 && skipped
+        ? "skipped-no-provider"
+        : res.status === 0
+          ? "passed-no-trajectory"
+          : "failed";
+  const notes = {
+    "blocked-workspace-build":
+      "live-agent lanes eagerly import first-party plugins; a dist-less worktree cannot resolve them. Run `bun run build` first (nightly real lane precondition). See BENCHMARK_E2E_README.md.",
+    "skipped-no-provider":
+      "e2e provider selector did not select a provider — see BENCHMARK_E2E_README.md.",
+  };
+  const verdict = {
+    family: "e2e",
+    lane: laneRel,
+    provider: runEnv.ELIZA_CHAT_VIA_CLI
+      ? `cli:${runEnv.ELIZA_CHAT_VIA_CLI}`
+      : "api-key",
+    model: runEnv.ELIZA_CLI_CODEX_MODEL ?? null,
+    status,
+    rows,
+    exitCode: res.status,
+    trajectoryFormat: "eliza_native_v1",
+    note: notes[status],
+  };
+  writeVerdict(itemDir, verdict);
+  return verdict;
+}
+
+/**
+ * Write the per-item verdict. The canonical eliza_native_v1 manifest
+ * (native.manifest.json) is written by the scenario-runner native-export itself,
+ * so the driver does not duplicate it.
+ */
+function writeVerdict(itemDir, verdict) {
+  writeFileSync(
+    path.join(itemDir, "verdict.json"),
+    JSON.stringify(verdict, null, 2),
+  );
+}
+
+// ── Main ─────────────────────────────────────────────────────────────────────
+
+function main() {
+  const provider = loadProviderEnv();
+  const runEnv = withCliRunEnv(provider.env);
+  mkdirSync(HARVEST_ROOT, { recursive: true });
+
+  let corpus;
+  if (FAMILY === "benchmark") {
+    corpus = BENCHMARK_SCENARIOS.filter(
+      (id) => !ITEM_FILTER || id === ITEM_FILTER,
+    ).map((id) => ({ id }));
+  } else if (FAMILY === "e2e") {
+    corpus = E2E_HARVESTABLE_LANES.filter(
+      (lane) => !LANE_FILTER || lane.includes(LANE_FILTER),
+    ).map((lane) => ({ lane }));
+  } else {
+    throw new Error(`unknown --family ${FAMILY} (expected benchmark|e2e)`);
+  }
+
+  const summary = {
+    startedAt: new Date().toISOString(),
+    family: FAMILY,
+    harvestRoot: HARVEST_ROOT,
+    providerSource: provider.source,
+    providerEnvKeys: Object.keys(runEnv),
+    totalDiscovered: corpus.length,
+    shard: `${SHARD_I}/${SHARD_N}`,
+    dryRun: DRY_RUN,
+    limit: LIMIT,
+    items: [],
+  };
+
+  let ran = 0;
+  for (let gi = 0; gi < corpus.length; gi += 1) {
+    if (SHARD_N > 1 && gi % SHARD_N !== SHARD_I) continue;
+    if (LIMIT && ran >= LIMIT) break;
+    const item = corpus[gi];
+    const id = item.id ?? item.lane;
+    const itemDir =
+      FAMILY === "benchmark"
+        ? path.join(HARVEST_ROOT, "benchmark", BENCHMARK_SUITE, slug(id))
+        : path.join(HARVEST_ROOT, "e2e", slug(id));
+    if (RESUME && existsSync(path.join(itemDir, "verdict.json"))) {
+      console.log(`[resume] skip (already harvested) ${FAMILY} :: ${id}`);
+      continue;
+    }
+    ran += 1;
+    if (DRY_RUN || DETERMINISTIC) {
+      console.log(`[dry-run] would harvest ${FAMILY} :: ${id}`);
+      summary.items.push({ item: id, gi, planned: true });
+      continue;
+    }
+    console.log(`[harvest #${gi}] ${FAMILY} :: ${id}`);
+    const verdict =
+      FAMILY === "benchmark"
+        ? runBenchmarkItem(id, runEnv)
+        : runE2eItem(id, runEnv);
+    summary.items.push(verdict);
+    console.log(
+      `[harvest]   → status=${verdict.status} rows=${verdict.rows} exit=${verdict.exitCode}`,
+    );
+  }
+
+  summary.finishedAt = new Date().toISOString();
+  summary.count = summary.items.length;
+  const summaryPath = path.join(
+    HARVEST_ROOT,
+    `harvest-${FAMILY}-summary-${DRY_RUN || DETERMINISTIC ? "dryrun" : "run"}-${Date.now()}.json`,
+  );
+  writeFileSync(summaryPath, JSON.stringify(summary, null, 2));
+  console.log(`\nsummary → ${summaryPath}`);
+  console.log(
+    `family=${FAMILY} items=${summary.count} provider=${provider.source}`,
+  );
+}
+
+main();

scripts/training-harvest/build-manifest.mjs

@@ -0,0 +1,297 @@
+#!/usr/bin/env node
+/**
+ * gpt-5.5 trajectory-training pipeline — Stage 2 corpus manifest builder.
+ *
+ * Enumerates the ENTIRE elizaOS test/eval corpus and emits a machine-readable
+ * manifest describing, per family, every item, the exact trajectory-emitting
+ * run command, and where the trajectory lands. Consumed by harvest-runner.mjs.
+ *
+ * This does NOT run any scenario/benchmark/e2e. It only discovers + counts.
+ *
+ * Families:
+ *   scenario   — @elizaos/scenario-runner drives a real AgentRuntime + PGLite.
+ *                Emits eliza_native_v1 trajectories natively via --export-native.
+ *   benchmark  — packages/benchmarks orchestrator (registry/commands.py).
+ *                Emits per-benchmark result JSON today; eliza_native_v1 requires
+ *                wiring the adapter runtime's trajectory recorder (see notes).
+ *   e2e        — *.live.e2e.test.ts / *.real.e2e.test.ts vitest lanes that drive
+ *                a real runtime. Trajectory capture requires ELIZA_SAVE_TRAJECTORIES
+ *                + ELIZA_TRAJECTORY_DIR then native-export conversion (see notes).
+ *
+ * Usage:
+ *   node scripts/training-harvest/build-manifest.mjs [--out <path>]
+ */
+import { execFileSync } from "node:child_process";
+import { existsSync, readdirSync, writeFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+import { listPackages } from "../../packages/scripts/lib/workspaces.mjs";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const PACKAGES = path.join(REPO_ROOT, "packages");
+
+function arg(name, fallback) {
+  const i = process.argv.indexOf(name);
+  return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
+}
+
+/** Recursively count files matching a suffix under a dir. */
+function countFiles(dir, suffix) {
+  if (!existsSync(dir)) return [];
+  const out = [];
+  const walk = (d) => {
+    for (const entry of readdirSync(d, { withFileTypes: true })) {
+      if (entry.name.startsWith("_")) continue; // scenario loader skips these
+      const full = path.join(d, entry.name);
+      if (entry.isDirectory()) walk(full);
+      else if (entry.name.endsWith(suffix)) out.push(full);
+    }
+  };
+  walk(dir);
+  return out;
+}
+
+// Measured expansion multipliers (eliza-scenarios list --count-scenarios).
+// `existing` = base scenario count, `total` = with persona/paraphrase expansion.
+// Sampled live this run; embedded so the manifest is self-describing without a
+// slow re-boot of the loader on every rebuild.
+const MEASURED_EXPANSION = {
+  "packages/test/scenarios": { existing: 707, total: 7777 },
+  "plugins/plugin-personal-assistant/test/scenarios": {
+    existing: 197,
+    total: 2167,
+  },
+  "plugins/plugin-app-control/test/scenarios": { existing: 15, total: 165 },
+  "plugins/plugin-health/test/scenarios": { existing: 8, total: 88 },
+  "plugins/plugin-cloud-apps/test/scenarios": { existing: 1, total: 11 },
+  "plugins/plugin-agent-orchestrator/test/scenarios": {
+    existing: 8,
+    total: 88,
+  },
+};
+
+// The test-harness's canonical corpus root. It keeps scenarios directly under
+// `scenarios/` (not `test/scenarios/`), so the per-package convention glob below
+// does not reach it; include it explicitly. This is the scenario-runner's
+// DEFAULT_SCENARIO_ROOT, a well-known path constant — not a plugin coupling.
+const DEFAULT_SCENARIO_ROOT = "packages/test/scenarios";
+
+// Every workspace package's `test/scenarios` dir that exists on disk, discovered
+// through the shared workspace seam (#12332) rather than a hardcoded plugin list.
+// `scenarioFamily` skips any dir with zero `.scenario.ts` files, so a package
+// without scenarios contributes nothing — adding or removing a plugin with
+// scenarios updates the corpus with no edit to this file.
+const SCENARIO_DIRS = [
+  DEFAULT_SCENARIO_ROOT,
+  ...listPackages({ repoRoot: REPO_ROOT }).map((pkg) =>
+    path.posix.join(pkg.dir, "test", "scenarios"),
+  ),
+]
+  .filter((rel) => existsSync(path.join(REPO_ROOT, rel)))
+  .filter((rel, index, all) => all.indexOf(rel) === index)
+  .sort((a, b) => a.localeCompare(b));
+
+const SCENARIO_CLI = "packages/scenario-runner/src/cli.ts";
+
+function scenarioFamily() {
+  const items = [];
+  for (const rel of SCENARIO_DIRS) {
+    const dir = path.join(REPO_ROOT, rel);
+    const files = countFiles(dir, ".scenario.ts");
+    if (files.length === 0) continue;
+    const expansion = MEASURED_EXPANSION[rel] ?? {
+      existing: files.length,
+      total: files.length * 10,
+    };
+    items.push({
+      id: rel.replace(/[/]/g, "__"),
+      dir: rel,
+      scenarioFiles: files.length,
+      baseScenarios: expansion.existing,
+      expandedScenarios: expansion.total,
+      // The driver enumerates concrete scenario ids at run time via `list`.
+      discover: {
+        cmd: "bun",
+        args: [
+          "--conditions",
+          "eliza-source",
+          "--tsconfig-override",
+          "<TSCONFIG>",
+          SCENARIO_CLI,
+          "list",
+          rel,
+        ],
+        parse: "json-lines-scenario-ids",
+      },
+    });
+  }
+  return {
+    kind: "scenario",
+    emitsTrajectory: "native",
+    trajectoryFormat: "eliza_native_v1",
+    runnerCli: SCENARIO_CLI,
+    tsconfig: "tsconfig.json",
+    // <ID>, <REPORT>, <RUNDIR>, <NATIVE> are substituted per item by the driver.
+    runInvocationTemplate: [
+      "bun",
+      "--conditions",
+      "eliza-source",
+      "--tsconfig-override",
+      "<TSCONFIG>",
+      SCENARIO_CLI,
+      "run",
+      "<DIR>",
+      "--scenario",
+      "<ID>",
+      "--report",
+      "<REPORT>",
+      "--run-dir",
+      "<RUNDIR>",
+      "--export-native",
+      "<NATIVE>",
+    ],
+    trajectoryLands: {
+      report:
+        "<RUNDIR>/report.json (or --report path): aggregate + per-scenario status",
+      perTurn:
+        "<RUNDIR>/trajectories/<agentId>/<trajId>.json (RecordedTrajectory)",
+      native:
+        "<NATIVE>: eliza_native_v1 JSONL (rows carry scenarioStatus + judgeScore)",
+      manifest: "<NATIVE>.manifest.json",
+    },
+    verdictSource:
+      "report .scenarios[].status === 'passed' | native row.scenarioStatus",
+    providerSeam:
+      "ELIZA_CHAT_VIA_CLI=codex → provider 'cli', model gpt-5.5, plugin @elizaos/plugin-cli-inference, reads ~/.codex/auth.json",
+    items,
+  };
+}
+
+function benchmarkFamily() {
+  let adapters = [];
+  let rawList = "";
+  try {
+    rawList = execFileSync(
+      "python3",
+      ["-m", "benchmarks.orchestrator", "list-benchmarks"],
+      { cwd: PACKAGES, encoding: "utf8", timeout: 120000 },
+    );
+  } catch (err) {
+    // list-benchmarks exits non-zero when there are "uncovered" benchmark
+    // directories, but still prints the full adapter list to stdout. Use it.
+    rawList = (err && err.stdout ? String(err.stdout) : "") || "";
+  }
+  for (const line of rawList.split("\n")) {
+    const m = line.match(/^-\s+(\S+)\s+dir=(\S+)\s+cwd=(.+)$/);
+    if (m) adapters.push({ id: m[1], dir: m[2], cwd: m[3].trim() });
+  }
+  if (adapters.length === 0)
+    adapters = [
+      { error: "orchestrator list-benchmarks produced no parseable adapters" },
+    ];
+  return {
+    kind: "benchmark",
+    emitsTrajectory: "wiring-needed",
+    trajectoryFormat: "eliza_native_v1 (after wiring)",
+    source: "packages/benchmarks/registry/commands.py",
+    listCommand:
+      "python3 -m benchmarks.orchestrator list-benchmarks (cwd=packages)",
+    runInvocationTemplate:
+      "python3 -m benchmarks.orchestrator run --benchmarks <ID> --provider cli --model gpt-5.5 (cwd=packages)",
+    resultLands:
+      "packages/benchmarks/benchmark_results/** (per-benchmark JSON; locate_result() in registry/commands.py). GITIGNORED.",
+    trajectoryWiring:
+      "The ~25 eliza-adapter-routed benchmarks boot a real AgentRuntime (serves /api/benchmark/message). Set ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR=<dir> on the adapter runtime, then run packages/scenario-runner native-export over <dir> to convert RecordedTrajectory JSON → eliza_native_v1. Non-eliza-adapter benchmarks (standard/*, python-only) do NOT boot the runtime and cannot emit native trajectories.",
+    providerSeam:
+      "orchestrator --provider cli maps to ELIZA_CHAT_VIA_CLI=codex for eliza-adapter benchmarks; verify per-adapter provider plumbing before Stage 2.",
+    adapterCount: adapters.filter((a) => a.id).length,
+    adapters,
+  };
+}
+
+function e2eFamily() {
+  const roots = ["packages", "plugins"];
+  const live = [];
+  const walk = (d) => {
+    if (!existsSync(d)) return;
+    for (const entry of readdirSync(d, { withFileTypes: true })) {
+      if (entry.name === "node_modules" || entry.name === "dist") continue;
+      const full = path.join(d, entry.name);
+      if (entry.isDirectory()) walk(full);
+      else if (
+        entry.name.endsWith(".live.e2e.test.ts") ||
+        entry.name.endsWith(".real.e2e.test.ts")
+      )
+        live.push(path.relative(REPO_ROOT, full));
+    }
+  };
+  for (const r of roots) walk(path.join(REPO_ROOT, r));
+  return {
+    kind: "e2e",
+    emitsTrajectory: "wiring-needed",
+    trajectoryFormat: "eliza_native_v1 (after wiring)",
+    runInvocationTemplate:
+      "ELIZA_SAVE_TRAJECTORIES=1 ELIZA_TRAJECTORY_DIR=<dir> bun --conditions eliza-source vitest run <testFile>",
+    trajectoryWiring:
+      "These vitest lanes drive a real AgentRuntime via createScenarioRuntime/real-runtime helpers. The runtime's JsonFileTrajectoryRecorder writes RecordedTrajectory JSON when ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR are set. Convert with scenario-runner native-export. Verdict = vitest pass/fail per file (coarser than per-scenario).",
+    providerSeam:
+      "Same ELIZA_CHAT_VIA_CLI=codex seam; the live-provider helper (packages/app-core/test/helpers/live-provider.ts) already recognizes CLI backends.",
+    liveLaneCount: live.length,
+    lanes: live.sort(),
+    scriptedRealServices: [
+      "packages/scenario-runner/scripts/real-llm-attachment-smoke.mjs",
+      "packages/scenario-runner/scripts/real-service-audio-roundtrip.mjs",
+      "packages/scenario-runner/scripts/real-service-voice-e2e.mjs",
+    ],
+  };
+}
+
+const manifest = {
+  schema: "gpt55_harvest_manifest",
+  schemaVersion: 1,
+  generatedAt: new Date().toISOString(),
+  repoRoot: REPO_ROOT,
+  goal: "Run every elizaOS scenario+benchmark+e2e through gpt-5.5 (Codex subscription), harvest correct eliza_native_v1 trajectories, GEPA-repair failures, fine-tune on Nebius.",
+  provider: {
+    mechanism:
+      "ELIZA_CHAT_VIA_CLI CLI-subscription backend (packages/core/src/testing/live-provider.ts selectCliProvider)",
+    backend: "codex",
+    model: "gpt-5.5",
+    modelOverrideEnv: "ELIZA_CLI_CODEX_MODEL",
+    plugin: "@elizaos/plugin-cli-inference",
+    credentialsPath:
+      "~/.codex/auth.json (ChatGPT-OAuth; eliza never sees the token)",
+    env: { ELIZA_CHAT_VIA_CLI: "codex", ELIZA_CLI_CODEX_MODEL: "gpt-5.5" },
+    note: "Stage-1 leg S1 proves ONE real scenario through this seam live. The driver consumes S1's proven provider env verbatim.",
+  },
+  trajectoryFormat: {
+    name: "eliza_native_v1",
+    definedIn:
+      "packages/core/src/services/trajectory-types.ts (ElizaNativeTrajectoryRow)",
+    contract: "packages/training/docs/dataset/CANONICAL_RECORD.md",
+    converter:
+      "packages/scenario-runner/src/native-export.ts (exportScenarioNativeJsonl)",
+    trainingPrep:
+      "packages/training/scripts/prepare_eliza1_trajectory_dataset.py",
+  },
+  families: {
+    scenario: scenarioFamily(),
+    benchmark: benchmarkFamily(),
+    e2e: e2eFamily(),
+  },
+};
+
+const outPath = arg("--out", path.join(__dirname, "manifest.json"));
+writeFileSync(outPath, JSON.stringify(manifest, null, 2));
+
+const s = manifest.families.scenario;
+const sBase = s.items.reduce((a, i) => a + i.baseScenarios, 0);
+const sExp = s.items.reduce((a, i) => a + i.expandedScenarios, 0);
+const sFiles = s.items.reduce((a, i) => a + i.scenarioFiles, 0);
+process.stdout.write(
+  `manifest → ${outPath}\n` +
+    `scenario family: ${s.items.length} dirs, ${sFiles} files, ${sBase} base scenarios, ${sExp} expanded\n` +
+    `benchmark family: ${manifest.families.benchmark.adapterCount} adapters\n` +
+    `e2e family: ${manifest.families.e2e.liveLaneCount} live lanes\n`,
+);

scripts/training-harvest/harvest-runner.mjs

@@ -0,0 +1,300 @@
+#!/usr/bin/env node
+/**
+ * gpt-5.5 trajectory-training pipeline — Stage 2 harvest driver.
+ *
+ * Given the Stage-2 manifest (build-manifest.mjs) and the Stage-1 provider
+ * incantation, iterate the corpus and run each item, capturing its
+ * eliza_native_v1 trajectory + pass/fail verdict into a single harvest tree:
+ *
+ *   <harvestRoot>/<family>/<item>/
+ *       report.json      scenario aggregate report (per-scenario status)
+ *       native.jsonl     eliza_native_v1 model-boundary rows (scenarioStatus+judgeScore)
+ *       native.jsonl.manifest.json
+ *       verdict.json     { item, status: "passed"|"failed"|"skipped"|"error", rows, judgeScore }
+ *       run/             per-turn RecordedTrajectory JSON (<run>/trajectories/**)
+ *       stdout.log / stderr.log
+ *
+ * PROVIDER PARAMETERIZATION (consumes Stage-1 output):
+ *   The provider env is injected, never hard-coded. Precedence:
+ *     1. --provider-env <file.json>  (a JSON object of env vars; what S1 writes)
+ *     2. $HARVEST_PROVIDER_ENV_FILE  (same, via env)
+ *     3. current process env if it already carries ELIZA_CHAT_VIA_CLI / an API key
+ *     4. --deterministic  → SCENARIO_USE_LLM_PROXY=1 (offline driver self-test)
+ *   For the real Stage-2 run, S1 emits { "ELIZA_CHAT_VIA_CLI": "codex",
+ *   "ELIZA_CLI_CODEX_MODEL": "gpt-5.5" }.
+ *
+ * This driver is SCOPED to the scenario family (the only family that emits
+ * eliza_native_v1 natively). Benchmark + e2e families require the trajectory
+ * wiring noted in the manifest and are enumerated but not executed here.
+ *
+ * Usage:
+ *   node harvest-runner.mjs --provider-env s1-provider.json [--family scenario]
+ *       [--dir <manifest.dir>] [--limit N] [--harvest-root <dir>] [--dry-run]
+ *   node harvest-runner.mjs --deterministic --limit 3 --dry-run   # driver self-test
+ */
+import { spawnSync } from "node:child_process";
+import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+
+function flag(name) {
+  return process.argv.includes(name);
+}
+function opt(name, fallback) {
+  const i = process.argv.indexOf(name);
+  return i >= 0 && process.argv[i + 1] ? process.argv[i + 1] : fallback;
+}
+
+const DRY_RUN = flag("--dry-run");
+const DETERMINISTIC = flag("--deterministic");
+const LIMIT = Number(opt("--limit", "0")) || 0; // 0 = no limit
+const FAMILY = opt("--family", "scenario");
+// --shard i/n : deterministic split of the discovered scenario list across N
+// parallel workers (worker i takes items where globalIndex % N === i). Lets the
+// big packages/test/scenarios dir run across several codex workers at once.
+const SHARD = opt("--shard", null); // "i/n"
+const [SHARD_I, SHARD_N] = SHARD
+  ? SHARD.split("/").map((x) => Number(x))
+  : [0, 1];
+// --resume : skip an item whose verdict.json already exists, so a killed /
+// rate-limited harvest picks up where it left off without re-spending on
+// already-captured trajectories.
+const RESUME = flag("--resume");
+const DIR_FILTER = opt("--dir", null);
+const MANIFEST_PATH = opt("--manifest", path.join(__dirname, "manifest.json"));
+const HARVEST_ROOT = path.resolve(
+  opt(
+    "--harvest-root",
+    path.join(REPO_ROOT, "reports", "training-harvest", "gpt55", "harvest"),
+  ),
+);
+const TSCONFIG = path.join(REPO_ROOT, "tsconfig.json");
+const SCENARIO_TIMEOUT_MS =
+  Number(opt("--item-timeout-ms", "300000")) || 300000;
+
+/** Load the provider env object per the precedence documented above. */
+function loadProviderEnv() {
+  const file = opt("--provider-env", process.env.HARVEST_PROVIDER_ENV_FILE);
+  if (file && existsSync(file)) {
+    const parsed = JSON.parse(readFileSync(file, "utf8"));
+    return { source: `file:${file}`, env: parsed };
+  }
+  if (DETERMINISTIC) {
+    return {
+      source: "deterministic-proxy",
+      env: { SCENARIO_USE_LLM_PROXY: "1" },
+    };
+  }
+  if (process.env.ELIZA_CHAT_VIA_CLI) {
+    return {
+      source: "inherited:ELIZA_CHAT_VIA_CLI",
+      env: {
+        ELIZA_CHAT_VIA_CLI: process.env.ELIZA_CHAT_VIA_CLI,
+        ...(process.env.ELIZA_CLI_CODEX_MODEL
+          ? { ELIZA_CLI_CODEX_MODEL: process.env.ELIZA_CLI_CODEX_MODEL }
+          : {}),
+      },
+    };
+  }
+  const apiKeys = [
+    "OPENAI_API_KEY",
+    "GROQ_API_KEY",
+    "ANTHROPIC_API_KEY",
+    "GOOGLE_GENERATIVE_AI_API_KEY",
+    "OPENROUTER_API_KEY",
+    "CEREBRAS_API_KEY",
+  ].filter((k) => process.env[k]);
+  if (apiKeys.length > 0) {
+    return { source: `inherited:${apiKeys[0]}`, env: {} };
+  }
+  throw new Error(
+    "no provider configured: pass --provider-env <s1.json>, --deterministic, or export ELIZA_CHAT_VIA_CLI / an API key",
+  );
+}
+
+/** Enumerate concrete scenario ids under a manifest scenario dir via `list`. */
+function discoverScenarioIds(dirRel, providerEnv) {
+  const args = [
+    "--conditions",
+    "eliza-source",
+    "--tsconfig-override",
+    TSCONFIG,
+    path.join(REPO_ROOT, "packages/scenario-runner/src/cli.ts"),
+    "list",
+    dirRel,
+  ];
+  const res = spawnSync("bun", args, {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: 240000,
+    env: { ...process.env, ...providerEnv },
+  });
+  const ids = [];
+  for (const line of (res.stdout || "").split("\n")) {
+    const t = line.trim();
+    // list prints bare scenario ids, one per line (plus a leading info log line).
+    if (t && !t.includes(" ") && !t.startsWith("[") && !t.startsWith("{"))
+      ids.push(t);
+  }
+  return [...new Set(ids)];
+}
+
+/** Run one scenario id, capturing report + native jsonl + verdict. */
+function runScenario(dirRel, id, providerEnv) {
+  const itemDir = path.join(HARVEST_ROOT, "scenario", slug(dirRel), slug(id));
+  mkdirSync(itemDir, { recursive: true });
+  const reportPath = path.join(itemDir, "report.json");
+  const runDir = path.join(itemDir, "run");
+  const nativePath = path.join(itemDir, "native.jsonl");
+  const args = [
+    "--conditions",
+    "eliza-source",
+    "--tsconfig-override",
+    TSCONFIG,
+    path.join(REPO_ROOT, "packages/scenario-runner/src/cli.ts"),
+    "run",
+    dirRel,
+    "--scenario",
+    id,
+    "--report",
+    reportPath,
+    "--run-dir",
+    runDir,
+    "--export-native",
+    nativePath,
+  ];
+  const res = spawnSync("bun", args, {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: SCENARIO_TIMEOUT_MS,
+    env: { ...process.env, ...providerEnv, ELIZA_SAVE_TRAJECTORIES: "1" },
+  });
+  writeFileSync(path.join(itemDir, "stdout.log"), res.stdout || "");
+  writeFileSync(path.join(itemDir, "stderr.log"), res.stderr || "");
+
+  const verdict = {
+    item: id,
+    dir: dirRel,
+    exitCode: res.status,
+    status: "error",
+    rows: 0,
+    judgeScore: null,
+    reportPath: existsSync(reportPath) ? reportPath : null,
+    nativePath: existsSync(nativePath) ? nativePath : null,
+  };
+  if (existsSync(reportPath)) {
+    try {
+      const report = JSON.parse(readFileSync(reportPath, "utf8"));
+      const sc =
+        (report.scenarios || []).find((x) => x.id === id) ||
+        report.scenarios?.[0];
+      if (sc) {
+        verdict.status = sc.status || "error";
+        if (typeof sc.judgeScore === "number")
+          verdict.judgeScore = sc.judgeScore;
+      }
+    } catch {
+      /* leave status=error; stderr.log holds detail */
+    }
+  }
+  if (existsSync(nativePath)) {
+    verdict.rows = readFileSync(nativePath, "utf8")
+      .split("\n")
+      .filter((l) => l.trim()).length;
+  }
+  writeFileSync(
+    path.join(itemDir, "verdict.json"),
+    JSON.stringify(verdict, null, 2),
+  );
+  return verdict;
+}
+
+function slug(s) {
+  return s.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "");
+}
+
+function main() {
+  const manifest = JSON.parse(readFileSync(MANIFEST_PATH, "utf8"));
+  const provider = loadProviderEnv();
+  mkdirSync(HARVEST_ROOT, { recursive: true });
+  const summary = {
+    startedAt: new Date().toISOString(),
+    manifest: MANIFEST_PATH,
+    harvestRoot: HARVEST_ROOT,
+    family: FAMILY,
+    providerSource: provider.source,
+    providerEnvKeys: Object.keys(provider.env),
+    dryRun: DRY_RUN,
+    limit: LIMIT,
+    items: [],
+  };
+
+  if (FAMILY !== "scenario") {
+    summary.note = `family '${FAMILY}' requires trajectory wiring (see manifest.families.${FAMILY}.trajectoryWiring); driver executes the scenario family only in Stage 1.`;
+    console.log(JSON.stringify(summary, null, 2));
+    return;
+  }
+
+  const dirs = manifest.families.scenario.items.filter(
+    (it) => !DIR_FILTER || it.dir === DIR_FILTER || it.id === DIR_FILTER,
+  );
+
+  // Flatten to a stable global list first so --shard splits deterministically
+  // across parallel workers and --resume can skip already-done items.
+  const allItems = [];
+  for (const dirItem of dirs) {
+    for (const id of discoverScenarioIds(dirItem.dir, provider.env)) {
+      allItems.push({ dir: dirItem.dir, id });
+    }
+  }
+  summary.totalDiscovered = allItems.length;
+  summary.shard = `${SHARD_I}/${SHARD_N}`;
+
+  let ran = 0;
+  for (let gi = 0; gi < allItems.length; gi += 1) {
+    if (SHARD_N > 1 && gi % SHARD_N !== SHARD_I) continue; // not this shard
+    if (LIMIT && ran >= LIMIT) break;
+    ran += 1;
+    const { dir, id } = allItems[gi];
+    if (RESUME) {
+      const doneMarker = path.join(
+        HARVEST_ROOT,
+        slug(dir),
+        slug(id),
+        "verdict.json",
+      );
+      if (existsSync(doneMarker)) {
+        console.log(`[resume] skip (already harvested) ${dir} :: ${id}`);
+        continue;
+      }
+    }
+    if (DRY_RUN) {
+      summary.items.push({ dir, id, gi, planned: true });
+      console.log(`[dry-run] would harvest scenario ${dir} :: ${id}`);
+      continue;
+    }
+    console.log(`[harvest #${gi}] ${dir} :: ${id}`);
+    const verdict = runScenario(dir, id, provider.env);
+    summary.items.push(verdict);
+    console.log(
+      `[harvest]   → status=${verdict.status} rows=${verdict.rows} judge=${verdict.judgeScore ?? "n/a"} exit=${verdict.exitCode}`,
+    );
+  }
+
+  summary.finishedAt = new Date().toISOString();
+  summary.count = summary.items.length;
+  const summaryPath = path.join(
+    HARVEST_ROOT,
+    `harvest-summary-${DRY_RUN ? "dryrun" : "run"}-${Date.now()}.json`,
+  );
+  writeFileSync(summaryPath, JSON.stringify(summary, null, 2));
+  console.log(`\nsummary → ${summaryPath}`);
+  console.log(
+    `items=${summary.count} provider=${provider.source} dryRun=${DRY_RUN}`,
+  );
+}
+
+main();

scripts/training-harvest/manifest.json

@@ -0,0 +1,550 @@
+{
+  "schema": "gpt55_harvest_manifest",
+  "schemaVersion": 1,
+  "generatedAt": "2026-07-02T22:59:34.130Z",
+  "repoRoot": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline",
+  "goal": "Run every elizaOS scenario+benchmark+e2e through gpt-5.5 (Codex subscription), harvest correct eliza_native_v1 trajectories, GEPA-repair failures, fine-tune on Nebius.",
+  "provider": {
+    "mechanism": "ELIZA_CHAT_VIA_CLI CLI-subscription backend (packages/core/src/testing/live-provider.ts selectCliProvider)",
+    "backend": "codex",
+    "model": "gpt-5.5",
+    "modelOverrideEnv": "ELIZA_CLI_CODEX_MODEL",
+    "plugin": "@elizaos/plugin-cli-inference",
+    "credentialsPath": "~/.codex/auth.json (ChatGPT-OAuth; eliza never sees the token)",
+    "env": {
+      "ELIZA_CHAT_VIA_CLI": "codex",
+      "ELIZA_CLI_CODEX_MODEL": "gpt-5.5"
+    },
+    "note": "Stage-1 leg S1 proves ONE real scenario through this seam live. The driver consumes S1's proven provider env verbatim."
+  },
+  "trajectoryFormat": {
+    "name": "eliza_native_v1",
+    "definedIn": "packages/core/src/services/trajectory-types.ts (ElizaNativeTrajectoryRow)",
+    "contract": "packages/training/docs/dataset/CANONICAL_RECORD.md",
+    "converter": "packages/scenario-runner/src/native-export.ts (exportScenarioNativeJsonl)",
+    "trainingPrep": "packages/training/scripts/prepare_eliza1_trajectory_dataset.py"
+  },
+  "families": {
+    "scenario": {
+      "kind": "scenario",
+      "emitsTrajectory": "native",
+      "trajectoryFormat": "eliza_native_v1",
+      "runnerCli": "packages/scenario-runner/src/cli.ts",
+      "tsconfig": "tsconfig.json",
+      "runInvocationTemplate": [
+        "bun",
+        "--conditions",
+        "eliza-source",
+        "--tsconfig-override",
+        "<TSCONFIG>",
+        "packages/scenario-runner/src/cli.ts",
+        "run",
+        "<DIR>",
+        "--scenario",
+        "<ID>",
+        "--report",
+        "<REPORT>",
+        "--run-dir",
+        "<RUNDIR>",
+        "--export-native",
+        "<NATIVE>"
+      ],
+      "trajectoryLands": {
+        "report": "<RUNDIR>/report.json (or --report path): aggregate + per-scenario status",
+        "perTurn": "<RUNDIR>/trajectories/<agentId>/<trajId>.json (RecordedTrajectory)",
+        "native": "<NATIVE>: eliza_native_v1 JSONL (rows carry scenarioStatus + judgeScore)",
+        "manifest": "<NATIVE>.manifest.json"
+      },
+      "verdictSource": "report .scenarios[].status === 'passed' | native row.scenarioStatus",
+      "providerSeam": "ELIZA_CHAT_VIA_CLI=codex → provider 'cli', model gpt-5.5, plugin @elizaos/plugin-cli-inference, reads ~/.codex/auth.json",
+      "items": [
+        {
+          "id": "packages__test__scenarios",
+          "dir": "packages/test/scenarios",
+          "scenarioFiles": 713,
+          "baseScenarios": 707,
+          "expandedScenarios": 7777,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "packages/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "packages__scenario-runner__test__scenarios",
+          "dir": "packages/scenario-runner/test/scenarios",
+          "scenarioFiles": 42,
+          "baseScenarios": 42,
+          "expandedScenarios": 420,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "packages/scenario-runner/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "plugins__plugin-personal-assistant__test__scenarios",
+          "dir": "plugins/plugin-personal-assistant/test/scenarios",
+          "scenarioFiles": 197,
+          "baseScenarios": 197,
+          "expandedScenarios": 2167,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "plugins/plugin-personal-assistant/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "plugins__plugin-app-control__test__scenarios",
+          "dir": "plugins/plugin-app-control/test/scenarios",
+          "scenarioFiles": 15,
+          "baseScenarios": 15,
+          "expandedScenarios": 165,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "plugins/plugin-app-control/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "plugins__plugin-health__test__scenarios",
+          "dir": "plugins/plugin-health/test/scenarios",
+          "scenarioFiles": 8,
+          "baseScenarios": 8,
+          "expandedScenarios": 88,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "plugins/plugin-health/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "plugins__plugin-cloud-apps__test__scenarios",
+          "dir": "plugins/plugin-cloud-apps/test/scenarios",
+          "scenarioFiles": 1,
+          "baseScenarios": 1,
+          "expandedScenarios": 11,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "plugins/plugin-cloud-apps/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        },
+        {
+          "id": "plugins__plugin-agent-orchestrator__test__scenarios",
+          "dir": "plugins/plugin-agent-orchestrator/test/scenarios",
+          "scenarioFiles": 8,
+          "baseScenarios": 8,
+          "expandedScenarios": 88,
+          "discover": {
+            "cmd": "bun",
+            "args": [
+              "--conditions",
+              "eliza-source",
+              "--tsconfig-override",
+              "<TSCONFIG>",
+              "packages/scenario-runner/src/cli.ts",
+              "list",
+              "plugins/plugin-agent-orchestrator/test/scenarios"
+            ],
+            "parse": "json-lines-scenario-ids"
+          }
+        }
+      ]
+    },
+    "benchmark": {
+      "kind": "benchmark",
+      "emitsTrajectory": "wiring-needed",
+      "trajectoryFormat": "eliza_native_v1 (after wiring)",
+      "source": "packages/benchmarks/registry/commands.py",
+      "listCommand": "python3 -m benchmarks.orchestrator list-benchmarks (cwd=packages)",
+      "runInvocationTemplate": "python3 -m benchmarks.orchestrator run --benchmarks <ID> --provider cli --model gpt-5.5 (cwd=packages)",
+      "resultLands": "packages/benchmarks/benchmark_results/** (per-benchmark JSON; locate_result() in registry/commands.py). GITIGNORED.",
+      "trajectoryWiring": "The ~25 eliza-adapter-routed benchmarks boot a real AgentRuntime (serves /api/benchmark/message). Set ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR=<dir> on the adapter runtime, then run packages/scenario-runner native-export over <dir> to convert RecordedTrajectory JSON → eliza_native_v1. Non-eliza-adapter benchmarks (standard/*, python-only) do NOT boot the runtime and cannot emit native trajectories.",
+      "providerSeam": "orchestrator --provider cli maps to ELIZA_CHAT_VIA_CLI=codex for eliza-adapter benchmarks; verify per-adapter provider plumbing before Stage 2.",
+      "adapterCount": 53,
+      "adapters": [
+        {
+          "id": "abliteration-robustness",
+          "dir": "abliteration-robustness",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "action-calling",
+          "dir": "action-calling",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "adhdbench",
+          "dir": "adhdbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/adhdbench"
+        },
+        {
+          "id": "agentbench",
+          "dir": "agentbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/agentbench"
+        },
+        {
+          "id": "app-eval",
+          "dir": "app-eval",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/app-eval"
+        },
+        {
+          "id": "bfcl",
+          "dir": "bfcl",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "clawbench",
+          "dir": "clawbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/clawbench"
+        },
+        {
+          "id": "configbench",
+          "dir": "configbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/configbench"
+        },
+        {
+          "id": "context_bench",
+          "dir": "context-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "eliza_1",
+          "dir": "eliza-1",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/eliza-1"
+        },
+        {
+          "id": "eliza_replay",
+          "dir": "eliza-adapter",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/eliza-adapter"
+        },
+        {
+          "id": "experience",
+          "dir": "experience",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/experience"
+        },
+        {
+          "id": "framework",
+          "dir": "framework",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "gauntlet",
+          "dir": "gauntlet",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/gauntlet"
+        },
+        {
+          "id": "gsm8k",
+          "dir": "standard",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "hermes_swe_env",
+          "dir": "hermes-adapter",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
+        },
+        {
+          "id": "hermes_tblite",
+          "dir": "hermes-adapter",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
+        },
+        {
+          "id": "hermes_terminalbench_2",
+          "dir": "hermes-adapter",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
+        },
+        {
+          "id": "hermes_yc_bench",
+          "dir": "hermes-adapter",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/hermes-adapter"
+        },
+        {
+          "id": "humaneval",
+          "dir": "standard",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "hyperliquid_bench",
+          "dir": "HyperliquidBench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "interrupt_bench",
+          "dir": "interrupt-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/interrupt-bench"
+        },
+        {
+          "id": "lifeops_bench",
+          "dir": "lifeops-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/lifeops-bench"
+        },
+        {
+          "id": "mind2web",
+          "dir": "mind2web",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "mint",
+          "dir": "mint",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "mmau",
+          "dir": "mmau-audio",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/mmau-audio"
+        },
+        {
+          "id": "mmlu",
+          "dir": "standard",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "mt_bench",
+          "dir": "standard",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "openclaw_bench",
+          "dir": "openclaw-benchmark",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/openclaw-benchmark"
+        },
+        {
+          "id": "orchestrator_lifecycle",
+          "dir": "orchestrator_lifecycle",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "osworld",
+          "dir": "OSWorld",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/OSWorld"
+        },
+        {
+          "id": "personality_bench",
+          "dir": "personality-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/personality-bench"
+        },
+        {
+          "id": "realm",
+          "dir": "realm",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "recall_bench",
+          "dir": "recall-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/recall-bench"
+        },
+        {
+          "id": "rlm_bench",
+          "dir": "rlm-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/rlm-bench"
+        },
+        {
+          "id": "scambench",
+          "dir": "scambench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "social_alpha",
+          "dir": "social-alpha",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/social-alpha"
+        },
+        {
+          "id": "solana",
+          "dir": "solana",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "swe_bench",
+          "dir": "swe_bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "swe_bench_orchestrated",
+          "dir": "swe_bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "tau_bench",
+          "dir": "tau-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/tau-bench"
+        },
+        {
+          "id": "terminal_bench",
+          "dir": "terminal-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/terminal-bench"
+        },
+        {
+          "id": "three_agent_dialogue",
+          "dir": "three-agent-dialogue",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/three-agent-dialogue"
+        },
+        {
+          "id": "trajectory_replay",
+          "dir": "standard",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "trust",
+          "dir": "trust",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/trust"
+        },
+        {
+          "id": "vending_bench",
+          "dir": "vending-bench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/vending-bench"
+        },
+        {
+          "id": "vision_language",
+          "dir": "vision-language",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/vision-language"
+        },
+        {
+          "id": "visualwebbench",
+          "dir": "visualwebbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        },
+        {
+          "id": "voiceagentbench",
+          "dir": "voiceagentbench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voiceagentbench"
+        },
+        {
+          "id": "voicebench",
+          "dir": "voicebench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voicebench"
+        },
+        {
+          "id": "voicebench_quality",
+          "dir": "voicebench-quality",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/voicebench-quality"
+        },
+        {
+          "id": "webshop",
+          "dir": "webshop",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages/benchmarks/webshop"
+        },
+        {
+          "id": "woobench",
+          "dir": "woobench",
+          "cwd": "/Users/shawwalters/eliza-workspace/milady/eliza/.claude/worktrees/gpt55-training-pipeline/packages"
+        }
+      ]
+    },
+    "e2e": {
+      "kind": "e2e",
+      "emitsTrajectory": "wiring-needed",
+      "trajectoryFormat": "eliza_native_v1 (after wiring)",
+      "runInvocationTemplate": "ELIZA_SAVE_TRAJECTORIES=1 ELIZA_TRAJECTORY_DIR=<dir> bun --conditions eliza-source vitest run <testFile>",
+      "trajectoryWiring": "These vitest lanes drive a real AgentRuntime via createScenarioRuntime/real-runtime helpers. The runtime's JsonFileTrajectoryRecorder writes RecordedTrajectory JSON when ELIZA_SAVE_TRAJECTORIES=1 + ELIZA_TRAJECTORY_DIR are set. Convert with scenario-runner native-export. Verdict = vitest pass/fail per file (coarser than per-scenario).",
+      "providerSeam": "Same ELIZA_CHAT_VIA_CLI=codex seam; the live-provider helper (packages/app-core/test/helpers/live-provider.ts) already recognizes CLI backends.",
+      "liveLaneCount": 51,
+      "lanes": [
+        "packages/app-core/test/app/memory-relationships.real.e2e.test.ts",
+        "packages/app-core/test/app/qa-checklist.real.e2e.test.ts",
+        "packages/app-core/test/app/streaming-visible-text.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/action-invocation.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/agent-runtime.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/cloud-auth.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/cloud-login-persist.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/cloud-providers.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/config-routes.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/connector-health.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/conversation-deterministic.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/database-conversation.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/experience-extraction.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/farcaster-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/feishu-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/health-and-dev-stack.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/lens-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/matrix-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/nostr-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/page-scoped-chat.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/permissions-and-sensitive.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/personality-routing.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/plugin-lifecycle.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/real-runtime-helpers.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/runtime-debug.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/telegram-connector.live.e2e.test.ts",
+        "packages/app-core/test/live-agent/views-interact-ws-roundtrip.real.e2e.test.ts",
+        "packages/app-core/test/live-agent/views-routes.real.e2e.test.ts",
+        "plugins/plugin-computeruse/src/__tests__/runtime.live.e2e.test.ts",
+        "plugins/plugin-computeruse/test/computeruse.real.e2e.test.ts",
+        "plugins/plugin-documents/test/documents-api.live.e2e.test.ts",
+        "plugins/plugin-form/test/form-plugin.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/assistant-user-journeys.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/journey-cerebras-eval.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/knowledge-graph-service.real.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-activity-signals.remote.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-chat.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-gmail-chat.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-memory.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-screen-context.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/lifeops-signal.real.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/reminder-review-job.real.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/selfcontrol-chat.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/selfcontrol-desktop.live.e2e.test.ts",
+        "plugins/plugin-personal-assistant/test/selfcontrol-dev.live.e2e.test.ts",
+        "plugins/plugin-shopify/test/shopify-api.real.e2e.test.ts",
+        "plugins/plugin-sql/src/__tests__/integration/pglite-e2e.real.e2e.test.ts",
+        "plugins/plugin-task-coordinator/test/coding-agent-codex-artifact.live.e2e.test.ts",
+        "plugins/plugin-task-coordinator/test/orchestrator-workbench.live.e2e.test.ts",
+        "plugins/plugin-training/test/training-api.live.e2e.test.ts",
+        "plugins/plugin-vision/test/vision.real.e2e.test.ts"
+      ],
+      "scriptedRealServices": [
+        "packages/scenario-runner/scripts/real-llm-attachment-smoke.mjs",
+        "packages/scenario-runner/scripts/real-service-audio-roundtrip.mjs",
+        "packages/scenario-runner/scripts/real-service-voice-e2e.mjs"
+      ]
+    }
+  }
+}

scripts/training-harvest/s1-provider.example.json

@@ -0,0 +1,4 @@
+{
+  "ELIZA_CHAT_VIA_CLI": "codex",
+  "ELIZA_CLI_CODEX_MODEL": "gpt-5.5"
+}

scripts/training-harvest/stage3-gepa-sweep.mjs

@@ -0,0 +1,828 @@
+#!/usr/bin/env node
+import { spawnSync } from "node:child_process";
+/**
+ * Stage 3 — GEPA optimization sweep over the gpt-5.5 harvest.
+ *
+ * Reads the Stage-2 harvest tree
+ * (reports/training-harvest/gpt55/harvest/{scenario,benchmark}/…),
+ * selects the GEPA-optimizable failures (real agent failures, NOT connector/cred
+ * env-gates — decided from `report.json`, never stderr grep), groups them by the
+ * plugin-training GEPA task the scorer actually supports, builds one
+ * `eliza_native_v1` GEPA dataset per task, emits the exact `train` commands, and
+ * (with --run) executes them: GEPA → artifact → re-run the failing scenarios on
+ * gpt-5.5-via-Codex → capture the (hopefully now-passing) trajectories back into
+ * harvest/scenario/ so Stage 4 picks them up.
+ *
+ * WHY the dataset for a task contains PASSING (gold) rows, not the failing ones:
+ * the native backend deliberately EXCLUDES failed-scenario rows as gold
+ * (`native.ts` isFailedScenarioSignal, #8795) — a failure's wrong output must
+ * never be optimized *toward*. GEPA optimizes a task's SYSTEM PROMPT against that
+ * task's gold (input → expected) rows; the failing scenarios supply the *inputs*
+ * the optimized prompt then has to get right, and the confirmation is the re-run.
+ * So the per-task GEPA dataset = the passing scenarios' native rows for that task.
+ * (The `--baseline` is inferred from the dataset's own system message when not
+ * passed; GEPA rewrites that.)
+ *
+ *   Modes:
+ *     --dry-run    (default)  classify + group + build datasets + PRINT commands.
+ *     --run                   actually execute GEPA per task (EXPENSIVE — gated).
+ *     --run --only <task>     GEPA a single task (the small end-to-end proof).
+ *     --rerun-only            skip GEPA; only re-run failing scenarios against an
+ *                             already-produced artifact store (--state-dir).
+ *
+ *   Resumable: per-task dataset + per-task artifact + per-scenario rerun verdict
+ *   are all on disk; re-invoking skips completed steps unless --force.
+ *
+ * NEVER commits, never echoes secrets. Scoring model is Cerebras (fast, sanctioned);
+ * the flip *validation* re-run is gpt-5.5-via-Codex (S1 provider env).
+ */
+import {
+  existsSync,
+  mkdirSync,
+  readdirSync,
+  readFileSync,
+  writeFileSync,
+} from "node:fs";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const REPO_ROOT = path.resolve(__dirname, "..", "..");
+const EVIDENCE_ROOT = path.join(REPO_ROOT, "reports/training-harvest/gpt55");
+const HARVEST_ROOT = path.join(EVIDENCE_ROOT, "harvest");
+const TSCONFIG = path.join(REPO_ROOT, "tsconfig.json");
+const S1_PROVIDER_PATH = "/tmp/s1-provider-full.json";
+
+// The harvest tree dirs are slug(realDir) (slashes → "_"), which is NOT
+// reversible, so rebuild slug → realDir from the manifest to give the rerun the
+// real scenario-runner directory. Matches harvest-runner.mjs's slug().
+const slugKey = (s) =>
+  s.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "");
+const SLUG_TO_DIR = (() => {
+  const map = {};
+  try {
+    const manifest = JSON.parse(
+      readFileSync(path.join(__dirname, "manifest.json"), "utf8"),
+    );
+    for (const it of manifest.families?.scenario?.items ?? []) {
+      if (it.dir) map[slugKey(it.dir)] = it.dir;
+    }
+  } catch {
+    /* manifest optional — fall back to best-effort un-slug below */
+  }
+  return map;
+})();
+
+// Stage-3 working tree (datasets, artifacts, reruns) — NOT the harvest dirs.
+const STAGE3_ROOT = path.join(EVIDENCE_ROOT, "s3-gepa", "sweep");
+const DATASET_DIR = path.join(STAGE3_ROOT, "datasets");
+const RERUN_DIR = path.join(STAGE3_ROOT, "reruns");
+const DEFAULT_STATE_DIR = path.join(STAGE3_ROOT, "state"); // GEPA artifact store
+
+// -------- CLI --------
+const argv = process.argv.slice(2);
+const flag = (n) => argv.includes(n);
+const opt = (n, d) => {
+  const i = argv.indexOf(n);
+  return i >= 0 && argv[i + 1] ? argv[i + 1] : d;
+};
+const RUN = flag("--run");
+const RERUN_ONLY = flag("--rerun-only");
+const DRY_RUN = !RUN && !RERUN_ONLY; // dry-run is the default
+const FORCE = flag("--force");
+const ONLY_TASK = opt("--only", null);
+const STATE_DIR = path.resolve(opt("--state-dir", DEFAULT_STATE_DIR));
+const MAX_RERUN = Number(opt("--max-rerun", "0")) || 0; // 0 = all failing scenarios for the task
+const SCENARIO_TIMEOUT_MS = Number(opt("--timeout-ms", "300000")) || 300000;
+// GEPA optimizes a SHARED prompt — sample the gold pool rather than scoring all
+// 300+ full-context rows every candidate. 0 = no cap.
+const SAMPLE_ROWS = Number(opt("--sample-rows", "0")) || 0;
+const MAX_ROW_CHARS = Number(opt("--max-row-chars", "0")) || 0;
+
+/**
+ * native `task_type` (stamped on every harvested row) → GEPA `--task` id.
+ * The valid `--task` ids are ALL_TRAJECTORY_TRAINING_TASKS in
+ * plugins/plugin-training/src/core/trajectory-task-datasets.ts. `evaluation`
+ * rows are EVALUATOR model calls, not a planner prompt task, and there is no
+ * `evaluation` GEPA task — they are not GEPA targets (dropped from grouping).
+ */
+const TASK_TYPE_TO_GEPA_TASK = {
+  should_respond: "should_respond",
+  context_routing: "context_routing",
+  action_planner: "action_planner",
+  response: "response",
+  media_description: "media_description",
+  view_context: "view_context",
+  calendar_extract: "calendar_extract",
+  schedule_plan: "schedule_plan",
+  reminder_dispatch: "reminder_dispatch",
+  inbox_triage: "inbox_triage",
+  meeting_prep: "meeting_prep",
+  morning_brief: "morning_brief",
+  health_checkin: "health_checkin",
+  screentime_recap: "screentime_recap",
+  // evaluation: intentionally omitted — not a GEPA prompt task.
+};
+
+// Tasks with a dedicated scorer (else default token-overlap). Informational —
+// used to annotate the emitted plan so the operator knows which flips are exact.
+const TASKS_WITH_DEDICATED_SCORER = new Set([
+  "action_planner",
+  "view_context",
+  "calendar_extract",
+  "schedule_plan",
+  "reminder_dispatch",
+  "inbox_triage",
+  "meeting_prep",
+  "morning_brief",
+  "health_checkin",
+  "screentime_recap",
+]);
+
+// ---------------------------------------------------------------------------
+// FAILURE CLASSIFIER — report.json only (authoritative), never stderr.
+// Mirrors the taxonomy in the harvest evidence: ~58% connector/cred env-gated
+// (excluded), ~42% real agent failures (GEPA-optimizable).
+// ---------------------------------------------------------------------------
+
+/** Collect every failure `detail` string from a scenario report. */
+function failureDetails(report) {
+  const out = [];
+  for (const s of report.scenarios ?? []) {
+    if (s.status !== "failed") continue;
+    for (const a of s.failedAssertions ?? []) {
+      out.push(typeof a === "string" ? a : String(a?.detail ?? ""));
+    }
+    for (const t of s.turns ?? []) {
+      for (const a of t.failedAssertions ?? []) {
+        out.push(typeof a === "string" ? a : String(a?.detail ?? ""));
+      }
+    }
+  }
+  return out;
+}
+
+const CONNECTOR_CRED_MARKERS = [
+  "gmail mock",
+  "gmail action",
+  "gmail mock request",
+  "gmailbatchmodify",
+  "gmaildraftcreated",
+  "gmailmessagesent",
+  "connector dispatch",
+  "approval queue",
+  "mock request",
+  "push sent",
+  "no push",
+  "messagedelivered",
+  "draftexists",
+  "interventionrequest",
+  "device_intent",
+  "source_connector_not_found",
+  "connector_not_found",
+  "certify",
+  "session-revoked",
+  "session_revoked",
+  "rate-limited",
+  "rate_limited",
+  "helper-disconnected",
+  "disconnected",
+  "not currently available",
+];
+
+/**
+ * Classify a failed scenario from its report + id. Returns one of:
+ *   CONNECTOR_CRED   — env-gated (missing connector / credential / mock). EXCLUDE.
+ *   ACTION_SELECTION — wrong action chosen / wrong action args. GEPA-optimizable.
+ *   JUDGE_BELOW      — response judge score under threshold. GEPA-optimizable.
+ *   OUTPUT_FORMAT    — responseIncludes/Excludes / structured-field mismatch. GEPA.
+ *   OTHER            — real failure, no clean task mapping. Not GEPA-targeted here.
+ */
+function classifyFailure(scenarioId, report) {
+  const details = failureDetails(report).map((d) => d.toLowerCase());
+  const joined = details.join(" || ");
+  const idl = scenarioId.toLowerCase();
+
+  // Env-gate wins: connector/certify scenarios are excluded even if they also
+  // trip an action assertion (the action fails *because* the connector is gone).
+  if (
+    idl.includes("connector.") ||
+    idl.includes(".certify") ||
+    CONNECTOR_CRED_MARKERS.some((m) => joined.includes(m))
+  ) {
+    return "CONNECTOR_CRED";
+  }
+  if (
+    joined.includes("no selected action") ||
+    joined.includes("expectedactions") ||
+    joined.includes("selectedactionarguments") ||
+    joined.includes("actioncalled") ||
+    /expected \d+ call\(s\) to/.test(joined)
+  ) {
+    return "ACTION_SELECTION";
+  }
+  if (joined.includes("responsejudge") || /score \d/.test(joined)) {
+    return "JUDGE_BELOW";
+  }
+  if (
+    joined.includes("responseincludes") ||
+    joined.includes("responseexcludes") ||
+    joined.includes("structured argument")
+  ) {
+    return "OUTPUT_FORMAT";
+  }
+  return "OTHER";
+}
+
+const GEPA_OPTIMIZABLE = new Set([
+  "ACTION_SELECTION",
+  "JUDGE_BELOW",
+  "OUTPUT_FORMAT",
+]);
+
+// ---------------------------------------------------------------------------
+// Harvest walk
+// ---------------------------------------------------------------------------
+
+/** Every item dir under harvest/{scenario,benchmark}/<family>/<item>/. */
+function walkHarvestItems() {
+  const items = [];
+  for (const family of ["scenario", "benchmark"]) {
+    const famRoot = path.join(HARVEST_ROOT, family);
+    if (!existsSync(famRoot)) continue;
+    for (const familyDir of readdirSync(famRoot)) {
+      const fdPath = path.join(famRoot, familyDir);
+      let children;
+      try {
+        children = readdirSync(fdPath);
+      } catch {
+        continue;
+      }
+      for (const item of children) {
+        const itemDir = path.join(fdPath, item);
+        const verdictPath = path.join(itemDir, "verdict.json");
+        if (!existsSync(verdictPath)) continue;
+        items.push({ family, familyDir, item, itemDir, verdictPath });
+      }
+    }
+  }
+  return items;
+}
+
+function readJson(p) {
+  return JSON.parse(readFileSync(p, "utf8"));
+}
+
+/** Read native.jsonl rows (skip blank/malformed lines). */
+function readNativeRows(p) {
+  if (!existsSync(p)) return [];
+  const rows = [];
+  for (const line of readFileSync(p, "utf8").split("\n")) {
+    const t = line.trim();
+    if (!t) continue;
+    try {
+      rows.push(JSON.parse(t));
+    } catch {
+      /* skip malformed */
+    }
+  }
+  return rows;
+}
+
+const rowTaskType = (row) =>
+  row?.metadata?.task_type ?? row?.metadata?.taskType ?? null;
+const rowScenarioStatus = (row) =>
+  row?.scenarioStatus ??
+  row?.metadata?.scenario_status ??
+  row?.metadata?.scenarioStatus ??
+  null;
+
+// ---------------------------------------------------------------------------
+// Selection + grouping
+// ---------------------------------------------------------------------------
+
+function selectAndGroup() {
+  const items = walkHarvestItems();
+  // Per-GEPA-task accumulation.
+  const perTask = {}; // task -> { goldRows:[], failingScenarios:[{item,dir,itemDir,cat}] }
+  const stats = {
+    itemsScanned: items.length,
+    passed: 0,
+    failed: 0,
+    error: 0,
+    other: 0,
+    failClassCounts: {},
+    excludedConnectorCred: 0,
+  };
+  const ensureTask = (task) =>
+    (perTask[task] ??= { goldRows: [], failingScenarios: [] });
+
+  for (const it of items) {
+    let verdict;
+    try {
+      verdict = readJson(it.verdictPath);
+    } catch {
+      stats.error += 1;
+      continue;
+    }
+    const status = verdict.status;
+    if (status === "passed") {
+      stats.passed += 1;
+      // Gold rows: every passing native row buckets into its GEPA task.
+      for (const row of readNativeRows(`${it.itemDir}/native.jsonl`)) {
+        if (rowScenarioStatus(row) === "failed") continue; // belt + suspenders
+        const gepaTask = TASK_TYPE_TO_GEPA_TASK[rowTaskType(row)];
+        if (!gepaTask) continue;
+        ensureTask(gepaTask).goldRows.push(row);
+      }
+      continue;
+    }
+    if (status !== "failed") {
+      stats.other += 1;
+      continue;
+    }
+    stats.failed += 1;
+
+    const reportPath = `${it.itemDir}/report.json`;
+    if (!existsSync(reportPath)) {
+      stats.failClassCounts.NO_REPORT =
+        (stats.failClassCounts.NO_REPORT ?? 0) + 1;
+      continue;
+    }
+    let report;
+    try {
+      report = readJson(reportPath);
+    } catch {
+      stats.failClassCounts.BAD_REPORT =
+        (stats.failClassCounts.BAD_REPORT ?? 0) + 1;
+      continue;
+    }
+    const cat = classifyFailure(it.item, report);
+    stats.failClassCounts[cat] = (stats.failClassCounts[cat] ?? 0) + 1;
+    if (cat === "CONNECTOR_CRED") {
+      stats.excludedConnectorCred += 1;
+      continue;
+    }
+    if (!GEPA_OPTIMIZABLE.has(cat)) continue; // OTHER etc — not GEPA-targeted
+
+    // Which GEPA task does this failure implicate? Use the failing scenario's
+    // OWN native rows' task_type (already stamped), preferring the task whose
+    // failure category matches: ACTION_SELECTION → action_planner, else the
+    // dominant non-evaluation task_type present.
+    const failRows = readNativeRows(`${it.itemDir}/native.jsonl`);
+    const taskTypesPresent = new Set(
+      failRows.map(rowTaskType).filter((t) => TASK_TYPE_TO_GEPA_TASK[t]),
+    );
+    let targetTask = null;
+    if (cat === "ACTION_SELECTION" && taskTypesPresent.has("action_planner")) {
+      targetTask = "action_planner";
+    } else if (taskTypesPresent.has("action_planner")) {
+      targetTask = "action_planner";
+    } else if (taskTypesPresent.has("should_respond")) {
+      targetTask = "should_respond";
+    } else {
+      // first LifeOps/other mapped task present
+      for (const tt of taskTypesPresent) {
+        targetTask = TASK_TYPE_TO_GEPA_TASK[tt];
+        if (targetTask) break;
+      }
+    }
+    if (!targetTask) continue;
+    ensureTask(targetTask).failingScenarios.push({
+      item: it.item,
+      familyDir: it.familyDir,
+      itemDir: it.itemDir,
+      cat,
+    });
+  }
+  return { perTask, stats };
+}
+
+/**
+ * Slim a harvested request down to ONLY the fields the native backend + GEPA
+ * scorer actually consume, so the GEPA dataset fits under the Cerebras
+ * queue/TPM ceilings instead of shipping 96K-600K-char rows.
+ *
+ * WHAT THE PIPELINE READS (verified against native.ts rowToExample + the
+ * scorers in scoring.ts): `request.system`, `request.prompt`, and
+ * `request.messages` filtered to system/user/assistant roles. The scorer then
+ * composes only `system + "\n\n" + user` and runs it through the model; the
+ * expected output comes from `response.text` / `response.toolCalls`.
+ *
+ * WHAT IS PURE DEAD WEIGHT (never read, but dominates row size):
+ *   - `request.tools` — the full 38-53-tool catalog serialized per row
+ *     (~260K chars on the big action_planner rows). should_respond is a binary
+ *     respond/ignore decision that needs no tool schemas at all; action_planner
+ *     is scored on the ACTION NAME extracted from the model's text output, not
+ *     on re-issuing the tool schema.
+ *   - `request.toolChoice`, `request.providerOptions` (~31K chars on the big
+ *     should_respond rows) — transport knobs the scorer never sees.
+ *   - `tool`-role and other non-{system,user,assistant} messages — the
+ *     multi-turn tool-result transcript the parser skips.
+ *
+ * Dropping these does NOT change any score (the scorer's model input is
+ * identical) — it just removes the tokens that saturate the rate limits.
+ */
+const SCORED_MESSAGE_ROLES = new Set(["system", "user", "assistant"]);
+function slimRequest(req) {
+  const slim = {};
+  if (typeof req.system === "string" && req.system.length > 0) {
+    slim.system = req.system;
+  }
+  if (typeof req.prompt === "string" && req.prompt.trim()) {
+    slim.prompt = req.prompt;
+  }
+  if (Array.isArray(req.messages)) {
+    slim.messages = req.messages
+      .filter((m) => m && SCORED_MESSAGE_ROLES.has(m.role))
+      .map((m) => ({ role: m.role, content: m.content }));
+  }
+  return slim;
+}
+
+/**
+ * Build the per-task GEPA dataset: PASSING (gold) native rows for that task,
+ * written in the exact `eliza_native_v1` shape the native backend parses. The
+ * request is slimmed (see slimRequest) to the fields the parser + scorer read,
+ * keeping metadata so the failed-scenario quality guard stays live.
+ */
+function buildTaskDataset(task, goldRows) {
+  mkdirSync(DATASET_DIR, { recursive: true });
+  const outPath = path.join(DATASET_DIR, `${task}.gepa.jsonl`);
+  const seen = new Set();
+  const lines = [];
+  for (const row of goldRows) {
+    if (
+      row.boundary !== "vercel_ai_sdk.generateText" &&
+      row.boundary !== "vercel_ai_sdk.streamText"
+    )
+      continue;
+    const req = row.request ?? {};
+    const resp = row.response ?? {};
+    // Must have a usable user turn + expected output (mirrors rowToExample).
+    const hasUser =
+      (Array.isArray(req.messages) &&
+        req.messages.some(
+          (m) => m.role === "user" && (m.content ?? "").trim(),
+        )) ||
+      (typeof req.prompt === "string" && req.prompt.trim());
+    const hasExpected =
+      (typeof resp.text === "string" && resp.text.trim()) ||
+      (Array.isArray(resp.toolCalls) && resp.toolCalls.length > 0);
+    if (!hasUser || !hasExpected) continue;
+    const rec = {
+      format: "eliza_native_v1",
+      schemaVersion: row.schemaVersion ?? 1,
+      boundary: row.boundary,
+      request: slimRequest(req),
+      response: resp,
+      metadata: row.metadata ?? {},
+    };
+    const key = JSON.stringify([
+      req.messages ?? req.prompt,
+      resp.text ?? resp.toolCalls,
+    ]);
+    if (seen.has(key)) continue; // dedup identical gold rows
+    seen.add(key);
+    lines.push(JSON.stringify(rec));
+  }
+  // --sample-rows N + --max-row-chars C : GEPA optimizes a SHARED prompt, so a
+  // size-filtered diverse sample gives the signal without blowing the Cerebras
+  // TPM on 189K-char full-tool-catalog rows. Drop oversized rows first (they
+  // exceed the model context / dominate the token budget), then take an evenly
+  // spaced diverse sample.
+  let kept = lines;
+  if (MAX_ROW_CHARS > 0) kept = kept.filter((l) => l.length <= MAX_ROW_CHARS);
+  if (SAMPLE_ROWS > 0 && kept.length > SAMPLE_ROWS) {
+    const n = kept.length;
+    kept = Array.from(
+      { length: SAMPLE_ROWS },
+      (_, i) => kept[Math.floor((i * n) / SAMPLE_ROWS)],
+    );
+  }
+  writeFileSync(outPath, kept.join("\n") + (kept.length ? "\n" : ""));
+  return { path: outPath, size: kept.length, poolSize: lines.length };
+}
+
+/** The exact `train` command for a task (as an argv array + a printable line). */
+function gepaCommand(task, datasetPath) {
+  const rel = path.relative(REPO_ROOT, datasetPath);
+  const printable =
+    `TRAIN_MODEL_PROVIDER=cerebras CEREBRAS_API_KEY=<key> ELIZA_STATE_DIR=${STATE_DIR} \\\n` +
+    `  bun run --cwd plugins/plugin-training train -- \\\n` +
+    `    --backend native --optimizer gepa --task ${task} \\\n` +
+    `    --dataset ${rel}`;
+  // baseline is inferred from the dataset's system message when --baseline omitted.
+  const argvArr = [
+    "run",
+    "--cwd",
+    "plugins/plugin-training",
+    "train",
+    "--",
+    "--backend",
+    "native",
+    "--optimizer",
+    "gepa",
+    "--task",
+    task,
+    "--dataset",
+    datasetPath,
+  ];
+  return { printable, argvArr };
+}
+
+// ---------------------------------------------------------------------------
+// Execution (gated behind --run)
+// ---------------------------------------------------------------------------
+
+function loadS1ProviderEnv() {
+  if (!existsSync(S1_PROVIDER_PATH)) {
+    throw new Error(
+      `S1 provider env not found at ${S1_PROVIDER_PATH} (needed for the gpt-5.5 re-run). ` +
+        `Provide it or export ELIZA_CHAT_VIA_CLI=codex ELIZA_CLI_CODEX_MODEL=gpt-5.5.`,
+    );
+  }
+  return readJson(S1_PROVIDER_PATH);
+}
+
+function runGepaForTask(task, datasetPath) {
+  const { argvArr } = gepaCommand(task, datasetPath);
+  const artifactCurrent = path.join(
+    STATE_DIR,
+    "optimized-prompts",
+    task === "context_routing" ? "should_respond" : task,
+    "current",
+  );
+  if (!FORCE && existsSync(artifactCurrent)) {
+    console.log(
+      `[gepa] ${task}: artifact exists, skip (use --force). ${artifactCurrent}`,
+    );
+    return { task, skipped: true, artifactCurrent };
+  }
+  if (!process.env.CEREBRAS_API_KEY) {
+    throw new Error(
+      "CEREBRAS_API_KEY is required to run GEPA scoring (the sanctioned fast eval adapter).",
+    );
+  }
+  console.log(
+    `[gepa] ${task}: running GEPA (dataset=${path.basename(datasetPath)})…`,
+  );
+  const res = spawnSync("bun", argvArr, {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: 30 * 60 * 1000,
+    env: {
+      ...process.env,
+      TRAIN_MODEL_PROVIDER: "cerebras",
+      ELIZA_STATE_DIR: STATE_DIR,
+    },
+  });
+  const logDir = path.join(STAGE3_ROOT, "gepa-logs");
+  mkdirSync(logDir, { recursive: true });
+  writeFileSync(path.join(logDir, `${task}.stdout.log`), res.stdout ?? "");
+  writeFileSync(path.join(logDir, `${task}.stderr.log`), res.stderr ?? "");
+  return {
+    task,
+    exitCode: res.status,
+    artifactCurrent,
+    artifactExists: existsSync(artifactCurrent),
+  };
+}
+
+/** Re-run one failing scenario with the GEPA artifact loaded, capture flip. */
+function rerunScenario(scn, providerEnv) {
+  // Real scenario-runner dir from the manifest slug map (the slug is not
+  // reversible by string replace; the old __→/ heuristic produced a bogus path
+  // like "packages_test_scenarios" → ENOENT, which errored every rerun).
+  const dirRel =
+    SLUG_TO_DIR[scn.familyDir] ?? scn.familyDir.replace(/__/g, "/");
+  const outDir = path.join(RERUN_DIR, scn.familyDir, scn.item);
+  mkdirSync(outDir, { recursive: true });
+  const reportPath = path.join(outDir, "report.json");
+  const runDir = path.join(outDir, "run");
+  const nativePath = path.join(outDir, "native.jsonl");
+  const verdictPath = path.join(outDir, "rerun-verdict.json");
+  if (!FORCE && existsSync(verdictPath)) {
+    return { ...readJson(verdictPath), resumed: true };
+  }
+  const args = [
+    "--conditions",
+    "eliza-source",
+    "--tsconfig-override",
+    TSCONFIG,
+    path.join(REPO_ROOT, "packages/scenario-runner/src/cli.ts"),
+    "run",
+    dirRel,
+    "--scenario",
+    scn.item,
+    "--report",
+    reportPath,
+    "--run-dir",
+    runDir,
+    "--export-native",
+    nativePath,
+  ];
+  const res = spawnSync("bun", args, {
+    cwd: REPO_ROOT,
+    encoding: "utf8",
+    timeout: SCENARIO_TIMEOUT_MS,
+    env: {
+      ...process.env,
+      ...providerEnv,
+      ELIZA_STATE_DIR: STATE_DIR, // <-- loads the GEPA artifact at boot
+      ELIZA_SAVE_TRAJECTORIES: "1",
+    },
+  });
+  writeFileSync(path.join(outDir, "stdout.log"), res.stdout ?? "");
+  writeFileSync(path.join(outDir, "stderr.log"), res.stderr ?? "");
+  let status = "error";
+  let judgeScore = null;
+  if (existsSync(reportPath)) {
+    try {
+      const report = readJson(reportPath);
+      const sc =
+        (report.scenarios ?? []).find((x) => x.id === scn.item) ??
+        report.scenarios?.[0];
+      if (sc) {
+        status = sc.status ?? "error";
+        if (typeof sc.judgeScore === "number") judgeScore = sc.judgeScore;
+      }
+    } catch {
+      /* leave error */
+    }
+  }
+  const flipped = status === "passed";
+  const verdict = {
+    item: scn.item,
+    dir: dirRel,
+    cat: scn.cat,
+    exitCode: res.status,
+    status,
+    judgeScore,
+    flipped,
+    nativePath: existsSync(nativePath) ? nativePath : null,
+  };
+  writeFileSync(verdictPath, JSON.stringify(verdict, null, 2));
+
+  // On a flip, copy the now-passing trajectory INTO harvest/scenario/ so Stage 4
+  // picks it up (additive: a distinct <item>__gepa dir; never overwrites the
+  // original failing capture).
+  if (flipped && existsSync(nativePath)) {
+    const harvestDest = path.join(
+      HARVEST_ROOT,
+      "scenario",
+      scn.familyDir,
+      `${scn.item}__gepa`,
+    );
+    mkdirSync(harvestDest, { recursive: true });
+    writeFileSync(
+      path.join(harvestDest, "native.jsonl"),
+      readFileSync(nativePath, "utf8"),
+    );
+    if (existsSync(reportPath)) {
+      writeFileSync(
+        path.join(harvestDest, "report.json"),
+        readFileSync(reportPath, "utf8"),
+      );
+    }
+    writeFileSync(
+      path.join(harvestDest, "verdict.json"),
+      JSON.stringify(
+        {
+          item: scn.item,
+          dir: dirRel,
+          status: "passed",
+          rows: readNativeRows(nativePath).length,
+          judgeScore,
+          source: "stage3-gepa-flip",
+          nativePath: path.join(harvestDest, "native.jsonl"),
+        },
+        null,
+        2,
+      ),
+    );
+    verdict.harvestDest = harvestDest;
+  }
+  return verdict;
+}
+
+// ---------------------------------------------------------------------------
+// Main
+// ---------------------------------------------------------------------------
+
+function main() {
+  mkdirSync(STAGE3_ROOT, { recursive: true });
+  const { perTask, stats } = selectAndGroup();
+
+  // Build datasets + plan for every task that has BOTH failing scenarios AND gold.
+  const plan = [];
+  for (const [task, bucket] of Object.entries(perTask)) {
+    if (ONLY_TASK && task !== ONLY_TASK) continue;
+    const ds = buildTaskDataset(task, bucket.goldRows);
+    const cmd = gepaCommand(task, ds.path);
+    plan.push({
+      task,
+      goldRows: bucket.goldRows.length,
+      datasetSize: ds.size,
+      datasetPath: ds.path,
+      failingScenarios: bucket.failingScenarios.length,
+      dedicatedScorer: TASKS_WITH_DEDICATED_SCORER.has(task),
+      command: cmd.printable,
+      _argv: cmd.argvArr,
+      _failing: bucket.failingScenarios,
+    });
+  }
+  // Sort: tasks with the most failing scenarios first (highest impact).
+  plan.sort((a, b) => b.failingScenarios - a.failingScenarios);
+
+  const summary = {
+    generatedAt: new Date().toISOString(),
+    mode: DRY_RUN ? "dry-run" : RERUN_ONLY ? "rerun-only" : "run",
+    stateDir: STATE_DIR,
+    stats,
+    tasks: plan.map((p) => ({
+      task: p.task,
+      goldRows: p.goldRows,
+      datasetSize: p.datasetSize,
+      failingScenarios: p.failingScenarios,
+      dedicatedScorer: p.dedicatedScorer,
+      datasetPath: path.relative(REPO_ROOT, p.datasetPath),
+    })),
+  };
+
+  // ---- DRY RUN: print the plan + commands, stop. ----
+  if (DRY_RUN) {
+    console.log("=== STAGE 3 GEPA SWEEP — DRY RUN ===\n");
+    console.log("Harvest scan:", JSON.stringify(stats, null, 2), "\n");
+    console.log("Per-task GEPA plan (highest impact first):\n");
+    for (const p of plan) {
+      const gepaable = p.datasetSize > 0 && p.failingScenarios > 0;
+      console.log(
+        `  task=${p.task}  gold=${p.goldRows}→dataset=${p.datasetSize}  ` +
+          `failingScenarios=${p.failingScenarios}  ` +
+          `scorer=${p.dedicatedScorer ? "dedicated" : "token-overlap"}  ` +
+          `${gepaable ? "RUNNABLE" : "SKIP(no dataset or no failures)"}`,
+      );
+      if (gepaable) console.log(`${p.command}\n`);
+    }
+    const summaryPath = path.join(STAGE3_ROOT, "stage3-plan.json");
+    writeFileSync(summaryPath, JSON.stringify(summary, null, 2));
+    console.log(`\nplan → ${path.relative(REPO_ROOT, summaryPath)}`);
+    console.log(
+      "\nNo GEPA executed. To run ONE task end-to-end (proof):\n" +
+        `  CEREBRAS_API_KEY=<key> node scripts/training-harvest/stage3-gepa-sweep.mjs --run --only action_planner --max-rerun 1`,
+    );
+    return;
+  }
+
+  // ---- RUN / RERUN-ONLY ----
+  const providerEnv = loadS1ProviderEnv();
+  const results = [];
+  for (const p of plan) {
+    if (p.datasetSize === 0 || p.failingScenarios === 0) {
+      console.log(
+        `[skip] ${p.task}: dataset=${p.datasetSize} failing=${p.failingScenarios}`,
+      );
+      continue;
+    }
+    let gepaResult = { task: p.task, skipped: RERUN_ONLY };
+    if (!RERUN_ONLY) {
+      gepaResult = runGepaForTask(p.task, p.datasetPath);
+      if (gepaResult.exitCode && gepaResult.exitCode !== 0) {
+        console.log(
+          `[gepa] ${p.task}: exit=${gepaResult.exitCode} — see gepa-logs; skipping rerun.`,
+        );
+        results.push({ task: p.task, gepaResult, reruns: [] });
+        continue;
+      }
+    }
+    // Re-run the failing scenarios for this task with the artifact loaded.
+    const failing = MAX_RERUN ? p._failing.slice(0, MAX_RERUN) : p._failing;
+    const reruns = [];
+    for (const scn of failing) {
+      console.log(`[rerun] ${p.task} :: ${scn.item} (was ${scn.cat})`);
+      const v = rerunScenario(scn, providerEnv);
+      console.log(
+        `[rerun]   → status=${v.status} flipped=${v.flipped} judge=${v.judgeScore ?? "n/a"}`,
+      );
+      reruns.push(v);
+    }
+    const flips = reruns.filter((r) => r.flipped).length;
+    results.push({
+      task: p.task,
+      gepaResult,
+      reruns,
+      flips,
+      rerunCount: reruns.length,
+    });
+    console.log(
+      `[task] ${p.task}: ${flips}/${reruns.length} flipped to passing\n`,
+    );
+  }
+  summary.results = results;
+  const summaryPath = path.join(STAGE3_ROOT, `stage3-run-${Date.now()}.json`);
+  writeFileSync(summaryPath, JSON.stringify(summary, null, 2));
+  console.log(`\nrun summary → ${path.relative(REPO_ROOT, summaryPath)}`);
+}
+
+main();

scripts/vast/README.md

@@ -0,0 +1,138 @@
+# vast.ai certification runner
+
+Automated full-tier certification on rented vast.ai GPUs, plus the local
+fallback (#14548, epic #14541). The product of a run is a **signed
+`certification.json`** — the artifact the develop→main promotion gate
+(`.github/workflows/certification-verify.yml`) verifies. Whether it was
+signed on a vast instance or on a laptop is invisible to the gate by design;
+the Ed25519 signature is what matters (trust model:
+[`.github/certification/README.md`](../../.github/certification/README.md)).
+
+## Files
+
+| File | What |
+| --- | --- |
+| `run-certification.mjs` | The vast driver: search → create → poll → pull → **destroy** (plain node, zero workspace deps) |
+| `run-certification.test.mjs` | Unit tests for the pure pieces (`bun run test:vast` / `node --test scripts/vast/*.test.mjs`) |
+| `local-certify.mjs` | One-command local fallback (same chain, same output) |
+| `../../docker/certification/Dockerfile.gpu` | The prebuilt image the instance boots (models + toolchain baked; onstart is capped at 16 KB) |
+| `../../.github/workflows/certification-vast.yml` | Dispatch/nightly workflow that drives the driver |
+| `../../.github/workflows/certification-image.yml` | Builds + pushes `ghcr.io/elizaos/certification-gpu` |
+
+## Automated run (vast.ai)
+
+Preferred path: dispatch **certification-vast** in the Actions tab (inputs:
+`sha`, `tier`, `max_dph`, `max_attempts`, `timeout_minutes`). Nightly runs
+happen only while the repo variable `ELIZA_VAST_CERT_ENABLED` is `'true'`.
+Required repo secrets: `VAST_API_KEY` (scoped vast key) and
+`ELIZA_CERT_SIGNING_KEY`; optional `ELIZA_CERT_PUSH_CMD` (shell run on the
+instance with `CERT_BUNDLE_DIR`/`CERT_FILE` exported, e.g. an `rclone copy`
+to R2) — without it the signed certification still travels back via the
+instance logs, but the full bundle stays on the instance and dies with it.
+
+From a laptop:
+
+```bash
+export VAST_API_KEY=...            # https://cloud.vast.ai/ → account → API keys
+export ELIZA_CERT_SIGNING_KEY=...  # PEM or base64-wrapped PEM
+node scripts/vast/run-certification.mjs --sha <full-sha> --tier full
+```
+
+Always available with **zero API calls and zero secrets**:
+
+```bash
+node scripts/vast/run-certification.mjs --sha <sha> --dry-run
+```
+
+prints the complete plan: offer query, client-side re-filters, redacted
+create payload, the exact onstart script (with its byte count vs the 16 KB
+vast cap), poll/timeout config, budget worst case, and the exit-code table.
+
+What a real run does, in order:
+
+1. `POST /search/asks/` — verified, rentable, `reliability2 > 0.98`,
+   `inet_down > 500`, `gpu_name` (default `RTX_4090`), `num_gpus 1`, priced
+   `<= --max-dph`, sorted cheapest-first. Results are **re-checked
+   client-side**; offers missing a field are rejected, never assumed fine.
+2. `PUT /asks/{offer}/` — boots `ghcr.io/elizaos/certification-gpu:latest`
+   (override: `--image`). The signing key and push command ride **only** in
+   the create-payload env; the onstart text contains no secret material.
+3. The onstart script clones the repo at exactly `--sha`, `bun run
+   install:light`, starts the baked gpu-vision `llama-server` when present,
+   then `bundle:create --tier <tier>` → `certify:rollup` → `certify:sign`
+   (reviewer kind `agent`), prints the signed certification between log
+   markers, and runs the push command if configured.
+4. Poll `GET /instances/{id}/` every 15 s, pulling log tails: success/failure
+   markers, `exited`, debounced `offline`/`unknown`, a loading-phase cap and
+   a hard `--timeout-minutes` wall clock all terminate the run.
+5. Pull final logs, extract `certification.json` into `--out`
+   (`vast-certification-output/` by default).
+6. **DESTROY the instance in a `finally`** — never stop (stopped vast
+   instances keep billing disk). 3 retries; if all fail the script exits 12
+   and prints an unmissable banner with the instance id and a one-line
+   `curl -X DELETE` to kill it by hand.
+
+## Costs
+
+| Item | Figure |
+| --- | --- |
+| RTX 4090 offer band (verified, reliable) | ~$0.31–0.40/hr; default budget cap `--max-dph 0.60` |
+| Typical full-tier run | 1–2 h ≈ **$0.35–0.80** |
+| Worst case per dispatch | `max_dph × max_attempts × timeout` = 0.60 × 3 × 2 h = **$3.60**, and retries only fire on host-side failures (stuck loading / instance lost), which bill minutes, not hours |
+| Nightly (if enabled) | ≈ $12–25/month |
+
+Every run logs its estimated spend (`elapsed × dph` per attempt, then the
+total) so the workflow log doubles as the cost record.
+
+## Kill paths
+
+Every failure destroys (or at minimum attempts to destroy) the instance
+first, then exits with a distinct code:
+
+| Exit | Meaning | What to do |
+| --- | --- | --- |
+| 2 | usage / missing key env / onstart over 16 KB | fix the invocation |
+| 3 | `VAST_API_KEY` dead or unscoped (HTTP 401/403) | rotate the key; run the local fallback meanwhile |
+| 4 | no eligible offers after client-side re-check | raise `--max-dph`, relax `--gpu-name`, or retry later |
+| 5 | `--max-attempts` exhausted on retryable host failures | vast is having a day; local fallback |
+| 6 | create rejected on every tried offer | as above |
+| 7 | instance stuck in `loading` past `--loading-timeout-minutes` | auto-retries next offer; else local fallback |
+| 8 | hard `--timeout-minutes` wall clock hit | check pulled logs — the chain is too slow or hung; not auto-retried (expensive) |
+| 9 | instance went `offline`/`unknown` (debounced ×3) | auto-retries next offer |
+| 10 | onstart chain failed (marker names the step in pulled logs) | fix the chain — deterministic, so **not** retried on another machine |
+| 11 | run succeeded but no parseable certification in logs | inspect `--out` logs; likely truncated tail — re-dispatch |
+| 12 | **destroy failed after 3 retries — instance still billing** | kill it NOW: `https://cloud.vast.ai/instances/` or `curl -X DELETE -H "Authorization: Bearer $VAST_API_KEY" https://console.vast.ai/api/v0/instances/<id>/` |
+
+The workflow mirrors any failure as a red run whose summary says exactly:
+*local certification required*, with the command below.
+
+## Local fallback (one command)
+
+Same chain, same signed output — on your machine (M-series covers the cpu
+tier; a CUDA box with `llama-server` + `bun run test:gpu-vision` models for
+gpu/full):
+
+```bash
+export ELIZA_CERT_SIGNING_KEY=...   # from the keyholder; never committed
+node scripts/vast/local-certify.mjs --tier full
+```
+
+It runs `bundle:create → certify:rollup → certify:sign` from
+`packages/evidence`, then copies `certification.json` to the repo root ready
+to commit on the promotion branch (plus prints the `evidence/bundle/` copy
+command the gate requires). `--no-sign` stops after rollup so you can
+hand-review `verdicts.json` first — signing refuses to mark mechanically
+non-pass subjects as `pass` either way, so the one-command path cannot
+fabricate green.
+
+## Image
+
+`docker/certification/Dockerfile.gpu` bakes CUDA `llama-server` (pinned to
+llama.cpp `b8525`, the minimum `scripts/gpu-vision` accepts), the
+sha256-pinned OCR/VLM GGUFs via `scripts/gpu-vision/setup.mjs --with-vlm`,
+Node 24, Bun 1.3.14, Playwright Chromium (+ OS deps), tesseract, and ffmpeg.
+Rebuilds publish from `.github/workflows/certification-image.yml`
+(dispatch, or automatically when the Dockerfile / model pins change) to
+`ghcr.io/elizaos/certification-gpu:{latest,sha-<short>}` using the
+workflow's own `GITHUB_TOKEN` — no extra registry secret. Secrets are never
+baked into the image.

scripts/vast/local-certify.mjs

@@ -0,0 +1,178 @@
+#!/usr/bin/env node
+/**
+ * One-command local certification fallback (#14548). Runs the exact chain
+ * the vast.ai onstart runs — packages/evidence bundle:create →
+ * certify:rollup → certify:sign — on this machine, so when vast is down or
+ * the API key is dead a certifier holding ELIZA_CERT_SIGNING_KEY can produce
+ * the same signed certification.json the develop→main gate verifies. Same
+ * commands, same output; the gate cannot tell the difference (by design —
+ * the signature is what matters).
+ *
+ * Signing enforces its own honesty: certify:sign refuses to sign
+ * mechanically non-pass subjects as pass, so "one command" cannot fabricate
+ * a green certification. Use --no-sign to stop after rollup, hand-review
+ * verdicts.json (waivers require notes), then run certify:sign yourself —
+ * that is the diligent-reviewer path the trust model expects.
+ */
+
+import { spawnSync } from "node:child_process";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const REPO_ROOT = path.resolve(
+  path.dirname(fileURLToPath(import.meta.url)),
+  "../..",
+);
+const SIGNING_KEY_ENV_VAR = "ELIZA_CERT_SIGNING_KEY";
+const TIERS = ["cpu", "gpu", "full"];
+
+const USAGE = `Usage: node scripts/vast/local-certify.mjs [--tier cpu|gpu|full] [options]
+
+Options:
+  --tier <cpu|gpu|full>   Certification tier (default: full)
+  --reviewer-id <id>      Reviewer identity in the signed cert (default: $USER@host)
+  --no-sign               Stop after rollup for hand review of verdicts.json
+  --help                  This text
+
+Requires env ${SIGNING_KEY_ENV_VAR} (PEM or base64-wrapped PEM) unless --no-sign.
+Output: the bundle dir under evidence/runs/ with certification.json inside,
+plus a copy of certification.json at the repo root ready to commit on the
+promotion branch (bundle → evidence/bundle/ per .github/certification/README.md).`;
+
+function parseArgs(argv) {
+  const opts = {
+    tier: "full",
+    reviewerId: `${os.userInfo().username}@${os.hostname()}`,
+    sign: true,
+  };
+  for (let index = 0; index < argv.length; index += 1) {
+    const arg = argv[index];
+    if (arg === "--tier") {
+      opts.tier = argv[++index];
+      if (!TIERS.includes(opts.tier)) {
+        throw new Error(
+          `--tier must be one of ${TIERS.join("|")}, got: ${opts.tier}`,
+        );
+      }
+    } else if (arg === "--reviewer-id") {
+      const value = argv[++index];
+      if (!value) throw new Error("--reviewer-id requires a value");
+      opts.reviewerId = value;
+    } else if (arg === "--no-sign") {
+      opts.sign = false;
+    } else if (arg === "--help") {
+      opts.help = true;
+    } else {
+      throw new Error(`unknown argument: ${arg}\n\n${USAGE}`);
+    }
+  }
+  return opts;
+}
+
+function run(label, command, args) {
+  console.log(`\n[local-certify] ${label}: ${command} ${args.join(" ")}`);
+  const result = spawnSync(command, args, { cwd: REPO_ROOT, stdio: "inherit" });
+  if (result.status !== 0) {
+    throw new Error(`${label} failed (exit ${result.status ?? "signal"})`);
+  }
+}
+
+function newestBundleDir() {
+  const runsDir = path.join(REPO_ROOT, "evidence", "runs");
+  const entries = fs
+    .readdirSync(runsDir, { withFileTypes: true })
+    .filter((entry) => entry.isDirectory())
+    .map((entry) => {
+      const dir = path.join(runsDir, entry.name);
+      return { dir, mtimeMs: fs.statSync(dir).mtimeMs };
+    })
+    .sort((a, b) => b.mtimeMs - a.mtimeMs);
+  if (entries.length === 0) {
+    throw new Error(`no bundle directory appeared under ${runsDir}`);
+  }
+  return entries[0].dir;
+}
+
+function main(argv, env) {
+  const opts = parseArgs(argv);
+  if (opts.help) {
+    console.log(USAGE);
+    return 0;
+  }
+  if (opts.sign && !env[SIGNING_KEY_ENV_VAR]) {
+    console.error(
+      `[local-certify] no signing key: export ${SIGNING_KEY_ENV_VAR} (or use --no-sign to produce reviewable verdicts without signing)`,
+    );
+    return 2;
+  }
+
+  run("bundle:create", "bun", [
+    "run",
+    "--cwd",
+    "packages/evidence",
+    "bundle:create",
+    "--",
+    "--tier",
+    opts.tier,
+  ]);
+  const bundleDir = newestBundleDir();
+  const verdictsPath = path.join(bundleDir, "verdicts.json");
+  run("certify:rollup", "bun", [
+    "run",
+    "--cwd",
+    "packages/evidence",
+    "certify:rollup",
+    "--",
+    "--bundle",
+    bundleDir,
+    "--out",
+    verdictsPath,
+  ]);
+
+  if (!opts.sign) {
+    console.log(`\n[local-certify] stopped before signing (--no-sign).`);
+    console.log(`[local-certify] review ${verdictsPath}, then:`);
+    console.log(
+      `  bun run --cwd packages/evidence certify:sign -- --bundle ${bundleDir} --verdicts ${verdictsPath} --reviewer-id <you> --reviewer-kind human`,
+    );
+    return 0;
+  }
+
+  run("certify:sign", "bun", [
+    "run",
+    "--cwd",
+    "packages/evidence",
+    "certify:sign",
+    "--",
+    "--bundle",
+    bundleDir,
+    "--verdicts",
+    verdictsPath,
+    "--reviewer-id",
+    opts.reviewerId,
+    "--reviewer-kind",
+    "human",
+  ]);
+
+  const certSource = path.join(bundleDir, "certification.json");
+  const certTarget = path.join(REPO_ROOT, "certification.json");
+  fs.copyFileSync(certSource, certTarget);
+  console.log(`\n[local-certify] done.`);
+  console.log(`[local-certify] bundle:        ${bundleDir}`);
+  console.log(
+    `[local-certify] certification: ${certTarget} (copied from the bundle)`,
+  );
+  console.log(
+    `[local-certify] for a promotion PR also commit the bundle: rm -rf evidence/bundle && mkdir -p evidence && cp -R '${bundleDir}' evidence/bundle`,
+  );
+  return 0;
+}
+
+try {
+  process.exitCode = main(process.argv.slice(2), process.env);
+} catch (error) {
+  console.error(`[local-certify] ${error.message}`);
+  process.exitCode = 1;
+}

scripts/vast/run-certification.test.mjs

@@ -0,0 +1,536 @@
+/**
+ * Unit tests for the pure pieces of the vast.ai certification runner: offer
+ * filtering/sorting, onstart assembly + the 16 KB vast cap, the poll state
+ * machine's kill paths, budget guards, CLI parsing, log-marker/certification
+ * extraction, and dry-run secret redaction. No network, no vast account —
+ * everything here is a pure function imported from run-certification.mjs
+ * (the real-instance acceptance run is owner-gated on VAST_API_KEY).
+ */
+
+import assert from "node:assert/strict";
+import { describe, it } from "node:test";
+import {
+  API_KEY_ENV_VAR,
+  assertOnstartSize,
+  buildCreatePayload,
+  buildDryRunPlan,
+  buildOfferQuery,
+  buildOnstart,
+  CERT_BEGIN_MARKER,
+  CERT_END_MARKER,
+  createPollState,
+  detectMarker,
+  EXIT,
+  extractCertification,
+  FAILURE_MARKER,
+  filterAndSortOffers,
+  isRetryableOutcome,
+  ONSTART_MAX_BYTES,
+  PUSH_CMD_ENV_VAR,
+  parseCliArgs,
+  redactCreatePayload,
+  reducePoll,
+  SIGNING_KEY_ENV_VAR,
+  SUCCESS_MARKER,
+  selectAttemptOffers,
+  UsageError,
+} from "./run-certification.mjs";
+
+const SHA = "a".repeat(40);
+
+function baseOpts(overrides = {}) {
+  return parseCliArgs(["--sha", SHA, "--dry-run"], {
+    [SIGNING_KEY_ENV_VAR]: undefined,
+    ...overrides.env,
+  });
+}
+
+function offer(overrides = {}) {
+  return {
+    id: 1,
+    dph_total: 0.35,
+    reliability2: 0.995,
+    inet_down: 900,
+    num_gpus: 1,
+    gpu_name: "RTX 4090",
+    ...overrides,
+  };
+}
+
+describe("buildOfferQuery", () => {
+  it("translates underscore GPU names and pins the acceptance filters", () => {
+    const query = buildOfferQuery(baseOpts());
+    assert.equal(query.gpu_name.eq, "RTX 4090");
+    assert.equal(query.num_gpus.eq, 1);
+    assert.equal(query.verified.eq, true);
+    assert.equal(query.rentable.eq, true);
+    assert.equal(query.reliability2.gt, 0.98);
+    assert.equal(query.inet_down.gt, 500);
+    assert.deepEqual(query.order, [["dph_total", "asc"]]);
+    assert.equal(query.type, "ask");
+  });
+});
+
+describe("filterAndSortOffers", () => {
+  it("sorts eligible offers cheapest-first with reliability tie-break", () => {
+    const { eligible } = filterAndSortOffers(
+      [
+        offer({ id: 1, dph_total: 0.4 }),
+        offer({ id: 2, dph_total: 0.3, reliability2: 0.99 }),
+        offer({ id: 3, dph_total: 0.3, reliability2: 0.999 }),
+      ],
+      baseOpts(),
+    );
+    assert.deepEqual(
+      eligible.map((entry) => entry.id),
+      [3, 2, 1],
+    );
+  });
+
+  it("rejects offers over --max-dph with a reason (budget guard)", () => {
+    const { eligible, rejected } = filterAndSortOffers(
+      [offer({ id: 7, dph_total: 0.99 })],
+      baseOpts(),
+    );
+    assert.equal(eligible.length, 0);
+    assert.equal(rejected[0].id, 7);
+    assert.match(rejected[0].reason, /--max-dph/);
+  });
+
+  it("re-checks reliability, inet_down and num_gpus client-side", () => {
+    const { eligible, rejected } = filterAndSortOffers(
+      [
+        offer({ id: 1, reliability2: 0.9 }),
+        offer({ id: 2, inet_down: 100 }),
+        offer({ id: 3, num_gpus: 4 }),
+        offer({ id: 4 }),
+      ],
+      baseOpts(),
+    );
+    assert.deepEqual(
+      eligible.map((entry) => entry.id),
+      [4],
+    );
+    assert.equal(rejected.length, 3);
+  });
+
+  it("falls back to legacy `reliability` and rejects offers missing both", () => {
+    const { eligible, rejected } = filterAndSortOffers(
+      [
+        offer({ id: 1, reliability2: undefined, reliability: 0.999 }),
+        offer({ id: 2, reliability2: undefined, reliability: undefined }),
+      ],
+      baseOpts(),
+    );
+    assert.deepEqual(
+      eligible.map((entry) => entry.id),
+      [1],
+    );
+    assert.match(rejected[0].reason, /missing reliability/);
+  });
+
+  it("never assumes compliance for offers missing dph_total or id", () => {
+    const { eligible } = filterAndSortOffers(
+      [offer({ id: undefined }), offer({ dph_total: undefined })],
+      baseOpts(),
+    );
+    assert.equal(eligible.length, 0);
+  });
+});
+
+describe("selectAttemptOffers", () => {
+  it("caps attempts at --max-attempts, cheapest-first", () => {
+    const { eligible } = filterAndSortOffers(
+      [1, 2, 3, 4, 5].map((id) => offer({ id, dph_total: id / 10 })),
+      baseOpts(),
+    );
+    const attempts = selectAttemptOffers(eligible, { maxAttempts: 2 });
+    assert.deepEqual(
+      attempts.map((entry) => entry.id),
+      [1, 2],
+    );
+  });
+});
+
+describe("buildOnstart", () => {
+  it("clones the exact sha and runs the certification chain in order", () => {
+    const onstart = buildOnstart(baseOpts());
+    const cloneIndex = onstart.indexOf(`git fetch --depth 1 origin ${SHA}`);
+    const installIndex = onstart.indexOf("bun run install:light");
+    const bundleIndex = onstart.indexOf("bundle:create -- --tier full");
+    const rollupIndex = onstart.indexOf("certify:rollup");
+    const signIndex = onstart.indexOf("certify:sign");
+    assert.ok(
+      cloneIndex > -1 && installIndex > cloneIndex,
+      "clone then install",
+    );
+    assert.ok(bundleIndex > installIndex, "bundle after install");
+    assert.ok(rollupIndex > bundleIndex, "rollup after bundle");
+    assert.ok(signIndex > rollupIndex, "sign after rollup");
+    assert.ok(onstart.includes(SUCCESS_MARKER));
+    assert.ok(onstart.includes(CERT_BEGIN_MARKER));
+    assert.ok(onstart.includes("ELIZA_EVIDENCE_RUNNER=vast"));
+  });
+
+  it("never embeds secret material in the onstart text", () => {
+    const opts = {
+      ...baseOpts(),
+      signingKey: "-----BEGIN PRIVATE KEY-----SECRETSECRET",
+      pushCmd: "aws s3 cp --secret-token hunter2",
+      apiKey: "vast-key-hunter3",
+    };
+    const onstart = buildOnstart(opts);
+    assert.ok(!onstart.includes("SECRETSECRET"));
+    assert.ok(!onstart.includes("hunter2"));
+    assert.ok(!onstart.includes("hunter3"));
+    // The push command is referenced by env var NAME only.
+    assert.ok(onstart.includes(`\${${PUSH_CMD_ENV_VAR}:-}`));
+  });
+
+  it("stays comfortably under the 16 KB vast cap with defaults", () => {
+    const bytes = assertOnstartSize(buildOnstart(baseOpts()));
+    assert.ok(
+      bytes < ONSTART_MAX_BYTES / 4,
+      `expected small onstart, got ${bytes} bytes`,
+    );
+  });
+});
+
+describe("assertOnstartSize", () => {
+  it("throws UsageError past 16 KB (multi-byte aware)", () => {
+    assert.throws(
+      () => assertOnstartSize("é".repeat(ONSTART_MAX_BYTES / 2 + 1)),
+      UsageError,
+    );
+    assert.throws(
+      () =>
+        assertOnstartSize(
+          buildOnstart({ ...baseOpts(), reviewerId: "x".repeat(17000) }),
+        ),
+      /16384/,
+    );
+  });
+
+  it("accepts exactly 16 KB", () => {
+    assert.equal(
+      assertOnstartSize("x".repeat(ONSTART_MAX_BYTES)),
+      ONSTART_MAX_BYTES,
+    );
+  });
+});
+
+describe("buildCreatePayload / redactCreatePayload", () => {
+  it("injects signing key and push cmd via env only, destroy-safe defaults", () => {
+    const opts = { ...baseOpts(), signingKey: "PEMPEM", pushCmd: "push it" };
+    const payload = buildCreatePayload(opts, "onstart");
+    assert.equal(payload.env[SIGNING_KEY_ENV_VAR], "PEMPEM");
+    assert.equal(payload.env[PUSH_CMD_ENV_VAR], "push it");
+    assert.equal(payload.env.ELIZA_EVIDENCE_RUNNER, "vast");
+    assert.equal(payload.client_id, "me");
+    assert.equal(payload.image, "ghcr.io/elizaos/certification-gpu:latest");
+    assert.equal(payload.label, `eliza-certification-${SHA.slice(0, 12)}`);
+  });
+
+  it("redacts every env value, keeps the key names visible", () => {
+    const payload = buildCreatePayload(
+      { ...baseOpts(), signingKey: "PEMPEM", pushCmd: "s3 cp secret" },
+      "onstart",
+    );
+    const redacted = redactCreatePayload(payload);
+    assert.deepEqual(
+      Object.values(redacted.env),
+      Object.keys(payload.env).map(() => "<redacted>"),
+    );
+    assert.ok(JSON.stringify(redacted).includes(SIGNING_KEY_ENV_VAR));
+    assert.ok(!JSON.stringify(redacted).includes("PEMPEM"));
+  });
+});
+
+describe("poll state machine", () => {
+  const config = { timeoutMs: 60 * 60_000, loadingTimeoutMs: 10 * 60_000 };
+
+  function run(snapshots, stateConfig = config) {
+    let state = createPollState(stateConfig);
+    for (const snapshot of snapshots) {
+      state = reducePoll(state, { marker: null, ...snapshot });
+      if (state.outcome) return state;
+    }
+    return state;
+  }
+
+  it("happy path: loading → running → success marker", () => {
+    const state = run([
+      { actualStatus: "loading", elapsedMs: 60_000 },
+      { actualStatus: "running", elapsedMs: 300_000 },
+      { actualStatus: "running", elapsedMs: 900_000, marker: "success" },
+    ]);
+    assert.deepEqual(state.outcome, { ok: true });
+  });
+
+  it("failure marker → ONSTART_FAILED even while still running", () => {
+    const state = run([
+      { actualStatus: "running", elapsedMs: 300_000, marker: "failure" },
+    ]);
+    assert.equal(state.outcome.ok, false);
+    assert.equal(state.outcome.code, EXIT.ONSTART_FAILED);
+  });
+
+  it("exited without success marker → ONSTART_FAILED", () => {
+    const state = run([
+      { actualStatus: "running", elapsedMs: 300_000 },
+      { actualStatus: "exited", elapsedMs: 600_000 },
+    ]);
+    assert.equal(state.outcome.code, EXIT.ONSTART_FAILED);
+    assert.match(state.outcome.reason, /without the success marker/);
+  });
+
+  it("exited WITH success marker in the same tick is a success", () => {
+    const state = run([
+      { actualStatus: "exited", elapsedMs: 600_000, marker: "success" },
+    ]);
+    assert.deepEqual(state.outcome, { ok: true });
+  });
+
+  it("stuck in loading past the loading timeout → STUCK_LOADING", () => {
+    const state = run([
+      { actualStatus: "loading", elapsedMs: 60_000 },
+      { actualStatus: "loading", elapsedMs: 11 * 60_000 },
+    ]);
+    assert.equal(state.outcome.code, EXIT.STUCK_LOADING);
+  });
+
+  it("blank/provisioning status counts as loading for the stuck check", () => {
+    const state = run([{ actualStatus: "", elapsedMs: 11 * 60_000 }]);
+    assert.equal(state.outcome.code, EXIT.STUCK_LOADING);
+  });
+
+  it("loading timeout does NOT fire once the instance has been running", () => {
+    const state = run([
+      { actualStatus: "running", elapsedMs: 60_000 },
+      { actualStatus: "loading", elapsedMs: 11 * 60_000 },
+    ]);
+    assert.equal(state.outcome, null);
+  });
+
+  it("offline/unknown are debounced, then INSTANCE_LOST", () => {
+    const state = run([
+      { actualStatus: "running", elapsedMs: 60_000 },
+      { actualStatus: "offline", elapsedMs: 120_000 },
+      { actualStatus: "unknown", elapsedMs: 180_000 },
+      { actualStatus: "offline", elapsedMs: 240_000 },
+    ]);
+    assert.equal(state.outcome.code, EXIT.INSTANCE_LOST);
+  });
+
+  it("a healthy poll resets the offline debounce", () => {
+    const state = run([
+      { actualStatus: "offline", elapsedMs: 60_000 },
+      { actualStatus: "offline", elapsedMs: 120_000 },
+      { actualStatus: "running", elapsedMs: 180_000 },
+      { actualStatus: "offline", elapsedMs: 240_000 },
+      { actualStatus: "offline", elapsedMs: 300_000 },
+    ]);
+    assert.equal(state.outcome, null);
+    assert.equal(state.consecutiveLost, 2);
+  });
+
+  it("hard wall-clock timeout → RUN_TIMEOUT", () => {
+    const state = run([{ actualStatus: "running", elapsedMs: 61 * 60_000 }]);
+    assert.equal(state.outcome.code, EXIT.RUN_TIMEOUT);
+  });
+
+  it("terminal state is sticky", () => {
+    let state = run([{ actualStatus: "running", elapsedMs: 61 * 60_000 }]);
+    state = reducePoll(state, {
+      actualStatus: "running",
+      elapsedMs: 62 * 60_000,
+      marker: "success",
+    });
+    assert.equal(state.outcome.code, EXIT.RUN_TIMEOUT);
+  });
+});
+
+describe("isRetryableOutcome", () => {
+  it("retries only host-side failures, never chain failures or timeouts", () => {
+    assert.ok(isRetryableOutcome(EXIT.STUCK_LOADING));
+    assert.ok(isRetryableOutcome(EXIT.INSTANCE_LOST));
+    assert.ok(isRetryableOutcome(EXIT.CREATE_FAILED));
+    assert.ok(!isRetryableOutcome(EXIT.ONSTART_FAILED));
+    assert.ok(!isRetryableOutcome(EXIT.RUN_TIMEOUT));
+    assert.ok(!isRetryableOutcome(EXIT.DEAD_API_KEY));
+  });
+});
+
+describe("VastApiError auth classification", () => {
+  it("maps vast's 404 + auth_error body to a dead key (live-API behavior)", async () => {
+    const { parseVastErrorCode, VastApiError } = await import(
+      "./run-certification.mjs"
+    );
+    // Verified against the live API: a dead key answers HTTP 404 with
+    // {"success":false,"error":"auth_error","msg":"Invalid user key"}.
+    const body =
+      '{"success":false,"error":"auth_error","msg":"Invalid user key"}';
+    assert.equal(parseVastErrorCode(body), "auth_error");
+    const authByCode = new VastApiError("x", {
+      status: 404,
+      errorCode: "auth_error",
+    });
+    assert.ok(authByCode.isAuthFailure);
+    const authByStatus = new VastApiError("x", { status: 401 });
+    assert.ok(authByStatus.isAuthFailure);
+    const plain404 = new VastApiError("x", { status: 404 });
+    assert.ok(!plain404.isAuthFailure);
+  });
+
+  it("returns undefined for HTML error pages, never throws", async () => {
+    const { parseVastErrorCode } = await import("./run-certification.mjs");
+    assert.equal(parseVastErrorCode("<html>404</html>"), undefined);
+    assert.equal(parseVastErrorCode('{"msg":"no error field"}'), undefined);
+    assert.equal(parseVastErrorCode(undefined), undefined);
+  });
+});
+
+describe("detectMarker / extractCertification", () => {
+  it("failure marker wins over a later success marker", () => {
+    assert.equal(
+      detectMarker(`${FAILURE_MARKER}: rollup\n${SUCCESS_MARKER}\n`),
+      "failure",
+    );
+    assert.equal(detectMarker(`all good\n${SUCCESS_MARKER}\n`), "success");
+    assert.equal(detectMarker("still going"), null);
+    assert.equal(detectMarker(""), null);
+    assert.equal(detectMarker(undefined), null);
+  });
+
+  it("extracts the LAST complete certification block and validates JSON", () => {
+    const first = JSON.stringify({ commit: "old" });
+    const second = JSON.stringify({ commit: SHA, verdicts: [] });
+    const logs = [
+      CERT_BEGIN_MARKER,
+      first,
+      CERT_END_MARKER,
+      "retry noise",
+      CERT_BEGIN_MARKER,
+      second,
+      CERT_END_MARKER,
+      SUCCESS_MARKER,
+    ].join("\n");
+    assert.equal(extractCertification(logs), second);
+  });
+
+  it("returns null for truncated or unparseable blocks (never corrupt output)", () => {
+    assert.equal(extractCertification(`${CERT_BEGIN_MARKER}\n{"a": 1`), null);
+    assert.equal(
+      extractCertification(
+        `${CERT_BEGIN_MARKER}\n{"a": tru\n${CERT_END_MARKER}`,
+      ),
+      null,
+    );
+    assert.equal(extractCertification("no markers"), null);
+  });
+});
+
+describe("parseCliArgs", () => {
+  it("requires --sha as hex and validates --tier", () => {
+    assert.throws(() => parseCliArgs(["--dry-run"], {}), /--sha is required/);
+    assert.throws(
+      () => parseCliArgs(["--sha", "not-hex!", "--dry-run"], {}),
+      /--sha/,
+    );
+    assert.throws(
+      () => parseCliArgs(["--sha", SHA, "--tier", "mega", "--dry-run"], {}),
+      /--tier must be one of/,
+    );
+  });
+
+  it("requires api + signing keys for real runs, but not for --dry-run", () => {
+    assert.throws(
+      () => parseCliArgs(["--sha", SHA], {}),
+      new RegExp(API_KEY_ENV_VAR),
+    );
+    assert.throws(
+      () => parseCliArgs(["--sha", SHA], { [API_KEY_ENV_VAR]: "k" }),
+      new RegExp(SIGNING_KEY_ENV_VAR),
+    );
+    const opts = parseCliArgs(["--sha", SHA, "--dry-run"], {});
+    assert.equal(opts.dryRun, true);
+  });
+
+  it("parses budget guards and rejects non-positive values", () => {
+    const opts = parseCliArgs(
+      [
+        "--sha",
+        SHA,
+        "--dry-run",
+        "--max-dph",
+        "0.45",
+        "--max-attempts",
+        "2",
+        "--timeout-minutes",
+        "90",
+      ],
+      {},
+    );
+    assert.equal(opts.maxDph, 0.45);
+    assert.equal(opts.maxAttempts, 2);
+    assert.equal(opts.timeoutMinutes, 90);
+    assert.throws(
+      () =>
+        parseCliArgs(["--sha", SHA, "--dry-run", "--max-attempts", "0"], {}),
+      /--max-attempts/,
+    );
+    assert.throws(
+      () =>
+        parseCliArgs(["--sha", SHA, "--dry-run", "--max-dph", "banana"], {}),
+      /--max-dph/,
+    );
+    assert.throws(
+      () => parseCliArgs(["--sha", SHA, "--dry-run", "--unknown"], {}),
+      /unknown argument/,
+    );
+  });
+
+  it("reads keys and push cmd from the environment", () => {
+    const opts = parseCliArgs(["--sha", SHA], {
+      [API_KEY_ENV_VAR]: "api-key",
+      [SIGNING_KEY_ENV_VAR]: "pem",
+      [PUSH_CMD_ENV_VAR]: "rclone copy",
+    });
+    assert.equal(opts.apiKey, "api-key");
+    assert.equal(opts.signingKey, "pem");
+    assert.equal(opts.pushCmd, "rclone copy");
+  });
+});
+
+describe("buildDryRunPlan", () => {
+  it("summarizes budget worst-case and never leaks secret values", () => {
+    const opts = parseCliArgs(
+      [
+        "--sha",
+        SHA,
+        "--dry-run",
+        "--max-dph",
+        "0.5",
+        "--max-attempts",
+        "2",
+        "--timeout-minutes",
+        "60",
+      ],
+      {
+        [SIGNING_KEY_ENV_VAR]: "-----BEGIN PRIVATE KEY-----LEAKME",
+        [API_KEY_ENV_VAR]: "vast-LEAKME",
+        [PUSH_CMD_ENV_VAR]: "s3 cp --token LEAKME",
+      },
+    );
+    const plan = buildDryRunPlan(opts);
+    assert.equal(plan.budget.worstCaseUsd, 1);
+    const serialized = JSON.stringify(plan);
+    assert.ok(!serialized.includes("LEAKME"));
+    assert.equal(
+      plan.secrets[SIGNING_KEY_ENV_VAR],
+      "present (create-env only)",
+    );
+    assert.ok(plan.onstart.includes(SHA));
+    assert.equal(plan.cleanup.retries, 3);
+  });
+});