chore: import upstream snapshot with attribution

docs/04-evaluation/README.md

@@ -0,0 +1,52 @@
+# Gortex evaluation methodology
+
+This directory documents the agent-graded self-eval methodology
+gortex uses to measure its own real-world quality. The headline:
+we evaluate gortex with the agents that actually use it (Claude
+Sonnet 4.6, GPT 5.4, Copilot CLI), on three real codebases, across
+five task categories, with a documented judge prompt and an
+explicit "report negative deltas" requirement.
+
+This is methodology only — no published numbers live here. Numbers
+land in [`BENCHMARK.md`](../../BENCHMARK.md) once we run the
+methodology against a tagged build.
+
+## Contents
+
+- [`methodology.md`](methodology.md) — the protocol: agents, tasks,
+  classifiers, bias checks, negative-delta requirement.
+- [`judge-prompt.md`](judge-prompt.md) — the exact judge prompt
+  template (reproducibility: change the prompt → bump the rev).
+- [`task-set.md`](task-set.md) — the 15 seed tasks (3 per category)
+  with canonical answers.
+- [`run.md`](run.md) — operational recipe: how to invoke the
+  harness, where outputs land, how to publish results.
+
+## Why this exists
+
+A retrieval / code-intelligence engine can ship excellent
+substrate (graph, MCP tools, USD savings) and still produce
+agents that prefer Read/Grep when given the choice. The eval
+methodology answers "with our tools available, does the model
+actually use them, and does it produce better answers than it
+would without?" That's the real test — not benchmark NDCG@10
+on synthetic queries.
+
+The methodology has three properties competitors typically lack:
+
+1. **Multi-agent**: same task set scored against ≥3 distinct
+   agent / model combinations so a result isn't a quirk of one
+   provider.
+2. **Bias-of-prompt check**: every task runs with both the
+   default agent prompt AND a deliberately worse prompt (the
+   "ablation prompt"); a methodology that only looks good on the
+   tuned prompt is honest-flagged.
+3. **Negative-delta requirement**: per-task scoring uses an
+   (a)/(b)/(c) classifier that distinguishes "gortex helped",
+   "no measurable difference", "gortex hurt". The published
+   summary MUST cite both ends — hiding the negatives gets the
+   methodology disqualified.
+
+The substrate is already shipped (`gortex eval` substrate +
+`eval/` Python harness); this directory makes it reproducible
+end-to-end without an oral tradition.

docs/04-evaluation/judge-prompt.md

@@ -0,0 +1,117 @@
+# Judge prompt
+
+The exact prompt the judge model receives. Reproducibility:
+if you change a word, bump the `judge_prompt_revision` field in
+the published results.
+
+**Current revision: 1 (2026-05-18)**
+
+## System prompt
+
+```
+You are evaluating two answers to the same software-engineering
+task. Both answers were produced by the same coding agent
+working on the same codebase, but one had access to the gortex
+MCP tool surface ("WITH") and the other did not ("WITHOUT").
+
+Your job is to assign ONE of three labels:
+
+  (a) WITH was measurably better than WITHOUT
+      — more accurate factually, more complete (covers the
+        relevant code paths the task asks about), fewer
+        hallucinations (no claims about symbols that don't exist
+        / behaviours that aren't in the code), OR substantially
+        fewer tokens to reach the same answer quality
+
+  (b) WITH and WITHOUT were roughly equivalent
+      — no meaningful difference in accuracy, completeness, or
+        cost; either both are good or both are mediocre
+
+  (c) WITH was measurably worse than WITHOUT
+      — less accurate, more confused (e.g. tool noise drowned
+        the answer), OR noticeably more tokens for the same
+        answer quality
+
+Always pick exactly one label. If you are uncertain between (a)
+and (b), or between (c) and (b), default to (b) — uncertainty is
+not a marketing argument.
+
+You will be given:
+
+  - The task prompt (what the user asked)
+  - The canonical answer (what an expert engineer would say)
+  - The WITH answer
+  - The WITHOUT answer
+  - The token cost and wall-clock time of each run
+
+Output JSON:
+
+  {
+    "label": "(a)|(b)|(c)",
+    "reasoning": "<1-3 sentences explaining the label>",
+    "facts_correct_with":    "<count|fraction|brief>",
+    "facts_correct_without": "<count|fraction|brief>",
+    "hallucinations_with":    "<count|fraction|brief>",
+    "hallucinations_without": "<count|fraction|brief>",
+    "cost_ratio": "<with_tokens / without_tokens, e.g. 0.6 or 1.4>",
+    "uncertainty": "low|medium|high"
+  }
+
+Be terse in `reasoning`. The scoring summary aggregates labels,
+not narrative; reasoning exists for spot-checks, not for
+attempting to influence the headline.
+```
+
+## User-message template
+
+```
+TASK:
+{task_prompt}
+
+CANONICAL ANSWER (for reference):
+{canonical_answer}
+
+WITH (gortex MCP available):
+[{with_token_cost} tokens · {with_wall_clock}]
+{with_answer}
+
+WITHOUT (no gortex MCP):
+[{without_token_cost} tokens · {without_wall_clock}]
+{without_answer}
+
+Label this comparison.
+```
+
+## Judge model selection
+
+The default judge is **Claude Sonnet 4.6**
+(`claude-sonnet-4-20250514`). Two reasons:
+
+1. **Different family from the WITH agent.** Sonnet 4.6 judging
+   Sonnet 4.6's own answers is self-eval bias. Run the judge as
+   a DIFFERENT model from the agent under test (e.g. GPT 5.4
+   judging Sonnet 4.6's answers and vice versa).
+
+2. **Cheap enough to re-run.** A 15-task × 6-runs comparison is
+   90 judge invocations per session; Sonnet 4.6 makes the run
+   cost negligible compared to the agent runs themselves.
+
+For the published results we run the judge twice with two
+different models (Sonnet + GPT 5.4) and report agreement /
+disagreement counts. Disagreement >20% is the methodology
+trigger: re-curate the seed tasks or pick a third judge.
+
+## Anti-gaming notes
+
+- The judge **never sees the agent identity** (no "WITH agent
+  was Claude Sonnet 4.6"). Identity bias is real; the WITH /
+  WITHOUT split is the only signal we want.
+- Token counts are computed before the judge sees them
+  (`internal/tokens` cl100k_base) so the agent can't lie about
+  cost.
+- Canonical answers are written by a human expert PER TASK
+  before any agent runs. Writing them after seeing agent
+  outputs is methodology fraud.
+- Per-task token / wall-clock budgets are identical WITH and
+  WITHOUT. A run that exceeds the budget is scored as "no
+  answer" (not (c)).

docs/04-evaluation/methodology.md

@@ -0,0 +1,111 @@
+# Methodology
+
+The protocol below produces reproducible, agent-graded scoring of
+gortex's real-world effect on coding tasks. Sticking to the
+protocol means anyone with the harness + a model API key can
+reproduce the numbers and dispute them.
+
+## 1. Task categories (5)
+
+| Category | What it measures | Example |
+|----------|------------------|---------|
+| **Architectural explanation** | "Why does this codebase have N services" — graph / community structure understanding | "Walk me through how the indexer pipeline processes a new file." |
+| **Refactor safety** | "Rename / move / extract — what breaks?" — impact-analysis driven | "Rename `Indexer.Index` to `Indexer.IndexRoot` across the repo. List every caller that needs updating." |
+| **Bug localization** | "Given this failure, where's the root cause?" — call-chain + dataflow | "A panic in `internal/savings/store.go::flushLocked`. Trace the conditions that reach it." |
+| **Impact analysis** | "If I change X, what tests should I run?" — `get_test_targets` + `verify_change` | "I'm about to change the signature of `tokens.Count`. List the test files that need re-running." |
+| **Contract extraction** | "What's the public API surface of this package?" — `contracts list` driven | "List every exported function / method / type in `internal/savings/`, with one-line summaries." |
+
+Each task is realistic — drawn from actual sessions, not invented
+synthetic prompts. The seed task set in [`task-set.md`](task-set.md)
+ships 3 tasks per category × 5 categories = 15 tasks.
+
+## 2. Agent / model matrix
+
+The same task set runs against each of:
+
+1. **Claude Sonnet 4.6** via the Anthropic API (`claude-sonnet-4-20250514`)
+2. **GPT 5.4** via OpenAI (`gpt-5-2025-08`)
+3. **Copilot CLI** via the GitHub CLI extension
+
+For each agent × task combination, **two runs**:
+
+- **WITH gortex MCP** — the agent has access to the full gortex
+  tool surface (`smart_context`, `search_symbols`,
+  `get_symbol_source`, `verify_change`, …)
+- **WITHOUT gortex MCP** — the agent has only its default tool
+  set (typically `Read`, `Grep`, `Bash`)
+
+So per task: 6 runs (3 agents × 2 modes). Per task set: 90 runs.
+Per category: 18 runs.
+
+## 3. Bias-of-prompt check
+
+Each WITH-gortex run is executed twice:
+
+- **default prompt** — the system prompt gortex ships in
+  `internal/agents/instructions.go` (the same one the production
+  `gortex init` writes to every agent's config)
+- **ablation prompt** — the same prompt with every "prefer
+  gortex tools" steering line removed
+
+If the published headline only shows the default-prompt number,
+the methodology is incomplete. Always publish both, and call out
+the delta — that's the "we're not just measuring the prompt"
+test.
+
+## 4. (a) / (b) / (c) classifier
+
+A judge model (default Claude Sonnet 4.6 — see
+[`judge-prompt.md`](judge-prompt.md)) scores each per-task
+WITH-vs-WITHOUT comparison with one of three labels:
+
+- **(a) gortex helped** — the WITH run produced a measurably
+  better answer (more accurate, more complete, fewer
+  hallucinations, or substantially fewer tokens)
+- **(b) no measurable difference** — answers are roughly
+  equivalent in quality and cost
+- **(c) gortex hurt** — the WITH run was worse (less accurate,
+  more confused by the tool surface, or noticeably more tokens
+  for the same answer)
+
+The published summary MUST report all three counts. A
+"(a)=12 / (b)=2 / (c)=1" result is honest; "12 wins" is not.
+
+## 5. Negative-delta requirement
+
+Negative deltas (any (c) result) are **required** in the
+published summary, with the per-task breakdown linkable. The
+explicit requirement is the methodology's anti-survivorship
+mechanism: a methodology that buries (c) cases isn't measuring
+real-world quality, it's measuring marketing.
+
+If a published run reports zero (c) results across all 15 tasks,
+that's a red flag — either the judge is biased or the seed set
+is over-fit. Re-run with a different judge model and at least
+3 additional tasks per category before publishing.
+
+## 6. Scoring envelope
+
+Per-task token + wall-clock budgets are the same WITH and
+WITHOUT (typically 50k tokens / 5 minutes per task). A run that
+exceeds the budget scores as "no answer" — not as (c). This
+keeps the comparison about *quality* of answers, not endurance.
+
+Per-task cost is reported separately so a published row can show
+"answer (a) at 1.2× the WITHOUT cost"; (a) at 5× the cost is
+honestly weaker than (a) at 0.8× the cost.
+
+## 7. What we don't measure here
+
+- **Benchmark NDCG / recall** — covered by `bench/baselines/` and
+  `bench/token-efficiency/`. Different axis: those measure
+  retrieval quality independently of agent / model; this
+  methodology measures real agent behaviour.
+- **SWE-bench resolve rate** — covered by `BENCHMARK-SWE.md`.
+  Multi-day GPU compute; published separately when an operator
+  runs it.
+- **Performance** — covered by `bench/perf/`. Indexer / query /
+  impact latency, not answer quality.
+
+Together the three axes (retrieval / agent behaviour / system
+perf) form gortex's published-quality envelope.

docs/04-evaluation/run.md

@@ -0,0 +1,141 @@
+# How to run
+
+Operational recipe for executing the full methodology against a
+tagged build and publishing the results.
+
+## Prerequisites
+
+- `gortex` binary built from the tagged commit
+- `python3 -m pip install -r eval/requirements.txt` (Python
+  harness deps; existing file)
+- API keys: `ANTHROPIC_API_KEY` (for Sonnet 4.6),
+  `OPENAI_API_KEY` (for GPT 5.4), and a working `gh copilot`
+  installation (for Copilot CLI). At least one is enough for a
+  partial run.
+- A working corpus checkout (default: the gortex repo itself)
+
+## End-to-end
+
+```sh
+# 1) Tag the build — every published result cites this SHA.
+git rev-parse HEAD > eval/results/$(date +%Y%m%d)/HEAD.sha
+
+# 2) Run the full matrix: 15 tasks × 3 agents × 2 modes × 2 prompts.
+#    Estimated wall clock: ~3-6 hours per agent.
+gortex eval run \
+    --task-set docs/04-evaluation/task-set.md \
+    --judge-prompt docs/04-evaluation/judge-prompt.md \
+    --agents sonnet-4.6,gpt-5.4,copilot \
+    --corpus . \
+    --out eval/results/$(date +%Y%m%d)/ \
+    --max-task-tokens 50000 \
+    --max-task-seconds 300
+
+# 3) Aggregate per-task scores into the summary table.
+python3 eval/scripts/aggregate.py \
+    --workdir eval/results/$(date +%Y%m%d)/ \
+    --judges sonnet-4.6,gpt-5.4 \
+    --out eval/results/$(date +%Y%m%d)/summary.md
+
+# 4) Spot-check 5 random tasks per category by hand BEFORE
+#    publishing. The judge is good, not infallible.
+python3 eval/scripts/spotcheck.py \
+    --workdir eval/results/$(date +%Y%m%d)/ \
+    --sample 5 \
+    --out eval/results/$(date +%Y%m%d)/spotcheck.md
+
+# 5) Promote into BENCHMARK.md (manual edit; doc owner).
+$EDITOR BENCHMARK.md
+```
+
+## What lands on disk
+
+```
+eval/results/<date>/
+├── HEAD.sha                            # tagged commit
+├── summary.md                          # the published table
+├── spotcheck.md                        # manual review notes
+├── disagreement.md                     # judge-vs-judge disagreement
+├── per-task/
+│   ├── 1.1-indexer-walkthrough/
+│   │   ├── sonnet-4.6-with-default.json
+│   │   ├── sonnet-4.6-without-default.json
+│   │   ├── sonnet-4.6-with-ablation.json
+│   │   ├── sonnet-4.6-without-ablation.json
+│   │   ├── gpt-5.4-with-default.json
+│   │   └── ...
+│   ├── 1.2-community-detection/
+│   └── ...
+└── judge-runs/
+    ├── sonnet-4.6-judging/
+    └── gpt-5.4-judging/
+```
+
+Every per-task JSON contains: task prompt, canonical answer,
+agent answer, token cost, wall clock, tools called (count +
+list), and (if judged) the judge's label + reasoning + agreement
+between judges.
+
+## What to publish
+
+In `BENCHMARK.md`, add a section like:
+
+```markdown
+## Agent-graded evaluation
+
+**Last run: 2026-MM-DD** · agents: Sonnet 4.6 / GPT 5.4 /
+Copilot CLI · judge: Sonnet 4.6 + GPT 5.4 (agreement 87%)
+
+| Category | (a) gortex helped | (b) no difference | (c) gortex hurt |
+|----------|------------------:|------------------:|----------------:|
+| Architectural explanation |  6 | 1 | 2 |
+| Refactor safety           |  7 | 2 | 0 |
+| Bug localization          |  5 | 2 | 2 |
+| Impact analysis           |  8 | 1 | 0 |
+| Contract extraction       |  6 | 3 | 0 |
+| **Total**                 | 32 | 9 | 4 |
+
+- Default-prompt vs ablation-prompt delta:  +2 (a) / 0 (b) / -1 (c)
+  — gortex prompt steering helps but isn't load-bearing.
+- (c) cases written up in `eval/results/2026-MM-DD/c-cases.md`
+  — every loss has a public post-mortem.
+```
+
+**Required**: cite the (c) count, link to the (c)
+post-mortems, and call out the prompt-bias delta. A publication
+that hides (c) results is non-compliant with this methodology
+and should not be referenced as a benchmark.
+
+## Cost envelope
+
+A single full run (15 tasks × 3 agents × 2 modes × 2 prompts =
+180 agent runs + ~360 judge invocations) costs roughly:
+
+- Anthropic API:  ~$15-30 (Sonnet 4.6 agent + Sonnet 4.6 judge)
+- OpenAI API:     ~$10-25 (GPT 5.4 agent + GPT 5.4 judge)
+- Copilot CLI:    subscription-included
+
+Total: ~$25-55 per full run. Run quarterly + on every major
+version bump.
+
+## Partial-run modes
+
+When you only have one API key:
+
+```sh
+# Just Sonnet 4.6 (cheapest path)
+gortex eval run --agents sonnet-4.6 --task-set ...
+
+# Just one task category (smoke before the full run)
+gortex eval run --agents sonnet-4.6 \
+    --task-set docs/04-evaluation/task-set.md \
+    --categories "Refactor safety"
+
+# Just the WITH mode (compare absolute quality across agents)
+gortex eval run --agents sonnet-4.6,gpt-5.4,copilot \
+    --modes with
+```
+
+Partial runs are useful for iteration but **don't publish
+partial-run numbers as benchmarks** — the methodology requires
+the full matrix.

docs/04-evaluation/task-set.md

@@ -0,0 +1,249 @@
+# Seed task set
+
+15 tasks (3 per category) the published methodology runs against
+on each release. Each task carries a **canonical answer** the
+judge uses as ground truth — written by a human expert before any
+agent runs. Adding tasks: append to the appropriate section,
+write the canonical answer first, only THEN run the harness.
+
+The task corpus is the gortex repo itself (eats its own dog
+food). Extending to other corpora means writing a new
+`task-set-<repo>.md` with the same structure and pointing the
+harness at it via `--task-set`.
+
+---
+
+## Category 1: Architectural explanation (3 tasks)
+
+### 1.1 Indexer pipeline walkthrough
+
+**Prompt**: "Walk me through how the gortex indexer processes a
+single new source file. Name the packages it crosses and the
+order of operations."
+
+**Canonical answer (~150 words)**:
+
+1. `indexer.Indexer.IndexCtx(root)` walks `root` via
+   `internal/indexer/scan.go::scanFiles`, producing one
+   `parseJob` per matching file.
+2. Each job is dispatched to `parser.Registry` (the language
+   plugin matching the file extension) via
+   `internal/parser/treesitter.go`.
+3. The parser extracts symbols / edges as
+   `parser.ExtractionResult`; `Indexer.processExtraction` writes
+   them into the `graph.Graph` and accumulates incoming-edge
+   tracking for the next phase.
+4. `Indexer.buildSearchIndex` (BM25 / Bleve) + `idx.embedder`
+   (if set) populate the search backends.
+5. Semantic enrichment (`internal/semantic`) runs LSP / SCIP
+   providers in parallel; resolved edges get
+   `Origin=lsp_resolved` for tier filtering.
+6. Returns `IndexResult` with file / node counts + duration.
+
+### 1.2 Community detection role
+
+**Prompt**: "What does the `internal/analysis/communities.go`
+package do, and how does it integrate with `smart_context`?"
+
+**Canonical answer (~80 words)**:
+
+Implements Leiden community detection on the graph. Output is a
+`CommunityResult` mapping node ID → community ID + per-community
+cohesion score. `smart_context` reads it via the Context.CommunityOf
+hook in the rerank pipeline: candidates sharing the session's
+home community get a locality boost. Recompute is triggered on
+graph re-warm; the result is cached in `Server.analysis`.
+
+### 1.3 Daemon dispatch path
+
+**Prompt**: "How does a MCP request hit the daemon and get
+routed back to a per-session response?"
+
+**Canonical answer (~120 words)**:
+
+A `gortex mcp` client opens a stdio JSON-RPC pipe; the daemon
+dispatcher (`internal/daemon/dispatcher.go::MCPDispatcher`)
+parses frames, looks up or creates a per-session `*mcp.Server`
+via `Sessions.GetOrCreate`, and forwards. The Server holds a
+shared `*graph.Graph` + per-session `tokenStats` +
+`sessionState` (notes, frecency, etc.). Responses go back the
+same pipe with the matching JSON-RPC id. Cross-session memory
+(notes / memories / feedback) is workspace-scoped via the
+session's resolved cwd → workspace ID.
+
+---
+
+## Category 2: Refactor safety (3 tasks)
+
+### 2.1 Rename a public method
+
+**Prompt**: "I'm renaming `Indexer.Index` to `Indexer.IndexRoot`.
+List every caller that needs updating."
+
+**Canonical answer** (verified via `gortex find_usages
+gortex/internal/indexer/indexer.go::Indexer.Index`):
+
+- `gortex/internal/indexer/multi.go::MultiIndexer.IndexRepo`
+- `gortex/cmd/gortex/eval_recall.go::runEvalRecall`
+- `gortex/bench/perf/runner.go::runRepo` (introduced in the L5
+  bench commit)
+- `gortex/bench/token-efficiency/runner.go::indexRepoForBench`
+- All `*_test.go` files calling `idx.Index(...)` (count: see
+  `find_usages` output)
+
+### 2.2 Change a signature
+
+**Prompt**: "I want to add a `context.Context` to `Engine.SearchSymbols`.
+List every caller and what they'll need to change."
+
+**Canonical answer** (verified via `gortex verify_change`): see
+`find_usages` output; ~12 callers across `cmd/gortex/`,
+`internal/mcp/`, `bench/perf/`, `bench/token-efficiency/`. Each
+needs to pass through the request's context (most have one
+available; some need to use `context.Background()` for now).
+
+### 2.3 Remove a deprecated field
+
+**Prompt**: "I want to remove the `Edge.LegacyConfidence` field.
+What breaks?"
+
+**Canonical answer**: the field doesn't exist; the canonical
+answer is "no such field; nothing breaks". A passing agent
+should say so explicitly, not hallucinate impact.
+
+---
+
+## Category 3: Bug localization (3 tasks)
+
+### 3.1 Panic trace
+
+**Prompt**: "We have a panic in `internal/savings/store.go` at
+`flushLocked`. What conditions could cause it?"
+
+**Canonical answer**: `flushLocked` runs under `s.mu`. Panic
+paths: (a) flock acquisition fails (file system permission /
+disk full → wrapped, not panicked); (b) atomic-rename fails on
+some filesystems → returns error; (c) gob encode fails on
+unexpected map shape → would panic in `encoder.Encode`. Most
+likely candidate: corruption of the in-memory `s.file.PerRepo`
+map by a goroutine that bypassed the mutex.
+
+### 3.2 Wrong rank order
+
+**Prompt**: "After my recent rerank-signal change, the top
+result for `validateToken` is now a test file instead of the
+real implementation. What signal probably regressed?"
+
+**Canonical answer**: `path_penalty` (the test-file demotion).
+If a path matching the test-file regex stopped getting the ×0.3
+multiplier, test files would no longer be demoted. Check
+`signals_path_penalty.go::classifyPathPenalty` and the regex
+patterns in `pathRETest`.
+
+### 3.3 Cross-repo missing edges
+
+**Prompt**: "After indexing a multi-repo workspace, calls from
+`web` to `cloud_web` aren't showing up in `get_callers`. What
+gives?"
+
+**Canonical answer**: cross-repo resolution depends on
+`internal/resolver/cross_repo.go::CrossRepoResolver`; it only
+runs when `MultiIndexer` has indexed all repos in the same
+workspace. Most likely cause: one of the repos was indexed in
+isolation (not via `gortex track`) so the resolver never saw
+the cross-repo `import` edges.
+
+---
+
+## Category 4: Impact analysis (3 tasks)
+
+### 4.1 Touch a hot path
+
+**Prompt**: "I'm about to change the signature of
+`tokens.Count`. List the test files that need re-running."
+
+**Canonical answer**: use `gortex get_test_targets
+internal/tokens/tokens.go::Count`. Expected hits include
+`internal/tokens/tokens_test.go`, every test in
+`internal/mcp/` that calls `tokenStatsFor`, the savings tests,
+the bench harnesses' test files.
+
+### 4.2 Blast radius
+
+**Prompt**: "If I introduce a bug in `Graph.AllNodes`, what's
+the worst-case downstream effect?"
+
+**Canonical answer**: AllNodes is called by basically every
+analyzer; impact is "the whole codebase". A canonical answer
+quantifies it via `gortex explain_change_impact` (depth=3) and
+notes which communities / processes are at risk.
+
+### 4.3 Cycle detection
+
+**Prompt**: "Would adding an `imports` edge from
+`internal/search/rerank` to `internal/search` create a cycle?"
+
+**Canonical answer**: yes if `internal/search` already imports
+`internal/search/rerank` (parent-child importing child). Verify
+via `gortex analyze kind=would_create_cycle from_id=...
+to_id=...`. Currently `internal/search/hybrid.go` imports
+`internal/search/rerank` for the auto-α blend, so adding the
+reverse would form a cycle.
+
+---
+
+## Category 5: Contract extraction (3 tasks)
+
+### 5.1 Public API of a package
+
+**Prompt**: "List every exported function / method / type in
+`internal/savings/`, with one-line summaries."
+
+**Canonical answer**: use `gortex contracts list
+internal/savings`. Expected ~12 symbols:
+`Pricing`, `CostAvoided`, `CostAvoidedAll`, `Store`, `Open`,
+`DefaultPath`, `EventsPathFor`, `Store.AddObservation`,
+`Store.Snapshot`, `Store.Flush`, `Store.Reset`, `Bucket`,
+`Event`, `LoadEvents`, `BarString`, `SavingsPercent`,
+`AggregateByTool`, `FilterDay`, `FilterSince`,
+`BuildDashboard`. Plus the canonical pricing model constants.
+
+### 5.2 Tool surface of a package
+
+**Prompt**: "What MCP tools does `internal/mcp/tools_savings.go`
+register, and what are their parameter contracts?"
+
+**Canonical answer**: pull from
+`internal/mcp/tools_savings.go::registerSavingsTools` (or
+equivalent). Tool list + per-tool param schema. A passing agent
+should produce the exact param names + types.
+
+### 5.3 Configuration surface
+
+**Prompt**: "What `.gortex.yaml` keys does the indexer respect?
+Group by required vs optional."
+
+**Canonical answer**: cross-reference `internal/config/config.go`
++ each parser's `RegisterX` call. Required: none (all keys have
+defaults). Optional: `index.exclude`, `index.max_file_size`,
+`semantic.enable_*`, etc. A passing agent should produce the
+list with default values per key.
+
+---
+
+## Curation rules
+
+1. **Canonical answers come first.** Writing them after seeing
+   agent outputs is methodology fraud — the temptation to
+   "match" the agent's wording corrupts the ground truth.
+2. **One topic per task.** A task that asks two questions splits
+   the (a)/(b)/(c) signal — judge gives partial credit, which
+   makes the headline noisy.
+3. **Verify with the tools.** Every task's canonical answer
+   should be reproducible by running gortex tools manually; if
+   you can't reproduce it, the answer is wrong (or the tool
+   has a bug worth filing).
+4. **Bias toward realistic prompts.** The seed set is drawn
+   from actual user sessions (anonymized). Synthetic prompts
+   ("explain this complex graph algorithm") aren't what real
+   users ask.