chore: import upstream snapshot with attribution
CI / benchmark (push) Has been skipped
install-script / posix-syntax (push) Successful in 6m1s
CI / build-onnx (push) Failing after 6m43s
init-smoke / dry-run (push) Failing after 15m57s
security / govulncheck (push) Has been cancelled
security / trivy-fs (push) Has been cancelled
CI / test (1.26, ubuntu-latest) (push) Has been cancelled
Scorecard supply-chain security / Scorecard analysis (push) Has been cancelled
CI / test (1.26, macos-latest) (push) Has been cancelled
CI / build-windows (push) Has been cancelled
CI / lint (push) Has been cancelled
install-script / powershell-syntax (push) Has been cancelled
install-script / install (macos-14) (push) Has been cancelled
install-script / install (ubuntu-latest) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:33:42 +08:00
commit a06f331eb8
3186 changed files with 689843 additions and 0 deletions
+76
View File
@@ -0,0 +1,76 @@
# Retrieval baselines bench
Reproducible per-baseline NDCG@10 + latency table across six
candidate retrieval systems. The harness ships **six** adapters
behind a single uniform Go-side surface; per-adapter `Available()`
checks let `--smoke` verify wiring without paying for the heavy
ones.
Supported adapters:
| Adapter | Type | Install |
|----------------|---------------|--------------------------------------------------------------------|
| ripgrep | Go-native | `brew install ripgrep` (or distro equivalent) |
| probe | Go-native | `cargo install probe-ai` |
| colgrep | Python | `pip install colgrep` (requires CUDA-capable GPU) |
| grepai | Python | `pip install grepai-cli` |
| coderankembed | Python+model | `pip install sentence-transformers transformers torch` (model ~440MB on first run) |
| semble | Python | `pip install semble` |
## Running
```sh
# Smoke check: report which adapters are available right now
go run ./bench/baselines -smoke
# Full table against the gortex repo itself, all adapters
go run ./bench/baselines
# Just the locally-installable ones (ripgrep + probe)
go run ./bench/baselines -against ripgrep,probe
# JSON output for downstream tooling
go run ./bench/baselines -format json -json bench/results/baselines.json
```
Flags:
- `-repo PATH` — corpus to query (default `.`)
- `-queries PATH` — JSON query set (default `queries.json` here)
- `-groundtruth PATH` — JSON per-query expected file paths (default
`groundtruth.json` here)
- `-against LIST` — comma-separated adapter names; unknown names
error out so a typo doesn't silently drop a baseline
- `-top-k N` — top-K per query (default 10, matches NDCG@10)
- `-out PATH` — primary output (default stdout)
- `-json PATH` — companion JSON metrics output
- `-format markdown|json` — primary format
- `-smoke` — skip runs; only probe `Available()` for each adapter
- `-timeout DURATION` — per-adapter wall-clock cap (default 5m)
## How NDCG@10 is computed
For each query the harness computes:
```
DCG@10 = Σ (gain_i / log2(i + 2)) for i in [0, min(K, |hits|))
IDCG@10 = Σ (1 / log2(i + 2)) for i in [0, min(K, |expected|))
NDCG@10 = DCG@10 / IDCG@10
```
with `gain_i = 1` when `hits[i]` is in the ground-truth set,
`0` otherwise. The reported number is the **mean** across queries
with a non-empty ground-truth entry. Queries the adapter failed
on (Error set) are skipped from the mean — they're shown in the
per-query JSON for debugging but don't penalize the headline.
## Adding an adapter
1. Add a struct in `adapters.go` that satisfies the `adapter`
interface (`Name`, `Available`, `Run`).
2. Register it in `allAdapters()` — that single list drives the CLI,
the smoke output, and `adapterByName`.
3. For Python baselines: embed `pythonBaselineAdapter` and call
`pythonModuleAvailable` / `runPythonModule` (or
`runPythonScript` if you need a custom wrapper). Document the
install in the table above.
+333
View File
@@ -0,0 +1,333 @@
// Package main defines the baseline adapter framework: a single Go
// surface that uniformly invokes external retrieval baselines
// (ripgrep, probe, ColGREP, grepai, CodeRankEmbed Hybrid, semble)
// against a shared query set, then reports per-baseline NDCG@10 +
// latency in a comparable table.
//
// Per-adapter contract:
//
// - Name() canonical baseline name, used in CLI / table cols
// - Available() cheap probe: does this baseline run on this box?
// - Run(queries) per-query result list; honest empty-on-failure
//
// Go-native baselines (ripgrep, probe) shell to their respective
// binaries. Python-heavy ones (ColGREP, grepai, CodeRankEmbed) shell
// to `python3 -m <module>` and require an explicit per-package install
// that the harness documents in bench/baselines/README.md but does
// NOT attempt automatically. This keeps `gortex eval baselines
// --smoke` cheap on a fresh CI runner — it reports each baseline's
// availability without trying to install anything.
package main
import (
"bytes"
"context"
"fmt"
"io"
"os/exec"
"sort"
"strings"
"time"
)
// adapter is the interface every baseline implementation satisfies.
type adapter interface {
Name() string
Available() (bool, string)
Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult
}
// queryResult is the per-query outcome from one adapter.
type queryResult struct {
Query string `json:"query"`
Hits []string `json:"hits"`
Latency time.Duration `json:"-"`
LatencyMs float64 `json:"latency_ms"`
Error string `json:"error,omitempty"`
}
// allAdapters returns the canonical adapter set, in stable order.
// Adding a new baseline = adding one line here + one adapter
// implementation. The smoke check uses this list directly so the
// CLI never drifts from the registry.
func allAdapters() []adapter {
return []adapter{
&ripgrepAdapter{},
&probeAdapter{},
&colgrepAdapter{},
&grepaiAdapter{},
&coderankAdapter{},
&sembleAdapter{},
}
}
// adapterByName looks up one adapter by canonical name; nil when no
// match (the caller should check before invoking).
func adapterByName(name string) adapter {
for _, a := range allAdapters() {
if strings.EqualFold(a.Name(), name) {
return a
}
}
return nil
}
// --- ripgrep --------------------------------------------------------
type ripgrepAdapter struct{}
func (*ripgrepAdapter) Name() string { return "ripgrep" }
func (*ripgrepAdapter) Available() (bool, string) {
if _, err := exec.LookPath("rg"); err != nil {
return false, "rg not on PATH"
}
return true, ""
}
func (*ripgrepAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
out := make([]queryResult, 0, len(queries))
for _, q := range queries {
start := time.Now()
cmd := exec.CommandContext(ctx, "rg", "--files-with-matches", q, repoRoot)
var buf bytes.Buffer
cmd.Stdout = &buf
cmd.Stderr = io.Discard
err := cmd.Run()
latency := time.Since(start)
r := queryResult{Query: q, Latency: latency, LatencyMs: msFrom(latency)}
if err != nil {
if ee, ok := err.(*exec.ExitError); ok && ee.ExitCode() == 1 {
// rg exit code 1 = no matches; not an error.
out = append(out, r)
continue
}
r.Error = err.Error()
out = append(out, r)
continue
}
hits := parseLines(buf.String(), repoRoot, topK)
r.Hits = hits
out = append(out, r)
}
return out
}
// --- probe ----------------------------------------------------------
type probeAdapter struct{}
func (*probeAdapter) Name() string { return "probe" }
func (*probeAdapter) Available() (bool, string) {
if _, err := exec.LookPath("probe"); err != nil {
return false, "probe binary not on PATH (install: cargo install probe-ai)"
}
return true, ""
}
func (*probeAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
out := make([]queryResult, 0, len(queries))
for _, q := range queries {
start := time.Now()
cmd := exec.CommandContext(ctx, "probe", "search", "--paths-only", q, repoRoot)
var buf bytes.Buffer
cmd.Stdout = &buf
cmd.Stderr = io.Discard
err := cmd.Run()
latency := time.Since(start)
r := queryResult{Query: q, Latency: latency, LatencyMs: msFrom(latency)}
if err != nil {
r.Error = err.Error()
out = append(out, r)
continue
}
r.Hits = parseLines(buf.String(), repoRoot, topK)
out = append(out, r)
}
return out
}
// --- ColGREP --------------------------------------------------------
type colgrepAdapter struct{ pythonBaselineAdapter }
func (*colgrepAdapter) Name() string { return "colgrep" }
func (a *colgrepAdapter) Available() (bool, string) {
return a.pythonModuleAvailable("colgrep",
"pip install colgrep (requires CUDA-capable GPU for the bundled ONNX model)")
}
func (a *colgrepAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
return a.runPythonModule(ctx, "colgrep", queries, repoRoot, topK)
}
// --- grepai ---------------------------------------------------------
type grepaiAdapter struct{ pythonBaselineAdapter }
func (*grepaiAdapter) Name() string { return "grepai" }
func (a *grepaiAdapter) Available() (bool, string) {
return a.pythonModuleAvailable("grepai_cli",
"pip install grepai-cli")
}
func (a *grepaiAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
return a.runPythonModule(ctx, "grepai_cli", queries, repoRoot, topK)
}
// --- CodeRankEmbed Hybrid -------------------------------------------
type coderankAdapter struct{ pythonBaselineAdapter }
func (*coderankAdapter) Name() string { return "coderankembed" }
func (a *coderankAdapter) Available() (bool, string) {
// CodeRankEmbed ships as a model on Hugging Face; the bench
// invoker is a small Python script we ship in
// bench/baselines/python/coderankembed_runner.py. The script
// requires `pip install sentence-transformers transformers
// torch` and is documented in bench/baselines/README.md.
return a.pythonModuleAvailable("sentence_transformers",
"pip install sentence-transformers transformers torch (multi-GB model download on first run)")
}
func (a *coderankAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
return a.runPythonScript(ctx, "bench/baselines/python/coderankembed_runner.py", queries, repoRoot, topK)
}
// --- semble ---------------------------------------------------------
type sembleAdapter struct{ pythonBaselineAdapter }
func (*sembleAdapter) Name() string { return "semble" }
func (a *sembleAdapter) Available() (bool, string) {
return a.pythonModuleAvailable("semble",
"pip install semble")
}
func (a *sembleAdapter) Run(ctx context.Context, queries []string, repoRoot string, topK int) []queryResult {
return a.runPythonModule(ctx, "semble.cli", queries, repoRoot, topK)
}
// --- shared Python-adapter plumbing ---------------------------------
// pythonBaselineAdapter embeds the common shell-out logic that all
// Python-based baselines reuse. Per-baseline behaviour comes from
// the module name + (optional) wrapper script path.
type pythonBaselineAdapter struct{}
// pythonModuleAvailable runs `python3 -c "import <mod>"`. Returns
// true when the import succeeds; otherwise returns false with the
// install hint so the smoke output is actionable.
func (pythonBaselineAdapter) pythonModuleAvailable(module, installHint string) (bool, string) {
pythons := []string{"python3", "python"}
var lastErr string
for _, py := range pythons {
if _, err := exec.LookPath(py); err != nil {
continue
}
cmd := exec.Command(py, "-c", "import "+module)
cmd.Stderr = io.Discard
if err := cmd.Run(); err == nil {
return true, ""
} else {
lastErr = err.Error()
}
}
if lastErr == "" {
return false, "python3 not on PATH"
}
return false, fmt.Sprintf("python module %q not importable (%s)", module, installHint)
}
// runPythonModule invokes the module's CLI via `python3 -m <module>`,
// forwarding the queries on stdin one per line. Mirrors how most
// Python retrieval baselines we wrap accept input.
func (pythonBaselineAdapter) runPythonModule(ctx context.Context, module string, queries []string, repoRoot string, topK int) []queryResult {
return pythonBaselineRun(ctx, []string{"-m", module}, queries, repoRoot, topK)
}
// runPythonScript invokes a wrapper script with the same I/O
// convention. The script lives under bench/baselines/python/ and is
// shipped with the harness so each adapter has a known-good entry
// point.
func (pythonBaselineAdapter) runPythonScript(ctx context.Context, scriptPath string, queries []string, repoRoot string, topK int) []queryResult {
return pythonBaselineRun(ctx, []string{scriptPath}, queries, repoRoot, topK)
}
// pythonBaselineRun is the shared dispatcher: pipes queries to the
// Python process on stdin and parses one-hit-per-line JSON from
// stdout. Each baseline's wrapper is responsible for emitting the
// canonical JSON shape: {"query": "...", "hits": ["path1", ...]}.
func pythonBaselineRun(ctx context.Context, args []string, queries []string, repoRoot string, topK int) []queryResult {
out := make([]queryResult, 0, len(queries))
for _, q := range queries {
start := time.Now()
// We invoke the script once per query so a per-query
// failure doesn't drag the whole run. Higher overhead than
// streaming but fundamentally honest.
baseArgs := append([]string{}, args...)
baseArgs = append(baseArgs, "--repo", repoRoot, "--top-k", fmt.Sprintf("%d", topK), "--query", q)
cmd := exec.CommandContext(ctx, "python3", baseArgs...)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
latency := time.Since(start)
r := queryResult{Query: q, Latency: latency, LatencyMs: msFrom(latency)}
if err != nil {
r.Error = fmt.Sprintf("%v: %s", err, strings.TrimSpace(stderr.String()))
out = append(out, r)
continue
}
r.Hits = parseLines(stdout.String(), repoRoot, topK)
out = append(out, r)
}
return out
}
// --- helpers --------------------------------------------------------
// parseLines splits the adapter's stdout into one hit per line,
// strips the repo-root prefix so all adapters return repo-relative
// paths, dedups while preserving order, and caps at topK.
func parseLines(s, repoRoot string, topK int) []string {
seen := map[string]bool{}
out := []string{}
for line := range strings.SplitSeq(s, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
rel := strings.TrimPrefix(line, repoRoot+"/")
if seen[rel] {
continue
}
seen[rel] = true
out = append(out, rel)
if topK > 0 && len(out) >= topK {
break
}
}
return out
}
func msFrom(d time.Duration) float64 {
return float64(d.Microseconds()) / 1000.0
}
// adapterNames returns the names in canonical order for table
// rendering / smoke output.
func adapterNames() []string {
all := allAdapters()
names := make([]string, len(all))
for i, a := range all {
names[i] = a.Name()
}
sort.Strings(names)
return names
}
+37
View File
@@ -0,0 +1,37 @@
{
"comment": "Per-query expected file paths against the gortex repo. NDCG@10 = mean over queries of DCG_at_10 / IDCG_at_10 where gain=1 for hits in this list and 0 otherwise. Mirrors the token-efficiency ground truth so the two benches are directly comparable.",
"queries": {
"AddObservation": [
"internal/savings/store.go"
],
"IsSymbolQuery": [
"internal/search/rerank/query_kind.go"
],
"FileCoherenceSignal": [
"internal/search/rerank/signals_file_coherence.go"
],
"alphaFuse": [
"internal/search/hybrid.go"
],
"savings dashboard rendering": [
"cmd/gortex/savings.go",
"internal/savings/events.go"
],
"rerank pipeline default signals": [
"internal/search/rerank/pipeline.go"
],
"Indexer Index method": [
"internal/indexer/indexer.go"
],
"graph build edges": [
"internal/graph/graph.go"
],
"MCP server start": [
"internal/mcp/server.go",
"cmd/gortex/mcp.go"
],
"token counting tiktoken": [
"internal/tokens/tokens.go"
]
}
}
+367
View File
@@ -0,0 +1,367 @@
// Command baselines runs the per-baseline retrieval comparison:
// dispatch the same query set through each registered adapter,
// compare the per-query hit lists against a shared ground truth,
// and emit a per-adapter NDCG@10 + latency table.
//
// Smoke mode (`--smoke`) skips actual runs and reports which adapters
// are available on the local box — useful from CI to verify wiring
// without paying for the heavy Python baselines.
package main
import (
"context"
"encoding/json"
"flag"
"fmt"
"math"
"os"
"path/filepath"
"sort"
"strings"
"time"
)
func main() {
repo := flag.String("repo", ".", "indexed corpus path")
queriesPath := flag.String("queries", "bench/baselines/queries.json", "JSON query set")
truthPath := flag.String("groundtruth", "bench/baselines/groundtruth.json", "JSON per-query expected file paths")
against := flag.String("against", "ripgrep,probe,colgrep,grepai,coderankembed,semble", "comma-separated adapter names to run (default: all)")
topK := flag.Int("top-k", 10, "top-K candidates per query (NDCG@10 uses K=10)")
out := flag.String("out", "", "output table path (default stdout)")
jsonOut := flag.String("json", "", "optional JSON metrics output")
format := flag.String("format", "markdown", "markdown | json")
smoke := flag.Bool("smoke", false, "skip runs; only probe availability of each requested adapter")
timeout := flag.Duration("timeout", 5*time.Minute, "per-adapter wall-clock cap")
flag.Parse()
requested, err := resolveAdapters(*against)
if err != nil {
die("%v", err)
}
if *smoke {
smokeReport(requested, *format, *out)
return
}
absRepo, err := filepath.Abs(*repo)
if err != nil {
die("repo path: %v", err)
}
if _, err := os.Stat(absRepo); err != nil {
die("repo path: %v", err)
}
queries, err := loadQueries(*queriesPath)
if err != nil {
die("queries: %v", err)
}
truth, err := loadGroundTruth(*truthPath)
if err != nil {
die("groundtruth: %v", err)
}
rows := make([]adapterRow, 0, len(requested))
for _, a := range requested {
fmt.Fprintf(os.Stderr, "[baselines] %s ... ", a.Name())
avail, why := a.Available()
if !avail {
fmt.Fprintf(os.Stderr, "skipped (%s)\n", why)
rows = append(rows, adapterRow{
Adapter: a.Name(),
Skipped: why,
})
continue
}
t0 := time.Now()
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
results := a.Run(ctx, queries, absRepo, *topK)
cancel()
elapsed := time.Since(t0)
row := adapterRow{
Adapter: a.Name(),
PerQuery: results,
TotalDuration: elapsed.String(),
}
row.MedianLatencyMs = medianLatency(results)
row.NDCGAt10 = ndcgAt10(results, truth, *topK)
fmt.Fprintf(os.Stderr, "NDCG@10 %.3f · median %.1fms · total %.1fs\n",
row.NDCGAt10, row.MedianLatencyMs, elapsed.Seconds())
rows = append(rows, row)
}
var primary []byte
switch strings.ToLower(*format) {
case "markdown", "md":
primary = []byte(renderMarkdown(rows))
case "json":
primary = mustMarshalJSON(rows)
default:
die("unknown --format %q", *format)
}
if err := writeOutput(*out, primary); err != nil {
die("write output: %v", err)
}
if *jsonOut != "" {
if err := writeOutput(*jsonOut, mustMarshalJSON(rows)); err != nil {
die("write json: %v", err)
}
}
}
// adapterRow is the per-baseline outcome with the columns the
// published table cares about.
type adapterRow struct {
Adapter string `json:"adapter"`
NDCGAt10 float64 `json:"ndcg_at_10"`
MedianLatencyMs float64 `json:"median_latency_ms"`
TotalDuration string `json:"total_duration"`
PerQuery []queryResult `json:"per_query,omitempty"`
Skipped string `json:"skipped,omitempty"`
}
// resolveAdapters expands the --against flag into the requested
// adapter set, preserving the caller's order. Unknown names error
// out so a typo doesn't silently drop a baseline.
func resolveAdapters(spec string) ([]adapter, error) {
if strings.TrimSpace(spec) == "" {
return allAdapters(), nil
}
var out []adapter
for tok := range strings.SplitSeq(spec, ",") {
name := strings.TrimSpace(tok)
if name == "" {
continue
}
a := adapterByName(name)
if a == nil {
return nil, fmt.Errorf("unknown adapter %q (known: %s)",
name, strings.Join(adapterNames(), ", "))
}
out = append(out, a)
}
if len(out) == 0 {
return allAdapters(), nil
}
return out, nil
}
// smokeReport prints one row per requested adapter showing
// available / not-available with the install hint. Output respects
// --format; default markdown.
func smokeReport(adapters []adapter, format, out string) {
if strings.ToLower(format) == "json" {
rows := make([]map[string]any, 0, len(adapters))
for _, a := range adapters {
avail, why := a.Available()
row := map[string]any{
"adapter": a.Name(),
"available": avail,
}
if why != "" {
row["why"] = why
}
rows = append(rows, row)
}
raw, _ := json.MarshalIndent(rows, "", " ")
_ = writeOutput(out, append(raw, '\n'))
return
}
var b strings.Builder
fmt.Fprintln(&b, "# Baseline-adapter smoke check")
fmt.Fprintln(&b)
fmt.Fprintln(&b, "| adapter | available | reason |")
fmt.Fprintln(&b, "|---------|:---------:|--------|")
for _, a := range adapters {
avail, why := a.Available()
mark := "✗"
if avail {
mark = "✓"
}
if why == "" {
why = "—"
}
fmt.Fprintf(&b, "| %s | %s | %s |\n", a.Name(), mark, why)
}
_ = writeOutput(out, []byte(b.String()))
}
// ndcgAt10 computes NDCG@10 across the result set. Per query: gain
// is 1 for ground-truth hits, 0 otherwise; DCG = sum gain_i /
// log2(i+2); IDCG = DCG of the perfect ordering. Mean across
// queries; 0 when the result set is empty.
func ndcgAt10(results []queryResult, truth map[string][]string, k int) float64 {
if len(results) == 0 {
return 0
}
var sum float64
counted := 0
for _, r := range results {
if r.Error != "" {
continue
}
expected := truth[r.Query]
if len(expected) == 0 {
continue
}
expSet := map[string]bool{}
for _, e := range expected {
expSet[e] = true
}
// DCG@k
var dcg float64
hits := r.Hits
if len(hits) > k {
hits = hits[:k]
}
for i, h := range hits {
if expSet[h] {
dcg += 1.0 / math.Log2(float64(i+2))
}
}
// IDCG@k: best case is min(len(expected), k) hits at the top.
idealHits := min(len(expected), k)
var idcg float64
for i := range idealHits {
idcg += 1.0 / math.Log2(float64(i+2))
}
if idcg == 0 {
continue
}
sum += dcg / idcg
counted++
}
if counted == 0 {
return 0
}
return sum / float64(counted)
}
func medianLatency(results []queryResult) float64 {
vs := make([]float64, 0, len(results))
for _, r := range results {
if r.Error == "" {
vs = append(vs, r.LatencyMs)
}
}
if len(vs) == 0 {
return 0
}
sort.Float64s(vs)
return vs[len(vs)/2]
}
// --- rendering ------------------------------------------------------
func renderMarkdown(rows []adapterRow) string {
var b strings.Builder
fmt.Fprintln(&b, "# Retrieval-baselines NDCG@10 + speed")
fmt.Fprintln(&b)
fmt.Fprintln(&b, "_NDCG@10 against the ground-truth set + median per-query latency for each baseline. Adapters reporting `skipped` weren't available on this box; see the install hints in `bench/baselines/README.md`._")
fmt.Fprintln(&b)
fmt.Fprintln(&b, "| adapter | NDCG@10 | median latency | total | status |")
fmt.Fprintln(&b, "|---------|--------:|---------------:|------:|--------|")
for _, r := range rows {
if r.Skipped != "" {
fmt.Fprintf(&b, "| %s | — | — | — | skipped: %s |\n", r.Adapter, r.Skipped)
continue
}
fmt.Fprintf(&b, "| %s | %.3f | %s | %s | ✓ |\n",
r.Adapter, r.NDCGAt10, fmtMs(r.MedianLatencyMs), r.TotalDuration)
}
fmt.Fprintln(&b)
fmt.Fprintln(&b, summaryLine(rows))
return b.String()
}
func summaryLine(rows []adapterRow) string {
ran := 0
bestName := ""
var bestScore float64
for _, r := range rows {
if r.Skipped != "" {
continue
}
ran++
if r.NDCGAt10 > bestScore {
bestScore = r.NDCGAt10
bestName = r.Adapter
}
}
if ran == 0 {
return "_no adapters available — install at least one baseline (see bench/baselines/README.md)_"
}
return fmt.Sprintf("**Summary:** %d/%d adapter(s) ran. Best NDCG@10: %s @ %.3f.",
ran, len(rows), bestName, bestScore)
}
func fmtMs(v float64) string {
if v == 0 {
return "—"
}
if v < 1 {
return fmt.Sprintf("%.2fms", v)
}
if v < 1000 {
return fmt.Sprintf("%.1fms", v)
}
return fmt.Sprintf("%.2fs", v/1000.0)
}
// --- helpers --------------------------------------------------------
func die(format string, args ...any) {
fmt.Fprintf(os.Stderr, "baselines: "+format+"\n", args...)
os.Exit(1)
}
func writeOutput(path string, body []byte) error {
if path == "" {
_, err := os.Stdout.Write(body)
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return os.WriteFile(path, body, 0o644)
}
func mustMarshalJSON(rows []adapterRow) []byte {
b, err := json.MarshalIndent(rows, "", " ")
if err != nil {
die("marshal json: %v", err)
}
return append(b, '\n')
}
func loadQueries(path string) ([]string, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var doc struct {
Queries []string `json:"queries"`
}
if err := json.Unmarshal(raw, &doc); err != nil {
return nil, fmt.Errorf("parse %s: %w", path, err)
}
return doc.Queries, nil
}
func loadGroundTruth(path string) (map[string][]string, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var doc struct {
Queries map[string][]string `json:"queries"`
}
if err := json.Unmarshal(raw, &doc); err != nil {
return nil, fmt.Errorf("parse %s: %w", path, err)
}
if doc.Queries == nil {
doc.Queries = map[string][]string{}
}
return doc.Queries, nil
}
+300
View File
@@ -0,0 +1,300 @@
package main
import (
"context"
"encoding/json"
"math"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestAllAdapters_RegisteredInStableOrder(t *testing.T) {
got := allAdapters()
want := []string{"ripgrep", "probe", "colgrep", "grepai", "coderankembed", "semble"}
if len(got) != len(want) {
t.Fatalf("adapter count = %d, want %d", len(got), len(want))
}
for i, a := range got {
if a.Name() != want[i] {
t.Errorf("adapter[%d] = %q, want %q", i, a.Name(), want[i])
}
}
}
func TestAdapterByName(t *testing.T) {
if a := adapterByName("ripgrep"); a == nil || a.Name() != "ripgrep" {
t.Errorf("adapterByName(ripgrep) = %v", a)
}
if a := adapterByName("RIPGREP"); a == nil { // case-insensitive
t.Error("adapterByName should be case-insensitive")
}
if a := adapterByName("nonexistent"); a != nil {
t.Errorf("adapterByName(nonexistent) should be nil, got %v", a)
}
}
func TestResolveAdapters_EmptyMeansAll(t *testing.T) {
got, err := resolveAdapters("")
if err != nil {
t.Fatal(err)
}
if len(got) != len(allAdapters()) {
t.Errorf("empty spec = %d adapters, want %d", len(got), len(allAdapters()))
}
}
func TestResolveAdapters_Subset(t *testing.T) {
got, err := resolveAdapters("ripgrep,probe")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 || got[0].Name() != "ripgrep" || got[1].Name() != "probe" {
t.Errorf("subset = %v, want [ripgrep, probe] in order", names(got))
}
}
func TestResolveAdapters_UnknownErrors(t *testing.T) {
if _, err := resolveAdapters("ripgrep,bogus"); err == nil {
t.Error("expected error for unknown adapter")
}
}
func TestNDCGAt10_PerfectRanking(t *testing.T) {
results := []queryResult{
{Query: "q", Hits: []string{"a", "b", "c"}},
}
truth := map[string][]string{"q": {"a", "b"}}
got := ndcgAt10(results, truth, 10)
// IDCG = 1/log2(2) + 1/log2(3) = 1 + 0.6309 = 1.6309
// DCG = 1/log2(2) + 1/log2(3) + 0 = same = 1.6309
// NDCG = 1.0
if got < 0.99 || got > 1.01 {
t.Errorf("perfect ranking NDCG@10 = %.4f, want 1.0", got)
}
}
func TestNDCGAt10_WrongOrderPenalized(t *testing.T) {
results := []queryResult{
{Query: "q", Hits: []string{"miss1", "miss2", "a", "b"}},
}
truth := map[string][]string{"q": {"a", "b"}}
got := ndcgAt10(results, truth, 10)
// DCG = 0 + 0 + 1/log2(4) + 1/log2(5) ≈ 0.5 + 0.431 = 0.931
// IDCG = 1/log2(2) + 1/log2(3) ≈ 1.0 + 0.631 = 1.631
// NDCG = 0.931/1.631 ≈ 0.571
if got > 0.7 {
t.Errorf("wrong-order NDCG@10 = %.4f, expected <0.7", got)
}
if got < 0.4 {
t.Errorf("wrong-order NDCG@10 = %.4f, expected >0.4", got)
}
}
func TestNDCGAt10_AllMissesScoreZero(t *testing.T) {
results := []queryResult{{Query: "q", Hits: []string{"x", "y"}}}
truth := map[string][]string{"q": {"a", "b"}}
if got := ndcgAt10(results, truth, 10); got != 0 {
t.Errorf("all-miss NDCG@10 = %.4f, want 0", got)
}
}
func TestNDCGAt10_EmptyTruthSkipped(t *testing.T) {
results := []queryResult{
{Query: "q1", Hits: []string{"a"}},
{Query: "q2", Hits: []string{"a"}},
}
truth := map[string][]string{"q2": {"a"}}
// q1 has no truth → skipped; q2 has 1 hit at position 0 → NDCG=1.
got := ndcgAt10(results, truth, 10)
if math.Abs(got-1.0) > 0.01 {
t.Errorf("mixed-truth NDCG@10 = %.4f, want ~1.0 (q1 should be skipped)", got)
}
}
func TestNDCGAt10_ErroredQuerySkipped(t *testing.T) {
results := []queryResult{
{Query: "q1", Error: "boom"},
{Query: "q2", Hits: []string{"a"}},
}
truth := map[string][]string{"q1": {"a"}, "q2": {"a"}}
// q1 errored → skipped; only q2 counts → NDCG=1.
got := ndcgAt10(results, truth, 10)
if math.Abs(got-1.0) > 0.01 {
t.Errorf("errored-skipped NDCG@10 = %.4f, want ~1.0", got)
}
}
func TestMedianLatency(t *testing.T) {
results := []queryResult{
{LatencyMs: 30},
{LatencyMs: 10},
{LatencyMs: 20},
}
if got := medianLatency(results); got != 20 {
t.Errorf("median = %.2f, want 20", got)
}
}
func TestMedianLatency_ErrorsExcluded(t *testing.T) {
results := []queryResult{
{LatencyMs: 30, Error: "boom"},
{LatencyMs: 10},
{LatencyMs: 20},
}
if got := medianLatency(results); got != 20 {
t.Errorf("median (errors excluded) = %.2f, want 20", got)
}
}
func TestParseLines_DedupAndCap(t *testing.T) {
body := `/repo/a.go
/repo/b.go
/repo/a.go
/repo/c.go
/repo/d.go
`
got := parseLines(body, "/repo", 3)
want := []string{"a.go", "b.go", "c.go"}
if len(got) != 3 {
t.Fatalf("got %v, want %v", got, want)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("got[%d] = %q, want %q", i, got[i], want[i])
}
}
}
func TestParseLines_TopKZeroMeansUnlimited(t *testing.T) {
body := "a\nb\nc\nd\ne\nf\ng\n"
got := parseLines(body, "", 0)
if len(got) != 7 {
t.Errorf("got %d lines, want 7 (topK=0 unlimited)", len(got))
}
}
func TestRenderMarkdown_HasHeaderAndRows(t *testing.T) {
rows := []adapterRow{
{Adapter: "ripgrep", NDCGAt10: 0.45, MedianLatencyMs: 12.3, TotalDuration: "1.2s"},
{Adapter: "colgrep", Skipped: "python module not importable"},
}
md := renderMarkdown(rows)
for _, want := range []string{
"# Retrieval-baselines NDCG@10",
"| ripgrep |",
"0.450",
"| colgrep |",
"skipped:",
"**Summary:**",
} {
if !strings.Contains(md, want) {
t.Errorf("markdown missing %q\n----\n%s", want, md)
}
}
}
func TestSmokeReport_Markdown(t *testing.T) {
dir := t.TempDir()
out := filepath.Join(dir, "smoke.md")
smokeReport([]adapter{&ripgrepAdapter{}, &colgrepAdapter{}}, "markdown", out)
raw, err := os.ReadFile(out)
if err != nil {
t.Fatal(err)
}
body := string(raw)
for _, want := range []string{
"# Baseline-adapter smoke check",
"| ripgrep |",
"| colgrep |",
} {
if !strings.Contains(body, want) {
t.Errorf("smoke markdown missing %q\n----\n%s", want, body)
}
}
}
func TestSmokeReport_JSON(t *testing.T) {
dir := t.TempDir()
out := filepath.Join(dir, "smoke.json")
smokeReport([]adapter{&ripgrepAdapter{}}, "json", out)
raw, err := os.ReadFile(out)
if err != nil {
t.Fatal(err)
}
var got []map[string]any
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("smoke JSON unparseable: %v\n%s", err, raw)
}
if len(got) != 1 || got[0]["adapter"] != "ripgrep" {
t.Errorf("smoke JSON shape wrong: %v", got)
}
if _, ok := got[0]["available"]; !ok {
t.Errorf("smoke JSON missing 'available' field: %v", got)
}
}
func TestRipgrepAdapter_NoMatchesReturnsEmpty(t *testing.T) {
a := &ripgrepAdapter{}
avail, _ := a.Available()
if !avail {
t.Skip("rg not installed on this box")
}
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "f.go"), []byte("package x\n"), 0o644); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
out := a.Run(ctx, []string{"definitely_not_in_the_corpus_XYZ"}, dir, 10)
if len(out) != 1 {
t.Fatalf("got %d results, want 1", len(out))
}
if out[0].Error != "" {
t.Errorf("no-match should not error, got %q", out[0].Error)
}
if len(out[0].Hits) != 0 {
t.Errorf("no-match should yield empty hits, got %v", out[0].Hits)
}
}
func TestLoadQueries(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "q.json")
if err := os.WriteFile(path, []byte(`{"queries":["a","b","c"]}`), 0o644); err != nil {
t.Fatal(err)
}
got, err := loadQueries(path)
if err != nil {
t.Fatal(err)
}
if len(got) != 3 {
t.Errorf("got %d, want 3", len(got))
}
}
func TestLoadGroundTruth(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "gt.json")
body := `{"queries":{"q":["a.go"]}}`
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
got, err := loadGroundTruth(path)
if err != nil {
t.Fatal(err)
}
if len(got["q"]) != 1 || got["q"][0] != "a.go" {
t.Errorf("got %v, want q:[a.go]", got)
}
}
func names(adapters []adapter) []string {
out := make([]string, len(adapters))
for i, a := range adapters {
out[i] = a.Name()
}
return out
}
@@ -0,0 +1,92 @@
#!/usr/bin/env python3
"""CodeRankEmbed Hybrid baseline runner.
Wrapper that lets the Go-side bench/baselines harness invoke
CodeRankEmbed Hybrid without per-baseline Go code growing model-
download logic. Usage:
python3 bench/baselines/python/coderankembed_runner.py \\
--repo PATH --query "validateToken" --top-k 10
Emits one repo-relative path per line on stdout. Errors go to
stderr; non-zero exit when the model isn't available.
Install: `pip install sentence-transformers transformers torch`.
First run downloads the CodeRankEmbed model (~440 MB).
"""
import argparse
import os
import sys
from pathlib import Path
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--repo", required=True, help="indexed corpus path")
ap.add_argument("--query", required=True, help="single query string")
ap.add_argument("--top-k", type=int, default=10)
args = ap.parse_args()
try:
from sentence_transformers import SentenceTransformer
except ImportError as e:
print(
f"coderankembed_runner: missing dependency ({e}). "
"pip install sentence-transformers transformers torch",
file=sys.stderr,
)
return 2
model = SentenceTransformer("nomic-ai/CodeRankEmbed", trust_remote_code=True)
# Index every file under repo (cheap for sub-million LoC; the
# ground-truth fixture is the gortex repo itself).
repo = Path(args.repo).resolve()
paths: list[Path] = []
texts: list[str] = []
for p in repo.rglob("*"):
if not p.is_file():
continue
if any(seg.startswith(".") for seg in p.relative_to(repo).parts):
continue
if p.suffix.lower() not in {
".go", ".py", ".ts", ".tsx", ".js", ".jsx", ".rs",
".java", ".kt", ".swift", ".rb", ".cs", ".cpp", ".c",
".h", ".hpp", ".md", ".yaml", ".yml", ".json",
}:
continue
try:
text = p.read_text(errors="ignore")
except OSError:
continue
if not text.strip():
continue
paths.append(p)
texts.append(text[:8000]) # truncate to keep the embed cheap
if not texts:
print(
"coderankembed_runner: no indexable files under repo",
file=sys.stderr,
)
return 1
embeds = model.encode(texts, show_progress_bar=False, convert_to_numpy=True)
qe = model.encode([args.query], show_progress_bar=False, convert_to_numpy=True)[0]
# Cosine similarity → rank.
import numpy as np
sims = embeds @ qe / (
(np.linalg.norm(embeds, axis=1) * np.linalg.norm(qe)) + 1e-12
)
order = np.argsort(-sims)[: args.top_k]
for idx in order:
rel = paths[idx].relative_to(repo)
print(rel.as_posix())
return 0
if __name__ == "__main__":
sys.exit(main())
+15
View File
@@ -0,0 +1,15 @@
{
"comment": "Query set for the retrieval-baselines NDCG@10 bench. Same shape as the token-efficiency set so the two benches stay comparable: an adapter that scores well here should also produce low tokens-per-response there.",
"queries": [
"AddObservation",
"IsSymbolQuery",
"FileCoherenceSignal",
"alphaFuse",
"savings dashboard rendering",
"rerank pipeline default signals",
"Indexer Index method",
"graph build edges",
"MCP server start",
"token counting tiktoken"
]
}
+105
View File
@@ -0,0 +1,105 @@
# Daemon-mode MCP-tool latency
Per-tool p50 / p95 / p99 latency for the production MCP dispatch
path. Builds an in-process MCP server against a target corpus,
fires N `Handler.CallToolStrict` invocations per tool, aggregates
latencies into a published table.
## What it measures
- **Handler-end-to-end latency** for each MCP tool: JSON arg parse
→ tool dispatch → handler logic → response encode. Same code
path the production stdio / HTTP / daemon-socket front-ends use.
- **Per-tool spread**: cheap tools (`graph_stats`, `get_callers`)
separate from heavy ones (`smart_context`, `get_repo_outline`)
so the published table shows realistic operating envelope.
## What it does NOT measure
- **Stdio framing** (gortex mcp's pipe overhead)
- **Daemon socket dispatch** (gortex daemon's UNIX socket / HTTP
ingress overhead)
- **Network RTT** (if reaching the daemon remotely)
Each adds a roughly constant ~0.1-1 ms per call on a warm pipe;
the handler latency below dominates user-perceived response time.
## Running
```sh
# Default: index `.` and fire 200 iters per tool
go run ./bench/daemon-latency
# Higher iter count for tighter percentiles
go run ./bench/daemon-latency -iter 500
# Specific subset of tools (useful for tuning one signal)
go run ./bench/daemon-latency -tools graph_stats,search_symbols
# CSV / JSON outputs for downstream tooling
go run ./bench/daemon-latency -csv bench/results/dl.csv -json bench/results/dl.json
```
Flags:
- `-repo PATH` — corpus to index (default `.`)
- `-iter N` — iterations per tool (default 200; warm-up of N/10
is added on top)
- `-tools LIST` — comma-separated subset
- `-out PATH` — primary output (default stdout)
- `-csv PATH` / `-json PATH` — companion outputs
- `-format markdown|csv|json` — primary format
Or via the CLI surface:
```sh
gortex bench daemon-latency --out-dir bench/results
```
## Tools benchmarked
| tool | shape |
|------|-------|
| `graph_stats` | no-arg snapshot; cheap |
| `search_symbols` | 1 query arg; rotated through 10 fixtures so a per-query cache doesn't trivially hit |
| `get_symbol_source` | 1 id arg; pinned to a sampled function from the indexed graph |
| `get_callers` | 1 id arg + limit |
| `find_usages` | 1 id arg |
| `get_file_summary` | 1 path arg; pinned to a sampled file |
| `smart_context` | 1 task arg; expensive, fewer iters per cycle |
| `get_repo_outline` | no-arg; walks whole graph |
Sampled targets are picked once at start so each tool sees the
same target across iterations — the per-call latency reflects
handler arithmetic, not target lookup.
## Methodology
- Warm-up of `iter/10` (min 5) per tool primes any lazy
initialisation in the handler / graph before the measured loop
starts.
- Per-iteration latency captured via `time.Since(start)` with
μs precision.
- Percentiles computed via the nearest-rank method:
`idx = (pct × n) / 100`. For N=200 → p95=sorted[190].
- Errors are counted in `error_rate` but their latencies are
still measured (an error path that takes 3× the happy-path
time is itself a signal).
## Honest caveats
- Numbers are operator-machine-specific. Absolute values vary 2-5×
across hardware classes; the **relative spread** between tools
(cheap vs heavy) is what publishes reproducibly.
- Cold-cache effects show up most in `search_symbols` (BM25
re-ranks under load) and `smart_context` (assembles fresh
context each call). Warm-up reduces but doesn't eliminate them.
- Smoke run on the gortex repo (71k nodes, Apple M3 Max):
- `graph_stats` p50 4.2ms · p95 5.5ms
- `search_symbols` p50 1.2ms · p95 22.4ms
- `get_symbol_source` p50 0.19ms · p95 0.9ms
- `get_callers` / `find_usages` p50 < 0.02ms (graph lookup)
- `smart_context` p50 1.5ms · p95 24ms
- `get_repo_outline` p50 60ms · p95 217ms
Median p95 across tools: 5.5 ms. Median p99: 5.9 ms.
+447
View File
@@ -0,0 +1,447 @@
// Command daemon-latency measures per-tool MCP dispatch latency.
// Builds an in-process MCP server against a target corpus, fires N
// `CallTool` invocations per tool, reports p50 / p95 / p99 per tool
// and a top-line summary.
//
// What it measures: tool-handler latency end-to-end through the
// real MCP dispatch path (`Handler.CallTool` invoked via the same
// `server.MCPServer` the production stdio / HTTP / daemon
// front-ends use). What it does NOT measure: stdio framing,
// daemon socket dispatch, JSON-RPC envelope overhead. Those add a
// small constant per call (typically <1 ms on a warm pipe); the
// handler latency dominates the user-perceived response time.
//
// The bench therefore reflects "daemon-mode handler cost", which
// is the load-bearing number for the daemon-latency publication.
package main
import (
"context"
"encoding/csv"
"encoding/json"
"flag"
"fmt"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"time"
"go.uber.org/zap"
"github.com/zzet/gortex/internal/config"
"github.com/zzet/gortex/internal/graph"
"github.com/zzet/gortex/internal/indexer"
gortexmcp "github.com/zzet/gortex/internal/mcp"
"github.com/zzet/gortex/internal/parser"
"github.com/zzet/gortex/internal/parser/languages"
"github.com/zzet/gortex/internal/query"
internalserver "github.com/zzet/gortex/internal/server"
)
// toolCall is one synthetic MCP request the bench fires per
// iteration. ArgsFn lets the bench vary the args across iterations
// (e.g. different query strings) so the dispatch path isn't
// trivially memoised by an upstream cache.
type toolCall struct {
Tool string
ArgsFn func(iter int) map[string]any
WarmupN int
IterN int
// SkipIfMissing lets a tool opt out when its substrate isn't
// in the indexed graph (e.g. nothing to call get_callers on).
SkipIfMissing func(g *graph.Graph) bool
}
// result captures the per-tool aggregate the bench publishes.
type result struct {
Tool string `json:"tool"`
Iters int `json:"iters"`
P50Ms float64 `json:"p50_ms"`
P95Ms float64 `json:"p95_ms"`
P99Ms float64 `json:"p99_ms"`
MeanMs float64 `json:"mean_ms"`
MaxMs float64 `json:"max_ms"`
ErrorRate float64 `json:"error_rate"`
Skipped string `json:"skipped,omitempty"`
Started time.Time `json:"-"`
}
func main() {
repo := flag.String("repo", ".", "corpus to index for the bench")
iter := flag.Int("iter", 200, "iterations per tool (warm-up of iter/10 is added on top)")
out := flag.String("out", "", "primary output path (default stdout)")
jsonOut := flag.String("json", "", "companion JSON metrics output")
csvOut := flag.String("csv", "", "companion CSV output")
format := flag.String("format", "markdown", "markdown | json | csv")
tools := flag.String("tools", "", "comma-separated subset (default: all known tools)")
flag.Parse()
absRepo, err := filepath.Abs(*repo)
if err != nil {
die("repo path: %v", err)
}
fmt.Fprintf(os.Stderr, "[daemon-latency] indexing %s...\n", absRepo)
g, srv := buildInProcessServer(absRepo)
fmt.Fprintf(os.Stderr, "[daemon-latency] indexed %d nodes\n", len(g.AllNodes()))
handler := internalserver.NewHandler(srv.MCPServer(), g, "bench", zap.NewNop())
// Build the call set against the freshly indexed graph so each
// synthetic request has at least some structural validity (a
// real symbol id, an extant file path).
calls := defaultCalls(g, *iter)
if *tools != "" {
calls = filterCalls(calls, strings.Split(*tools, ","))
}
rows := make([]result, 0, len(calls))
for _, c := range calls {
if c.SkipIfMissing != nil && c.SkipIfMissing(g) {
rows = append(rows, result{Tool: c.Tool, Iters: 0, Skipped: "no eligible substrate in indexed graph"})
fmt.Fprintf(os.Stderr, "[daemon-latency] %-22s skipped (no substrate)\n", c.Tool)
continue
}
row := runOne(handler, c)
rows = append(rows, row)
fmt.Fprintf(os.Stderr, "[daemon-latency] %-22s p50=%6.2fms p95=%6.2fms p99=%6.2fms iters=%d errs=%.0f%%\n",
c.Tool, row.P50Ms, row.P95Ms, row.P99Ms, row.Iters, row.ErrorRate*100)
}
var primary []byte
switch strings.ToLower(*format) {
case "markdown", "md":
primary = []byte(renderMarkdown(rows, absRepo, g))
case "csv":
primary = []byte(renderCSV(rows))
case "json":
primary = mustMarshalJSON(rows)
default:
die("unknown --format %q", *format)
}
if err := writeOutput(*out, primary); err != nil {
die("write output: %v", err)
}
if *csvOut != "" {
if err := writeOutput(*csvOut, []byte(renderCSV(rows))); err != nil {
die("write csv: %v", err)
}
}
if *jsonOut != "" {
if err := writeOutput(*jsonOut, mustMarshalJSON(rows)); err != nil {
die("write json: %v", err)
}
}
}
// --- in-process server ---------------------------------------------
// buildInProcessServer wires the same Server the production stdio /
// daemon front-ends use, against a fresh in-process graph of repoRoot.
// Identical wiring to `cmd/gortex/eval_recall.go`'s indexed-server
// path so the bench reflects production handler arithmetic.
func buildInProcessServer(repoRoot string) (*graph.Graph, *gortexmcp.Server) {
g := graph.New()
reg := parser.NewRegistry()
languages.RegisterAll(reg)
cfg := config.Config{}
idx := indexer.New(g, reg, cfg.Index, zap.NewNop())
if _, err := idx.Index(repoRoot); err != nil {
die("index %s: %v", repoRoot, err)
}
eng := query.NewEngine(g)
eng.SetSearch(idx.Search())
srv := gortexmcp.NewServer(eng, g, idx, nil, zap.NewNop(), cfg.Guards.Rules)
srv.RunAnalysis()
return g, srv
}
// --- call set -------------------------------------------------------
// defaultCalls returns the canonical bench surface. We focus on
// tools agents actually call in production (the headline savings
// drivers) — covering both cheap (graph_stats) and expensive
// (smart_context) shapes so the published table shows the spread.
func defaultCalls(g *graph.Graph, iter int) []toolCall {
if iter <= 0 {
iter = 200
}
warmup := max(iter/10, 5)
// Pick representative symbol IDs / file paths from the indexed
// graph so the synthetic requests have real targets.
var sampleFnID, sampleFilePath string
for _, n := range g.AllNodes() {
if n == nil {
continue
}
if sampleFnID == "" && (n.Kind == graph.KindFunction || n.Kind == graph.KindMethod) {
sampleFnID = n.ID
}
if sampleFilePath == "" && n.Kind == graph.KindFile && n.FilePath != "" {
sampleFilePath = n.FilePath
}
if sampleFnID != "" && sampleFilePath != "" {
break
}
}
queries := []string{
"validateToken", "Indexer", "search", "newServer",
"handler", "config", "graph", "rerank", "query", "savings",
}
return []toolCall{
{
Tool: "graph_stats",
ArgsFn: func(_ int) map[string]any { return map[string]any{} },
WarmupN: warmup, IterN: iter,
},
{
Tool: "search_symbols",
ArgsFn: func(i int) map[string]any {
return map[string]any{
"query": queries[i%len(queries)],
"limit": float64(20),
}
},
WarmupN: warmup, IterN: iter,
},
{
Tool: "get_symbol_source",
ArgsFn: func(_ int) map[string]any {
return map[string]any{"id": sampleFnID}
},
WarmupN: warmup, IterN: iter,
SkipIfMissing: func(g *graph.Graph) bool { return sampleFnID == "" },
},
{
Tool: "get_callers",
ArgsFn: func(_ int) map[string]any {
return map[string]any{"id": sampleFnID, "limit": float64(50)}
},
WarmupN: warmup, IterN: iter,
SkipIfMissing: func(g *graph.Graph) bool { return sampleFnID == "" },
},
{
Tool: "find_usages",
ArgsFn: func(_ int) map[string]any {
return map[string]any{"id": sampleFnID}
},
WarmupN: warmup, IterN: iter,
SkipIfMissing: func(g *graph.Graph) bool { return sampleFnID == "" },
},
{
Tool: "get_file_summary",
ArgsFn: func(_ int) map[string]any {
return map[string]any{"path": sampleFilePath}
},
WarmupN: warmup, IterN: iter,
SkipIfMissing: func(g *graph.Graph) bool { return sampleFilePath == "" },
},
{
Tool: "smart_context",
ArgsFn: func(i int) map[string]any {
return map[string]any{"task": "find " + queries[i%len(queries)]}
},
// smart_context is heavy — fewer iterations so the
// whole bench stays reasonable. Still produces a
// credible p50/p95 with 30-50 samples.
WarmupN: 3, IterN: iter / 5,
},
{
Tool: "get_repo_outline",
ArgsFn: func(_ int) map[string]any {
return map[string]any{}
},
WarmupN: warmup, IterN: iter,
},
}
}
func filterCalls(calls []toolCall, names []string) []toolCall {
want := map[string]bool{}
for _, n := range names {
want[strings.TrimSpace(n)] = true
}
out := make([]toolCall, 0, len(calls))
for _, c := range calls {
if want[c.Tool] {
out = append(out, c)
}
}
return out
}
// --- run loop -------------------------------------------------------
func runOne(handler *internalserver.Handler, c toolCall) result {
ctx := context.Background()
// Warm-up: prime any lazy initialisation in the handler /
// graph so the measured iterations are steady-state.
for i := range c.WarmupN {
_, _ = handler.CallToolStrict(ctx, c.Tool, c.ArgsFn(i))
}
latencies := make([]time.Duration, 0, c.IterN)
errors := 0
for i := range c.IterN {
t := time.Now()
_, err := handler.CallToolStrict(ctx, c.Tool, c.ArgsFn(i))
latencies = append(latencies, time.Since(t))
if err != nil {
errors++
}
}
r := result{
Tool: c.Tool,
Iters: c.IterN,
}
if len(latencies) > 0 {
r.P50Ms = pctMs(latencies, 50)
r.P95Ms = pctMs(latencies, 95)
r.P99Ms = pctMs(latencies, 99)
r.MaxMs = pctMs(latencies, 100)
r.MeanMs = meanMs(latencies)
r.ErrorRate = float64(errors) / float64(len(latencies))
}
return r
}
func pctMs(xs []time.Duration, pct int) float64 {
if len(xs) == 0 {
return 0
}
sorted := make([]time.Duration, len(xs))
copy(sorted, xs)
slices.Sort(sorted)
idx := (pct * len(sorted)) / 100
if idx >= len(sorted) {
idx = len(sorted) - 1
}
return float64(sorted[idx].Microseconds()) / 1000.0
}
func meanMs(xs []time.Duration) float64 {
if len(xs) == 0 {
return 0
}
var sum time.Duration
for _, x := range xs {
sum += x
}
avg := sum / time.Duration(len(xs))
return float64(avg.Microseconds()) / 1000.0
}
// --- rendering ------------------------------------------------------
func renderMarkdown(rows []result, repoRoot string, g *graph.Graph) string {
var b strings.Builder
fmt.Fprintln(&b, "# Daemon-mode MCP-tool latency")
fmt.Fprintln(&b)
fmt.Fprintf(&b, "_Corpus: `%s` (%d nodes). In-process handler dispatch — measures `Handler.CallToolStrict` end-to-end. Daemon socket overhead adds typically <1 ms on a warm pipe; the handler latency below dominates user-perceived response time._\n",
repoRoot, len(g.AllNodes()))
fmt.Fprintln(&b)
fmt.Fprintln(&b, "| tool | iters | p50 | p95 | p99 | mean | max | errors |")
fmt.Fprintln(&b, "|------|------:|----:|----:|----:|-----:|----:|-------:|")
for _, r := range rows {
if r.Skipped != "" {
fmt.Fprintf(&b, "| %s | — | — | — | — | — | — | skipped: %s |\n", r.Tool, r.Skipped)
continue
}
fmt.Fprintf(&b, "| %s | %d | %s | %s | %s | %s | %s | %.0f%% |\n",
r.Tool, r.Iters,
fmtMs(r.P50Ms), fmtMs(r.P95Ms), fmtMs(r.P99Ms),
fmtMs(r.MeanMs), fmtMs(r.MaxMs),
r.ErrorRate*100,
)
}
fmt.Fprintln(&b)
fmt.Fprintln(&b, summary(rows))
return b.String()
}
func renderCSV(rows []result) string {
var b strings.Builder
w := csv.NewWriter(&b)
_ = w.Write([]string{"tool", "iters", "p50_ms", "p95_ms", "p99_ms", "mean_ms", "max_ms", "error_rate", "skipped"})
for _, r := range rows {
_ = w.Write([]string{
r.Tool,
fmt.Sprintf("%d", r.Iters),
fmt.Sprintf("%.3f", r.P50Ms),
fmt.Sprintf("%.3f", r.P95Ms),
fmt.Sprintf("%.3f", r.P99Ms),
fmt.Sprintf("%.3f", r.MeanMs),
fmt.Sprintf("%.3f", r.MaxMs),
fmt.Sprintf("%.4f", r.ErrorRate),
r.Skipped,
})
}
w.Flush()
return b.String()
}
func mustMarshalJSON(rows []result) []byte {
b, err := json.MarshalIndent(rows, "", " ")
if err != nil {
die("marshal json: %v", err)
}
return append(b, '\n')
}
func summary(rows []result) string {
ran := 0
var p95s, p99s []float64
for _, r := range rows {
if r.Skipped != "" {
continue
}
ran++
p95s = append(p95s, r.P95Ms)
p99s = append(p99s, r.P99Ms)
}
if ran == 0 {
return "_no tools ran (all skipped)_"
}
sort.Float64s(p95s)
sort.Float64s(p99s)
medianP95 := p95s[len(p95s)/2]
medianP99 := p99s[len(p99s)/2]
return fmt.Sprintf("**Summary:** %d/%d tools ran. Median p95 across tools: %s. Median p99: %s.",
ran, len(rows), fmtMs(medianP95), fmtMs(medianP99))
}
func fmtMs(v float64) string {
switch {
case v == 0:
return "—"
case v < 1.0:
return fmt.Sprintf("%.2fms", v)
case v < 1000:
return fmt.Sprintf("%.1fms", v)
default:
return fmt.Sprintf("%.2fs", v/1000.0)
}
}
// --- helpers --------------------------------------------------------
func die(format string, args ...any) {
fmt.Fprintf(os.Stderr, "daemon-latency: "+format+"\n", args...)
os.Exit(1)
}
func writeOutput(path string, body []byte) error {
if path == "" {
_, err := os.Stdout.Write(body)
return err
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return err
}
return os.WriteFile(path, body, 0o644)
}
+200
View File
@@ -0,0 +1,200 @@
package main
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/zzet/gortex/internal/graph"
)
func TestPctMs_NearestRank(t *testing.T) {
// Build [1, 2, …, 100] ms.
xs := make([]time.Duration, 100)
for i := range xs {
xs[i] = time.Duration(i+1) * time.Millisecond
}
cases := map[int]float64{
50: 51.0, // idx = 50*100/100 = 50 → sorted[50] = 51ms
95: 96.0, // idx = 95 → 96ms
99: 100.0, // idx = 99 → 100ms
}
for p, want := range cases {
got := pctMs(xs, p)
if got != want {
t.Errorf("pctMs(%d) = %.2f, want %.2f", p, got, want)
}
}
}
func TestPctMs_EmptyReturnsZero(t *testing.T) {
if got := pctMs(nil, 50); got != 0 {
t.Errorf("pctMs(nil) = %.2f, want 0", got)
}
}
func TestPctMs_SingleSampleAllPctReturnIt(t *testing.T) {
xs := []time.Duration{5 * time.Millisecond}
for _, p := range []int{50, 95, 99} {
if got := pctMs(xs, p); got != 5.0 {
t.Errorf("pctMs(single, %d) = %.2f, want 5.0", p, got)
}
}
}
func TestMeanMs(t *testing.T) {
xs := []time.Duration{
1 * time.Millisecond,
2 * time.Millisecond,
3 * time.Millisecond,
4 * time.Millisecond,
}
if got := meanMs(xs); got != 2.5 {
t.Errorf("meanMs = %.2f, want 2.5", got)
}
if got := meanMs(nil); got != 0 {
t.Errorf("meanMs(nil) = %.2f, want 0", got)
}
}
func TestFmtMs_Buckets(t *testing.T) {
cases := map[float64]string{
0: "—",
0.25: "0.25ms",
3.7: "3.7ms",
1500: "1.50s",
60_000: "60.00s",
}
for in, want := range cases {
if got := fmtMs(in); got != want {
t.Errorf("fmtMs(%v) = %q, want %q", in, got, want)
}
}
}
func TestFilterCalls_KeepsRequestedSubset(t *testing.T) {
g := graph.New()
all := defaultCalls(g, 10)
got := filterCalls(all, []string{"graph_stats", "search_symbols"})
if len(got) != 2 {
t.Fatalf("got %d, want 2", len(got))
}
gotNames := map[string]bool{}
for _, c := range got {
gotNames[c.Tool] = true
}
if !gotNames["graph_stats"] || !gotNames["search_symbols"] {
t.Errorf("got %v, want graph_stats + search_symbols", gotNames)
}
}
func TestDefaultCalls_PopulatesSubstrate(t *testing.T) {
g := graph.New()
g.AddNode(&graph.Node{ID: "f.go::Sym", Name: "Sym", Kind: graph.KindFunction, FilePath: "f.go"})
g.AddNode(&graph.Node{ID: "file:f.go", Kind: graph.KindFile, FilePath: "f.go"})
calls := defaultCalls(g, 50)
if len(calls) == 0 {
t.Fatal("defaultCalls returned 0 entries")
}
// At least the headline tools must be present.
want := map[string]bool{
"graph_stats": true, "search_symbols": true,
"get_symbol_source": true, "smart_context": true,
}
got := map[string]bool{}
for _, c := range calls {
got[c.Tool] = true
}
for w := range want {
if !got[w] {
t.Errorf("default call set missing %q", w)
}
}
}
func TestDefaultCalls_SkipsTargetlessSubstrate(t *testing.T) {
g := graph.New()
// No function / method nodes → get_symbol_source / get_callers /
// find_usages should report SkipIfMissing=true.
calls := defaultCalls(g, 10)
for _, c := range calls {
if c.Tool == "get_symbol_source" || c.Tool == "get_callers" || c.Tool == "find_usages" {
if c.SkipIfMissing == nil || !c.SkipIfMissing(g) {
t.Errorf("%s should skip when no callable symbols available", c.Tool)
}
}
}
}
func TestRenderMarkdown_HasHeaderAndRows(t *testing.T) {
rows := []result{
{Tool: "graph_stats", Iters: 200, P50Ms: 1.2, P95Ms: 4.5, P99Ms: 8.9, MeanMs: 2.0, MaxMs: 12.5},
{Tool: "smart_context", Iters: 40, P50Ms: 50.0, P95Ms: 200.0, P99Ms: 300.0, MeanMs: 80.0, MaxMs: 350.0},
{Tool: "skipped_one", Skipped: "no substrate"},
}
g := graph.New()
md := renderMarkdown(rows, "/tmp/repo", g)
for _, want := range []string{
"# Daemon-mode MCP-tool latency",
"| graph_stats |",
"| smart_context |",
"| skipped_one |",
"skipped:",
"**Summary:**",
"2/3 tools",
} {
if !strings.Contains(md, want) {
t.Errorf("markdown missing %q\n----\n%s", want, md)
}
}
}
func TestRenderCSV_HasHeaderAndRows(t *testing.T) {
rows := []result{
{Tool: "graph_stats", Iters: 200, P50Ms: 1.2, P95Ms: 4.5, P99Ms: 8.9},
}
out := renderCSV(rows)
if !strings.HasPrefix(out, "tool,iters,p50_ms,p95_ms,p99_ms,") {
t.Errorf("CSV header missing or wrong:\n%s", out)
}
if !strings.Contains(out, "graph_stats,200,") {
t.Errorf("CSV body missing graph_stats row:\n%s", out)
}
}
func TestMustMarshalJSON_RoundTrip(t *testing.T) {
rows := []result{
{Tool: "graph_stats", Iters: 100, P95Ms: 5.5, P99Ms: 8.2},
}
raw := mustMarshalJSON(rows)
var got []result
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("round-trip: %v", err)
}
if len(got) != 1 || got[0].Tool != "graph_stats" || got[0].P95Ms != 5.5 {
t.Errorf("round-trip lost data: %+v", got)
}
}
func TestSummary_AllSkippedYieldsExplicitMessage(t *testing.T) {
rows := []result{{Tool: "x", Skipped: "y"}, {Tool: "z", Skipped: "w"}}
got := summary(rows)
if !strings.Contains(got, "no tools ran") {
t.Errorf("summary should explicitly say no tools ran, got %q", got)
}
}
func TestSummary_MedianP95AndP99(t *testing.T) {
rows := []result{
{Tool: "a", P95Ms: 1, P99Ms: 2},
{Tool: "b", P95Ms: 5, P99Ms: 10},
{Tool: "c", P95Ms: 10, P99Ms: 20},
}
got := summary(rows)
// Median p95 of [1, 5, 10] = 5
if !strings.Contains(got, "Median p95 across tools: 5.0ms") {
t.Errorf("summary missing median p95=5.0ms; got:\n%s", got)
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"name": "gortex-di-angular-fixture",
"private": true,
"description": "Minimal Angular fixture for Gortex DI recall evaluation. Not installed — static corpus.",
"dependencies": {
"@angular/core": "^17.0.0"
}
}
+110
View File
@@ -0,0 +1,110 @@
# Angular DI-recall fixture
#
# Modern Angular has two DI shapes:
# 1. Constructor injection: `constructor(private svc: Svc) {}` — same
# shape as NestJS. Type-aware resolution already handles this.
# 2. inject() function: `private svc = inject(Svc)` — field
# initializer calls `inject(Target)` to resolve the dependency.
# The call site is a function call to `inject`, not to Target.
# Without a special pass, the dependency is invisible — same
# shape as FastAPI's Depends(target) in that respect.
#
# Angular's @Injectable({ providedIn: 'root' }) is metadata only —
# doesn't affect graph shape, just marks the class as injectable. No
# extraction needed for the recall queries in this fixture.
name: gortex-angular-di
cases:
# ------------------------------------------------------------------
# Tier 1 — exact: named symbols
# ------------------------------------------------------------------
- id: exact-UsersService
tier: exact
query: UsersService
expected: [src/app/users/users.service.ts::UsersService]
- id: exact-AuthService
tier: exact
query: AuthService
expected: [src/app/auth/auth.service.ts::AuthService]
- id: exact-UsersListComponent
tier: exact
query: UsersListComponent
expected: [src/app/users/users.controller.ts::UsersListComponent]
- id: exact-findOne
tier: exact
query: findOne
expected: [src/app/users/users.service.ts::UsersService.findOne]
# ------------------------------------------------------------------
# Tier 2 — concept
# ------------------------------------------------------------------
- id: concept-list-all-users
tier: concept
query: list every user
expected: [src/app/users/users.service.ts::UsersService.findAll]
- id: concept-validate-credentials
tier: concept
query: validate user credentials
expected: [src/app/auth/auth.service.ts::AuthService.validate]
# ------------------------------------------------------------------
# Tier 3 — di: classic constructor injection with explicit types
# ------------------------------------------------------------------
# UsersListComponent has `constructor(private auth: AuthService)`.
# `this.auth.validate(...)` in validateAndList resolves via the
# parameter-property typing pass — Angular uses the same TS shape
# as NestJS here.
- id: di-call_chain-validateAndList
tier: di
query: "call_chain:src/app/users/users.controller.ts::UsersListComponent.validateAndList"
expected:
- src/app/auth/auth.service.ts::AuthService.validate
# ------------------------------------------------------------------
# Tier 4 — di_gap: inject() function-style injection
# ------------------------------------------------------------------
# AuthService uses `private users = inject(UsersService)`. The
# `users.findOne(userId)` call site can only resolve to UsersService.findOne
# if the graph knows `this.users` has type UsersService. Without
# inject()-aware extraction, the field's type is unknown and the
# call chain stops here.
- id: di_gap-call_chain-AuthService-validate
tier: di_gap
query: "call_chain:src/app/auth/auth.service.ts::AuthService.validate"
expected:
- src/app/users/users.service.ts::UsersService.findOne
# UsersListComponent.displayList calls `this.users.findAll()` where
# `users = inject(UsersService)` — same gap.
- id: di_gap-call_chain-displayList
tier: di_gap
query: "call_chain:src/app/users/users.controller.ts::UsersListComponent.displayList"
expected:
- src/app/users/users.service.ts::UsersService.findAll
# Direct callers query: who uses UsersService? With inject()
# extraction, the answer includes classes that inject() it, not
# just constructor-injected consumers.
- id: di_gap-callers-UsersService-findAll
tier: di_gap
query: "callers:src/app/users/users.service.ts::UsersService.findAll"
expected:
- src/app/users/users.controller.ts::UsersListComponent.displayList
- id: di_gap-callers-UsersService-findOne
tier: di_gap
query: "callers:src/app/users/users.service.ts::UsersService.findOne"
expected:
- src/app/auth/auth.service.ts::AuthService.validate
# Ambiguous method names: SessionService also has a findOne() method.
# SessionService.currentUser calls `this.users.findOne(...)` where
# `users = inject(UsersService)`. A name-only fallback would pick
# either SessionService.findOne or UsersService.findOne arbitrarily.
# This case tests whether inject()-aware type resolution routes the
# call correctly to UsersService.findOne.
- id: di_gap-call_chain-SessionService-currentUser
tier: di_gap
query: "call_chain:src/app/auth/session.service.ts::SessionService.currentUser"
expected:
- src/app/users/users.service.ts::UsersService.findOne
@@ -0,0 +1,18 @@
import { Injectable, inject } from '@angular/core';
import { UsersService } from '../users/users.service';
// Modern Angular: `inject()` function-style DI. No constructor required
// — the service is resolved via Angular's injector context at field-
// initialization time. This is the shape ordinary static analysis
// misses entirely, because the dependency edge lives inside an
// argument of a generic-looking function call (`inject(X)`) rather
// than in a typed constructor param.
@Injectable({ providedIn: 'root' })
export class AuthService {
private readonly users = inject(UsersService);
validate(userId: string, token: string): boolean {
const u = this.users.findOne(userId);
return u !== undefined && token === `tok_${u.id}`;
}
}
@@ -0,0 +1,25 @@
import { Injectable, inject } from '@angular/core';
import { UsersService } from '../users/users.service';
// Second service with a findOne() method — creates a name-collision
// with UsersService.findOne. A resolver that picks up the class field
// `this.users = inject(UsersService)` as UsersService-typed will
// correctly route `this.users.findOne(...)` to UsersService. A
// name-only fallback would arbitrarily pick one of the two findOne
// implementations.
@Injectable({ providedIn: 'root' })
export class SessionService {
private readonly users = inject(UsersService);
// Same method name as UsersService — deliberate collision so the
// call `this.users.findOne(...)` forces type-aware resolution.
findOne(sessionId: string): { userId: string } | undefined {
return { userId: sessionId };
}
currentUser(sessionId: string): string | undefined {
const s = this.findOne(sessionId);
if (!s) return undefined;
return this.users.findOne(s.userId)?.email;
}
}
@@ -0,0 +1,4 @@
export interface User {
id: string;
email: string;
}
@@ -0,0 +1,25 @@
import { Component, inject } from '@angular/core';
import { UsersService } from './users.service';
import { AuthService } from '../auth/auth.service';
// A plain-ish component-like class that mixes both DI styles:
// - `inject(UsersService)` — function form
// - constructor parameter-property — classic Angular (pre-14)
@Component({
selector: 'users-list',
template: '',
})
export class UsersListComponent {
private readonly users = inject(UsersService);
constructor(private readonly auth: AuthService) {}
displayList(): string[] {
return this.users.findAll().map((u) => u.email);
}
validateAndList(userId: string, token: string): string[] {
this.auth.validate(userId, token);
return this.displayList();
}
}
@@ -0,0 +1,26 @@
import { Injectable } from '@angular/core';
import { User } from './user.model';
// Modern Angular `providedIn: 'root'` — makes the service available
// app-wide without a NgModule providers array. The service is a plain
// TypeScript class with a few methods agents would want to trace from
// their call sites.
@Injectable({ providedIn: 'root' })
export class UsersService {
private readonly store = new Map<string, User>();
findOne(id: string): User | undefined {
return this.store.get(id);
}
findAll(): User[] {
return Array.from(this.store.values());
}
create(email: string): User {
const id = `usr_${this.store.size + 1}`;
const u: User = { id, email };
this.store.set(id, u);
return u;
}
}
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "esnext",
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"strict": true,
"baseUrl": "./src"
},
"include": ["src/**/*.ts"]
}
@@ -0,0 +1,20 @@
from typing import Annotated
from fastapi import Depends
from app.auth.service import AuthService
# Function-style dependency that wraps a class-based dependency —
# standard FastAPI pattern for "require this to be valid before the
# handler runs". The returned AuthService instance is what the
# handler ends up with when it declares `auth: AuthService = Depends(require_auth)`.
def require_auth(auth: AuthService = Depends(AuthService)) -> AuthService:
return auth
# PEP 593 / modern FastAPI shorthand — same resolution as the default-
# value form but typed via Annotated. Both shapes must resolve to
# AuthService.validate for any call-chain query from a handler using
# `auth: CurrentAuth` to reach the underlying implementation.
CurrentAuth = Annotated[AuthService, Depends(AuthService)]
@@ -0,0 +1,19 @@
from fastapi import Depends, HTTPException
from app.users.service import UserService
class AuthService:
"""Class-based dependency that ITSELF depends on another service.
FastAPI resolves the nested Depends chain automatically; statically
the dependency is visible via the __init__ parameter's Depends()
default value."""
def __init__(self, users: UserService = Depends(UserService)) -> None:
self._users = users
def validate(self, user_id: str, token: str) -> bool:
u = self._users.find_one(user_id)
if u is None or token != f"tok_{u.id}":
raise HTTPException(status_code=401)
return True
@@ -0,0 +1,14 @@
from dataclasses import dataclass
@dataclass
class Settings:
database_url: str = "postgres://localhost/test"
feature_flags: dict[str, bool] = None
def get_settings() -> Settings:
"""Factory-style dependency. FastAPI will call this once per request
(unless cached with @lru_cache) and inject the result into any
handler that declares `settings: Settings = Depends(get_settings)`."""
return Settings(feature_flags={"beta": True})
+7
View File
@@ -0,0 +1,7 @@
from fastapi import FastAPI
from app.users.router import router as users_router
app = FastAPI()
app.include_router(users_router)
@@ -0,0 +1,8 @@
from dataclasses import dataclass
@dataclass
class User:
id: str
email: str
name: str
@@ -0,0 +1,51 @@
from typing import Annotated
from fastapi import APIRouter, Depends
from app.auth.deps import CurrentAuth, require_auth
from app.auth.service import AuthService
from app.config.settings import Settings, get_settings
from app.users.models import User
from app.users.service import UserService
router = APIRouter(prefix="/users")
# Default-value form: `users: UserService = Depends(UserService)`.
# Most common shape in real FastAPI code.
@router.get("/")
def list_users(users: UserService = Depends(UserService)) -> list[User]:
return users.find_all()
# Nested / chained Depends: the handler depends on require_auth, which
# itself depends on AuthService, which depends on UserService. Whole
# chain is discoverable statically — each Depends() call exposes its
# target function/class as a default-value expression.
@router.get("/{user_id}")
def get_user(
user_id: str,
users: UserService = Depends(UserService),
auth: AuthService = Depends(require_auth),
) -> User:
u = users.find_one(user_id)
if u is None:
raise ValueError(f"no user {user_id}")
# Touch the auth-resolved service so the call chain from get_user
# reaches AuthService.validate and by extension UserService.find_one.
_ = auth.validate(user_id, f"tok_{u.id}")
return u
# Annotated-form Depends (PEP 593): `auth: Annotated[AuthService, Depends(...)]`.
# Semantically equivalent to the default-value form; a modern FastAPI
# codebase uses it everywhere.
@router.post("/")
def create_user(
email: str,
name: str,
users: Annotated[UserService, Depends(UserService)],
settings: Annotated[Settings, Depends(get_settings)],
) -> User:
_ = settings.database_url
return users.create(email, name)
@@ -0,0 +1,24 @@
from typing import Optional
from app.users.models import User
class UserService:
"""Plain service class — registered as a FastAPI dependency via
class-based `Depends(UserService)`. Its methods are called from
route handlers that receive the instance through a parameter."""
def __init__(self) -> None:
self._users: dict[str, User] = {}
def find_one(self, user_id: str) -> Optional[User]:
return self._users.get(user_id)
def find_all(self) -> list[User]:
return list(self._users.values())
def create(self, email: str, name: str) -> User:
uid = f"usr_{len(self._users) + 1}"
u = User(id=uid, email=email, name=name)
self._users[uid] = u
return u
+8
View File
@@ -0,0 +1,8 @@
[project]
name = "gortex-di-fastapi-fixture"
version = "0.0.0"
description = "Minimal FastAPI fixture for Gortex DI recall evaluation. Not installed — static corpus for the indexer to parse."
requires-python = ">=3.10"
dependencies = [
"fastapi>=0.100",
]
+124
View File
@@ -0,0 +1,124 @@
# FastAPI DI-recall fixture
#
# FastAPI's dependency injection is less "magic" than NestJS — every
# dependency is the default value of a parameter, either via
# `= Depends(target)` or `Annotated[T, Depends(target)]`. Target can be
# a callable (function) or a class. Static analysis CAN see the target
# because it's a value expression at the call site.
#
# Tiers (same as the nestjs fixture):
# exact — symbol-name text queries
# concept — natural-language paraphrases
# di — shapes that a type-aware Python resolver would already
# resolve (explicit class parameter types)
# di_gap — shapes that require Depends-aware extraction to traverse
# (nested Depends, function-wrapped Depends, Annotated)
#
# Expected IDs use the repo-stripped path form the Gortex TypeScript
# and Python extractors produce, e.g.
# app/users/router.py::get_user
# app/users/service.py::UserService.find_one
name: gortex-fastapi-di
cases:
# ------------------------------------------------------------------
# Tier 1 — exact: can the Python extractor find named symbols?
# ------------------------------------------------------------------
- id: exact-UserService
tier: exact
query: UserService
expected: [app/users/service.py::UserService]
- id: exact-AuthService
tier: exact
query: AuthService
expected: [app/auth/service.py::AuthService]
- id: exact-get_settings
tier: exact
query: get_settings
expected: [app/config/settings.py::get_settings]
- id: exact-require_auth
tier: exact
query: require_auth
expected: [app/auth/deps.py::require_auth]
- id: exact-find_one
tier: exact
query: find_one
expected: [app/users/service.py::UserService.find_one]
# ------------------------------------------------------------------
# Tier 2 — concept: paraphrase queries
# ------------------------------------------------------------------
- id: concept-look-up-user
tier: concept
query: look up a user by their id
expected: [app/users/service.py::UserService.find_one]
- id: concept-validate-token
tier: concept
query: validate an auth token
expected: [app/auth/service.py::AuthService.validate]
- id: concept-application-settings
tier: concept
query: application settings
expected: [app/config/settings.py::Settings]
# ------------------------------------------------------------------
# Tier 3 — di: class-based Depends with explicit types
# ------------------------------------------------------------------
# AuthService's constructor has `users: UserService = Depends(UserService)`.
# The declared type `UserService` is explicit Python, so a type-aware
# resolver can link `self._users.find_one(...)` to UserService.find_one.
- id: di-call_chain-AuthService-validate
tier: di
query: "call_chain:app/auth/service.py::AuthService.validate"
expected: [app/users/service.py::UserService.find_one]
# Handler with class Depends — `users: UserService = Depends(UserService)`.
# Same class-type annotation shape.
- id: di-call_chain-list_users
tier: di
query: "call_chain:app/users/router.py::list_users"
expected: [app/users/service.py::UserService.find_all]
# ------------------------------------------------------------------
# Tier 4 — di_gap: shapes requiring Depends-aware extraction
# ------------------------------------------------------------------
# Function-wrapped Depends: the handler gets AuthService via a
# require_auth dependency that itself returns whatever its own
# Depends yields. The call chain from get_user should reach both
# require_auth and the nested AuthService.validate.
- id: di_gap-call_chain-get_user
tier: di_gap
query: "call_chain:app/users/router.py::get_user"
expected:
# Body calls (explicit invocation on the injected instance).
- app/users/service.py::UserService.find_one
- app/auth/service.py::AuthService.validate
# Depends chain — the handler effectively calls each target at
# request time via FastAPI's resolver.
- app/auth/deps.py::require_auth
# require_auth → AuthService → UserService (two hops of Depends).
- id: di_gap-callers-AuthService-validate
tier: di_gap
query: "callers:app/auth/service.py::AuthService.validate"
expected: [app/users/router.py::get_user]
# Annotated[T, Depends(target)] — modern FastAPI shorthand. The
# handler declares `users: Annotated[UserService, Depends(UserService)]`
# and calls users.create(...). The Depends target is a class; its
# type is resolvable from the annotation; the method call should
# link to UserService.create.
- id: di_gap-call_chain-create_user
tier: di_gap
query: "call_chain:app/users/router.py::create_user"
expected:
- app/users/service.py::UserService.create
- app/config/settings.py::get_settings
# Factory-function Depends: `settings: Annotated[Settings, Depends(get_settings)]`.
# get_settings returns Settings; the handler then touches settings.database_url.
# With Depends-aware extraction, callers of get_settings should include the handler.
- id: di_gap-callers-get_settings
tier: di_gap
query: "callers:app/config/settings.py::get_settings"
expected: [app/users/router.py::create_user]
@@ -0,0 +1,42 @@
<?php
namespace App\Http\Controllers;
use App\Http\Middleware\AdminMiddleware;
use App\Http\Middleware\AuthenticateMiddleware;
use App\Repositories\UserRepository;
use App\Services\Clock;
class UserController
{
// Constructor-injected typed dependencies. Laravel's container
// autowires from the type hints; UserRepository resolves through
// the binding in AppServiceProvider::register() to
// EloquentUserRepository.
public function __construct(
private UserRepository $users,
private Clock $clock,
) {
// Controller middleware: registered imperatively in the
// constructor. 'auth' applies to every action; 'admin' is
// filtered with ->only(['destroy']), same shape as Rails
// before_action + only:.
$this->middleware(AuthenticateMiddleware::class);
$this->middleware(AdminMiddleware::class)->only(['destroy']);
}
public function index(): array
{
return $this->users->all();
}
public function show(string $id): ?array
{
return $this->users->find($id);
}
public function destroy(string $id): string
{
return 'deleted at ' . $this->clock->now();
}
}
@@ -0,0 +1,16 @@
<?php
namespace App\Http\Middleware;
use Closure;
class AdminMiddleware
{
public function handle($request, Closure $next)
{
if (empty($request->user['admin'])) {
return response('forbidden', 403);
}
return $next($request);
}
}
@@ -0,0 +1,21 @@
<?php
namespace App\Http\Middleware;
use Closure;
// Laravel middleware. handle() is what the framework calls before the
// controller action runs. There's no explicit call site in the
// controller or route definition — the middleware is referenced by
// its short alias ("auth") or class name in a call like
// $this->middleware('auth').
class AuthenticateMiddleware
{
public function handle($request, Closure $next)
{
if (empty($request->user)) {
return response('unauthorized', 401);
}
return $next($request);
}
}
@@ -0,0 +1,28 @@
<?php
namespace App\Providers;
use App\Repositories\EloquentUserRepository;
use App\Repositories\UserRepository;
use App\Services\Clock;
use Illuminate\Support\ServiceProvider;
class AppServiceProvider extends ServiceProvider
{
// register() is Laravel's container-binding entry point. Every
// bind/singleton/instance call here declares that when someone
// asks the container for the first argument, give them an
// instance produced from the second. These bindings are the DI
// gap that matters — consumers typed against the interface won't
// reach the implementation without this info.
public function register(): void
{
// bind: a fresh EloquentUserRepository per resolve.
$this->app->bind(UserRepository::class, EloquentUserRepository::class);
// singleton: one Clock shared across the app lifetime.
$this->app->singleton(Clock::class, function ($app) {
return new Clock('UTC');
});
}
}
@@ -0,0 +1,20 @@
<?php
namespace App\Repositories;
// Concrete UserRepository. Service provider binds this to the
// interface — without that binding, the Laravel container has no way
// to pick an implementation from the interface alone. Same shape as
// NestJS useClass / Spring @Bean return-type binding.
class EloquentUserRepository implements UserRepository
{
public function find(string $id): ?array
{
return ['id' => $id];
}
public function all(): array
{
return [];
}
}
@@ -0,0 +1,9 @@
<?php
namespace App\Repositories;
interface UserRepository
{
public function find(string $id): ?array;
public function all(): array;
}
@@ -0,0 +1,13 @@
<?php
namespace App\Services;
class Clock
{
public function __construct(private string $timezone = 'UTC') {}
public function now(): string
{
return 'now-in-' . $this->timezone;
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"name": "gortex/di-laravel-fixture",
"type": "project",
"description": "Minimal Laravel fixture for Gortex DI recall — not installed.",
"require": {
"laravel/framework": "^11.0"
}
}
+79
View File
@@ -0,0 +1,79 @@
# Laravel DI-recall fixture
#
# Two DI gaps matter in real-world Laravel code:
# 1. Controller middleware (`$this->middleware('auth')` in the ctor,
# optionally filtered with ->only([...]) / ->except([...])).
# Same decorator-dispatch shape as NestJS @UseGuards / Rails
# before_action / Phoenix plug. Framework invokes middleware's
# handle() before the action; no explicit call site.
# 2. Service provider bindings (`$this->app->bind(Interface::class,
# Impl::class)` and ->singleton(...)). Container-registered
# interface-to-implementation or token-to-factory mappings —
# same shape as NestJS useClass / Spring @Bean.
name: gortex-laravel-di
cases:
# exact
- id: exact-UserController
tier: exact
query: UserController
expected: [app/Http/Controllers/UserController.php::UserController]
- id: exact-UserRepository
tier: exact
query: UserRepository
expected: [app/Repositories/UserRepository.php::UserRepository]
- id: exact-EloquentUserRepository
tier: exact
query: EloquentUserRepository
expected: [app/Repositories/EloquentUserRepository.php::EloquentUserRepository]
- id: exact-AuthenticateMiddleware
tier: exact
query: AuthenticateMiddleware
expected: [app/Http/Middleware/AuthenticateMiddleware.php::AuthenticateMiddleware]
# di — typed constructor injection. Should work via PHP type system
# if the extractor captures constructor param types.
- id: di-call_chain-index
tier: di
query: "call_chain:app/Http/Controllers/UserController.php::UserController.index"
expected:
- app/Repositories/UserRepository.php::UserRepository.all
# di_gap: middleware dispatch
- id: di_gap-callers-AuthenticateMiddleware-handle
tier: di_gap
query: "callers:app/Http/Middleware/AuthenticateMiddleware.php::AuthenticateMiddleware.handle"
expected:
- app/Http/Controllers/UserController.php::UserController.index
- app/Http/Controllers/UserController.php::UserController.show
- app/Http/Controllers/UserController.php::UserController.destroy
# Admin middleware is only: ['destroy'] — just one action binds it.
- id: di_gap-callers-AdminMiddleware-handle
tier: di_gap
query: "callers:app/Http/Middleware/AdminMiddleware.php::AdminMiddleware.handle"
expected:
- app/Http/Controllers/UserController.php::UserController.destroy
- id: di_gap-call_chain-UserController-show
tier: di_gap
query: "call_chain:app/Http/Controllers/UserController.php::UserController.show"
expected:
- app/Http/Middleware/AuthenticateMiddleware.php::AuthenticateMiddleware.handle
# di_gap: service provider bind(Interface, Impl)
# $this->app->bind(UserRepository::class, EloquentUserRepository::class)
# binds the interface to the impl. usages:UserRepository should
# include AppServiceProvider as the binding site.
- id: di_gap-usages-UserRepository
tier: di_gap
query: "usages:app/Repositories/UserRepository.php::UserRepository"
expected:
- app/Providers/AppServiceProvider.php::AppServiceProvider
# singleton binding — Clock is bound by a factory closure.
- id: di_gap-usages-Clock
tier: di_gap
query: "usages:app/Services/Clock.php::Clock"
expected:
- app/Providers/AppServiceProvider.php::AppServiceProvider
+8
View File
@@ -0,0 +1,8 @@
<?php
use App\Http\Controllers\UserController;
use Illuminate\Support\Facades\Route;
Route::get('/users', [UserController::class, 'index']);
Route::get('/users/{id}', [UserController::class, 'show']);
Route::delete('/users/{id}', [UserController::class, 'destroy']);
+9
View File
@@ -0,0 +1,9 @@
{
"name": "gortex-di-nestjs-fixture",
"private": true,
"description": "Minimal NestJS fixture for Gortex DI recall evaluation. Not installed — purely a static corpus for the indexer to parse.",
"dependencies": {
"@nestjs/common": "^10.0.0",
"@nestjs/core": "^10.0.0"
}
}
+256
View File
@@ -0,0 +1,256 @@
# NestJS DI-recall fixture
#
# Measures what Gortex's retrieval stack can find on a minimal NestJS
# app *without* DI-aware extraction — i.e. what's reachable from plain
# BM25 + engine fallback + raw call graph. Option 1 (extend `contracts`
# to DI) should move the "di" tier numbers up without regressing the
# other two.
#
# Tiers
# -----
# exact — symbol-name / signature queries. Text rankers should ace
# these; failure means the TypeScript extractor missed something.
# concept — natural-language paraphrases. Baseline expectation similar
# to the Go concept tier.
# di — relational queries that only return the right answer when a
# DI edge exists in the graph. Queries use the GraphRanker's
# "callers:<id>" / "call_chain:<id>" format. Expected IDs are
# the controllers / services that inject the subject via a
# constructor parameter. Today these return 0% because no
# such edge is emitted.
#
# Expected ID format: `<relative-ts-path>::<ClassName>.<method>` —
# matches what the in-tree TypeScript extractor produces.
name: gortex-nestjs-di
cases:
# ------------------------------------------------------------------
# Tier 1 — exact: can the TS extractor find each symbol by name?
# ------------------------------------------------------------------
- id: exact-UsersService
tier: exact
query: UsersService
expected: [src/users/users.service.ts::UsersService]
- id: exact-UsersController
tier: exact
query: UsersController
expected: [src/users/users.controller.ts::UsersController]
- id: exact-AuthService
tier: exact
query: AuthService
expected: [src/auth/auth.service.ts::AuthService]
- id: exact-AuthGuard
tier: exact
query: AuthGuard
expected: [src/auth/auth.guard.ts::AuthGuard]
- id: exact-findOne
tier: exact
query: findOne
expected: [src/users/users.service.ts::UsersService.findOne]
# ------------------------------------------------------------------
# Tier 2 — concept: paraphrased / intent-based text queries
# ------------------------------------------------------------------
- id: concept-look-up-user-by-id
tier: concept
query: look up a user by their id
expected: [src/users/users.service.ts::UsersService.findOne]
- id: concept-http-guard
tier: concept
query: guard that validates an incoming request
expected: [src/auth/auth.guard.ts::AuthGuard.canActivate]
- id: concept-issue-auth-token
tier: concept
query: issue an authentication token
expected: [src/auth/auth.service.ts::AuthService.issueToken]
- id: concept-create-new-user
tier: concept
query: create a new user record
expected: [src/users/users.service.ts::UsersService.create]
- id: concept-list-all-users
tier: concept
query: list every user
expected: [src/users/users.service.ts::UsersService.findAll]
# ------------------------------------------------------------------
# Tier 3 — di: requires a DI edge to answer correctly
# ------------------------------------------------------------------
# Who injects UsersService? Answer: UsersController, AuthService.
# Without DI edges, get_callers on the service's methods sees nothing
# past the class's own internal references.
- id: di-callers-UsersService-findOne
tier: di
query: "callers:src/users/users.service.ts::UsersService.findOne"
expected:
- src/users/users.controller.ts::UsersController.getUser
- src/auth/auth.service.ts::AuthService.validate
- src/auth/auth.service.ts::AuthService.issueToken
- id: di-callers-UsersService-findAll
tier: di
query: "callers:src/users/users.service.ts::UsersService.findAll"
expected:
- src/users/users.controller.ts::UsersController.list
- id: di-callers-UsersService-create
tier: di
query: "callers:src/users/users.service.ts::UsersService.create"
expected:
- src/users/users.controller.ts::UsersController.create
- id: di-callers-AuthService-validate
tier: di
query: "callers:src/auth/auth.service.ts::AuthService.validate"
expected:
- src/auth/auth.guard.ts::AuthGuard.canActivate
# Call chain FROM a controller method should reach the service method
# it injects. Tests forward traversal through DI boundaries.
- id: di-call_chain-UsersController-getUser
tier: di
query: "call_chain:src/users/users.controller.ts::UsersController.getUser"
expected:
- src/users/users.service.ts::UsersService.findOne
- id: di-call_chain-AuthGuard-canActivate
tier: di
query: "call_chain:src/auth/auth.guard.ts::AuthGuard.canActivate"
expected:
- src/auth/auth.service.ts::AuthService.validate
- src/users/users.service.ts::UsersService.findOne
# ------------------------------------------------------------------
# Tier 4 — di_gap: shapes Tier-2 type resolution cannot handle.
# These queries should be near-0% without module-aware extraction;
# they're the cases that would justify shipping option 1.
# ------------------------------------------------------------------
# Abstract-class injection: NotificationsController injects Notifier
# (abstract), module binds it to EmailNotifier. Finding the caller of
# EmailNotifier.notify requires knowing that binding.
- id: di_gap-callers-EmailNotifier-notify
tier: di_gap
query: "callers:src/notifications/email-notifier.service.ts::EmailNotifier.notify"
expected:
- src/notifications/notifications.controller.ts::NotificationsController.send
# Call-chain forward from the controller into the concrete method
# behind the abstract binding. Same gap, traversing the other way.
- id: di_gap-call_chain-NotificationsController-send
tier: di_gap
query: "call_chain:src/notifications/notifications.controller.ts::NotificationsController.send"
expected:
# Both are legitimate downstream calls from the handler: the guard
# fires before the body via @UseGuards dispatch, the body's
# `this.notifier.notify()` resolves through the useClass binding.
- src/notifications/email-notifier.service.ts::EmailNotifier.notify
- src/auth/auth.guard.ts::AuthGuard.canActivate
# @Inject(DATABASE_URL) token consumer. There's no type linking
# ConfigService's dbUrl param to the `{ provide: DATABASE_URL,
# useValue: ... }` entry — only the provider registry knows.
- id: di_gap-usages-DATABASE_URL
tier: di_gap
query: "usages:src/config/config.tokens.ts::DATABASE_URL"
expected:
- src/config/config.service.ts::ConfigService
- src/config/config.module.ts::ConfigModule
- id: di_gap-usages-FEATURE_FLAGS
tier: di_gap
query: "usages:src/config/config.tokens.ts::FEATURE_FLAGS"
expected:
- src/config/config.service.ts::ConfigService
- src/config/config.module.ts::ConfigModule
# --- @UseGuards decorator-dispatch ---
# NotificationsController.send is decorated with @UseGuards(AuthGuard).
# The framework invokes AuthGuard.canActivate before the handler runs,
# but there is no explicit call site — the binding lives in the
# decorator metadata. get_callers on canActivate should include the
# guarded handler.
- id: di_gap-callers-AuthGuard-canActivate
tier: di_gap
query: "callers:src/auth/auth.guard.ts::AuthGuard.canActivate"
expected:
- src/notifications/notifications.controller.ts::NotificationsController.send
# --- Factory provider with inject: [Dep] ---
# BillingService injects DB_CONNECTION, which BillingModule binds to a
# DatabaseConnection produced by a factory that takes ConfigService.
# Two signals to check:
# 1. Token-level: usages of DB_CONNECTION.
# 2. Call chain: BillingService.charge should reach
# DatabaseConnection.query via the factory's produced binding.
- id: di_gap-usages-DB_CONNECTION
tier: di_gap
query: "usages:src/billing/billing.tokens.ts::DB_CONNECTION"
expected:
- src/billing/billing.service.ts::BillingService
- src/billing/billing.module.ts::BillingModule
- id: di_gap-call_chain-BillingService-charge
tier: di_gap
query: "call_chain:src/billing/billing.service.ts::BillingService.charge"
expected:
- src/billing/database.connection.ts::DatabaseConnection.query
# --- forwardRef circular injection ---
# FeatureAService and FeatureBService inject each other via
# `forwardRef(() => …)`. Static type resolution can't evaluate the
# arrow, so without DI extraction the receiver types of `this.b` and
# `this.a` are unknown and call chains across the cycle break.
- id: di_gap-call_chain-FeatureAService-doA
tier: di_gap
query: "call_chain:src/feature/feature-a.service.ts::FeatureAService.doA"
expected:
- src/feature/feature-b.service.ts::FeatureBService.doB
- id: di_gap-call_chain-FeatureBService-callA
tier: di_gap
query: "call_chain:src/feature/feature-b.service.ts::FeatureBService.callA"
expected:
- src/feature/feature-a.service.ts::FeatureAService.doA
# --- Property (field) injection ---
# NestJS permits @Inject on class fields instead of constructor params.
# AuditService uses both the implicit (`@Inject() field!: T`) and
# explicit-token (`@Inject(TOKEN) field: ...`) shapes. Without field-
# level decorator extraction the call chain stops at AuditService —
# the call to `this.users.findOne` has no receiver_type on its edge,
# and find_usages on DATABASE_URL misses AuditService entirely.
- id: di_gap-call_chain-AuditService-recordLogin
tier: di_gap
query: "call_chain:src/audit/audit.service.ts::AuditService.recordLogin"
expected:
- src/users/users.service.ts::UsersService.findOne
- id: di_gap-usages-DATABASE_URL-includes-audit
tier: di_gap
query: "usages:src/config/config.tokens.ts::DATABASE_URL"
expected:
- src/audit/audit.service.ts::AuditService
# --- Dynamic modules (forRoot / forFeature) ---
# CacheModule.forRoot(ttl) returns a DynamicModule whose providers
# array binds CACHE_TTL_SECONDS to the runtime ttl value. The binding
# lives inside a static method body, not a @Module decorator — so
# without dynamic-module extraction find_usages on the token surfaces
# only the consumer, never the provider module.
- id: di_gap-usages-CACHE_TTL_SECONDS
tier: di_gap
query: "usages:src/cache/cache.tokens.ts::CACHE_TTL_SECONDS"
expected:
- src/cache/cache.service.ts::CacheService
- src/cache/cache.module.ts::CacheModule
@@ -0,0 +1,23 @@
import { Module } from '@nestjs/common';
import { UsersModule } from './users/users.module';
import { AuthModule } from './auth/auth.module';
import { NotificationsModule } from './notifications/notifications.module';
import { ConfigModule } from './config/config.module';
import { BillingModule } from './billing/billing.module';
import { FeatureModule } from './feature/feature.module';
import { AuditModule } from './audit/audit.module';
import { CacheModule } from './cache/cache.module';
@Module({
imports: [
UsersModule,
AuthModule,
NotificationsModule,
ConfigModule,
BillingModule,
FeatureModule,
AuditModule,
CacheModule.forRoot(60),
],
})
export class AppModule {}
@@ -0,0 +1,11 @@
import { Module } from '@nestjs/common';
import { UsersModule } from '../users/users.module';
import { ConfigModule } from '../config/config.module';
import { AuditService } from './audit.service';
@Module({
imports: [UsersModule, ConfigModule],
providers: [AuditService],
exports: [AuditService],
})
export class AuditModule {}
@@ -0,0 +1,22 @@
import { Inject, Injectable } from '@nestjs/common';
import { UsersService } from '../users/users.service';
import { DATABASE_URL } from '../config/config.tokens';
// Property injection: NestJS supports @Inject on class fields for cases
// where the class can't declare a constructor (or the author prefers it
// over parameter properties). Two shapes exercised:
// - `@Inject() field!: T` — implicit token, class type drives binding
// - `@Inject(TOKEN) field: ...` — explicit token, same as constructor form
@Injectable()
export class AuditService {
@Inject()
private readonly users!: UsersService;
@Inject(DATABASE_URL)
private readonly dbUrl!: string;
async recordLogin(userId: string): Promise<string> {
const user = await this.users.findOne(userId);
return `audit(${this.dbUrl}): login by ${user.id}`;
}
}
@@ -0,0 +1,22 @@
import {
CanActivate,
ExecutionContext,
Injectable,
UnauthorizedException,
} from '@nestjs/common';
import { AuthService } from './auth.service';
@Injectable()
export class AuthGuard implements CanActivate {
constructor(private readonly authService: AuthService) {}
async canActivate(context: ExecutionContext): Promise<boolean> {
const req = context.switchToHttp().getRequest();
const userId = req.headers['x-user-id'];
const token = req.headers['x-token'];
if (!userId || !token) {
throw new UnauthorizedException('missing credentials');
}
return this.authService.validate(userId, token);
}
}
@@ -0,0 +1,11 @@
import { Module } from '@nestjs/common';
import { UsersModule } from '../users/users.module';
import { AuthService } from './auth.service';
import { AuthGuard } from './auth.guard';
@Module({
imports: [UsersModule],
providers: [AuthService, AuthGuard],
exports: [AuthService, AuthGuard],
})
export class AuthModule {}
@@ -0,0 +1,20 @@
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { UsersService } from '../users/users.service';
@Injectable()
export class AuthService {
constructor(private readonly usersService: UsersService) {}
async validate(userId: string, token: string): Promise<boolean> {
const user = await this.usersService.findOne(userId);
if (!user || token !== `tok_${user.id}`) {
throw new UnauthorizedException('invalid token');
}
return true;
}
async issueToken(userId: string): Promise<string> {
const user = await this.usersService.findOne(userId);
return `tok_${user.id}`;
}
}
@@ -0,0 +1,29 @@
import { Module } from '@nestjs/common';
import { ConfigModule } from '../config/config.module';
import { ConfigService } from '../config/config.service';
import { DB_CONNECTION } from './billing.tokens';
import { DatabaseConnection } from './database.connection';
import { BillingService } from './billing.service';
// Factory provider: the binding DB_CONNECTION → DatabaseConnection is
// produced by calling `dbFactory(cfg)` at module bootstrap. `inject:
// [ConfigService]` is how Nest passes the factory its dependencies.
// There is no `new DatabaseConnection(...)` call anywhere else in the
// codebase; any graph analysis that wants to traverse "consumers of
// DB_CONNECTION → DatabaseConnection methods" must read this factory.
const dbFactory = (cfg: ConfigService) =>
new DatabaseConnection(cfg.getDatabaseUrl());
@Module({
imports: [ConfigModule],
providers: [
{
provide: DB_CONNECTION,
useFactory: dbFactory,
inject: [ConfigService],
},
BillingService,
],
exports: [BillingService],
})
export class BillingModule {}
@@ -0,0 +1,27 @@
import { Inject, Injectable } from '@nestjs/common';
import { DatabaseConnection } from './database.connection';
import { DB_CONNECTION } from './billing.tokens';
// Consumer of the factory-provided DatabaseConnection. The `db` param's
// declared type IS DatabaseConnection — so with parameter-property typing
// the receiver type resolution should work for `this.db.query(...)`. But
// the *binding* from the token DB_CONNECTION to DatabaseConnection lives
// only inside the factory — a graph walk from any consumer to "who
// produces DB_CONNECTION" has no edge to follow without DI extraction.
@Injectable()
export class BillingService {
constructor(
@Inject(DB_CONNECTION) private readonly db: DatabaseConnection,
) {}
async charge(userId: string, cents: number): Promise<void> {
await this.db.query(`INSERT INTO charges (user_id, amount) VALUES ('${userId}', ${cents})`);
}
async totals(userId: string): Promise<number> {
const rows = await this.db.query<{ total: number }>(
`SELECT SUM(amount) AS total FROM charges WHERE user_id = '${userId}'`,
);
return rows[0]?.total ?? 0;
}
}
@@ -0,0 +1,4 @@
// Separate file for the injection token so the module import graph
// reflects real NestJS practice — consumers import the token without
// pulling in the module and its factory provider.
export const DB_CONNECTION = 'DB_CONNECTION';
@@ -0,0 +1,17 @@
// Concrete class wired up through a factory provider. The only place
// the binding `"DB_CONNECTION" → DatabaseConnection` appears is inside
// a `useFactory` in billing.module.ts — no `new DatabaseConnection(...)`
// call survives anywhere else, so Tier-2 type resolution can't reach
// this class from consumers that inject `@Inject('DB_CONNECTION')`.
export class DatabaseConnection {
constructor(public readonly url: string) {}
async query<T>(sql: string): Promise<T[]> {
console.log(`[${this.url}] ${sql}`);
return [];
}
async close(): Promise<void> {
// Close the pool in real code.
}
}
+22
View File
@@ -0,0 +1,22 @@
import { DynamicModule, Module } from '@nestjs/common';
import { CACHE_TTL_SECONDS } from './cache.tokens';
import { CacheService } from './cache.service';
// Dynamic module: NestJS's forRoot / forFeature pattern returns a
// runtime-computed module config. The providers array here is
// structurally identical to a @Module providers array — same
// { provide: X, useValue: ... } shape — but lives inside a static
// method body instead of a decorator.
@Module({})
export class CacheModule {
static forRoot(ttlSeconds: number): DynamicModule {
return {
module: CacheModule,
providers: [
{ provide: CACHE_TTL_SECONDS, useValue: ttlSeconds },
CacheService,
],
exports: [CacheService],
};
}
}
+15
View File
@@ -0,0 +1,15 @@
import { Inject, Injectable } from '@nestjs/common';
import { CACHE_TTL_SECONDS } from './cache.tokens';
// Consumer of a token provided only through CacheModule.forRoot().
// Without dynamic-module extraction the graph sees the @Inject here
// but no provider, so find_usages(CACHE_TTL_SECONDS) returns only
// this consumer and leaves orphan-detection incomplete.
@Injectable()
export class CacheService {
constructor(@Inject(CACHE_TTL_SECONDS) private readonly ttl: number) {}
ttlSeconds(): number {
return this.ttl;
}
}
+1
View File
@@ -0,0 +1 @@
export const CACHE_TTL_SECONDS = 'CACHE_TTL_SECONDS';
@@ -0,0 +1,19 @@
import { Injectable } from '@nestjs/common';
import { ConfigService } from './config.service';
// Second-hop consumer: ConfigConsumer calls ConfigService, which itself
// received its values via @Inject(TOKEN). The test is whether the graph
// can reach *from* ConfigConsumer *to* the methods on ConfigService —
// standard typed injection, should work — and whether the TOKEN-keyed
// providers show up in any cross-reference (they won't, without DI
// extraction).
@Injectable()
export class ConfigConsumer {
constructor(private readonly config: ConfigService) {}
describe(): string {
return `db=${this.config.getDatabaseUrl()} flags=${JSON.stringify(
this.config.isEnabled('beta'),
)}`;
}
}
@@ -0,0 +1,15 @@
import { Module } from '@nestjs/common';
import { DATABASE_URL, FEATURE_FLAGS } from './config.tokens';
import { ConfigService } from './config.service';
import { ConfigConsumer } from './config.consumer';
@Module({
providers: [
{ provide: DATABASE_URL, useValue: 'postgres://localhost/test' },
{ provide: FEATURE_FLAGS, useValue: { beta: true } },
ConfigService,
ConfigConsumer,
],
exports: [ConfigService, ConfigConsumer],
})
export class ConfigModule {}
@@ -0,0 +1,21 @@
import { Inject, Injectable } from '@nestjs/common';
import { DATABASE_URL, FEATURE_FLAGS } from './config.tokens';
@Injectable()
export class ConfigService {
// Both constructor params receive their values via string-keyed
// @Inject — no type info survives into the graph for the binder to
// traverse.
constructor(
@Inject(DATABASE_URL) private readonly dbUrl: string,
@Inject(FEATURE_FLAGS) private readonly flags: Record<string, boolean>,
) {}
getDatabaseUrl(): string {
return this.dbUrl;
}
isEnabled(flag: string): boolean {
return Boolean(this.flags[flag]);
}
}
@@ -0,0 +1,8 @@
// String-keyed injection tokens. These are the NestJS idiom for
// providing values that have no class identity — environment URLs,
// configuration primitives, feature flags. A consumer asks for them
// via `@Inject(DATABASE_URL)`; the Module provides them via
// `{ provide: DATABASE_URL, useValue: '...' }`. No type-aware
// resolution can cross this boundary.
export const DATABASE_URL = 'DATABASE_URL';
export const FEATURE_FLAGS = 'FEATURE_FLAGS';
@@ -0,0 +1,20 @@
import { forwardRef, Inject, Injectable } from '@nestjs/common';
import { FeatureBService } from './feature-b.service';
// Two services that reference each other. NestJS requires forwardRef()
// to break the import cycle; the lazy arrow function `() => FeatureBService`
// is evaluated after module init. Static extraction can't call the arrow
// at parse time, so the `b` parameter's declared type is effectively
// unknown (or `any`) at extraction. Type-aware resolution can't link
// `this.b.doB()` to FeatureBService.doB without following the forwardRef.
@Injectable()
export class FeatureAService {
constructor(
@Inject(forwardRef(() => FeatureBService))
private readonly b: FeatureBService,
) {}
doA(): string {
return this.b.doB() + ' from A';
}
}
@@ -0,0 +1,18 @@
import { forwardRef, Inject, Injectable } from '@nestjs/common';
import { FeatureAService } from './feature-a.service';
@Injectable()
export class FeatureBService {
constructor(
@Inject(forwardRef(() => FeatureAService))
private readonly a: FeatureAService,
) {}
doB(): string {
return 'B';
}
callA(): string {
return this.a.doA();
}
}
@@ -0,0 +1,9 @@
import { Module } from '@nestjs/common';
import { FeatureAService } from './feature-a.service';
import { FeatureBService } from './feature-b.service';
@Module({
providers: [FeatureAService, FeatureBService],
exports: [FeatureAService, FeatureBService],
})
export class FeatureModule {}
@@ -0,0 +1,11 @@
import { Injectable } from '@nestjs/common';
import { Notifier } from './notifier.interface';
@Injectable()
export class EmailNotifier extends Notifier {
async notify(userId: string, message: string): Promise<void> {
// In real code this would call an SMTP client; here the logic is
// deliberately trivial — the fixture only needs the method to exist.
console.log(`email to ${userId}: ${message}`);
}
}
@@ -0,0 +1,22 @@
import { Body, Controller, Post, UseGuards } from '@nestjs/common';
import { Notifier } from './notifier.interface';
import { AuthGuard } from '../auth/auth.guard';
@Controller('notifications')
export class NotificationsController {
// Injected by the abstract base class — the runtime instance will
// be EmailNotifier (see notifications.module.ts), but the static
// type here is `Notifier` so a type-aware resolver has no way to
// pick the right concrete method.
constructor(private readonly notifier: Notifier) {}
// @UseGuards binds the guard's canActivate() to this route. There is
// no explicit call site to AuthGuard.canActivate anywhere in source —
// the framework invokes it at request time. Without decorator-dispatch
// extraction, get_callers on canActivate is empty for this endpoint.
@Post('send')
@UseGuards(AuthGuard)
async send(@Body('userId') userId: string, @Body('message') message: string): Promise<void> {
await this.notifier.notify(userId, message);
}
}
@@ -0,0 +1,20 @@
import { Module } from '@nestjs/common';
import { Notifier } from './notifier.interface';
import { EmailNotifier } from './email-notifier.service';
import { SmsNotifier } from './sms-notifier.service';
import { NotificationsController } from './notifications.controller';
import { AuthModule } from '../auth/auth.module';
// The `provide: Notifier, useClass: EmailNotifier` binding is the only
// place a static reader can learn that NotificationsController's
// `notifier` parameter resolves to EmailNotifier specifically.
@Module({
imports: [AuthModule],
controllers: [NotificationsController],
providers: [
{ provide: Notifier, useClass: EmailNotifier },
SmsNotifier,
],
exports: [Notifier, SmsNotifier],
})
export class NotificationsModule {}
@@ -0,0 +1,8 @@
// Abstract base class used as the injection token — NestJS supports
// `constructor(private n: Notifier)` where Notifier is an abstract class
// and the Module provides `{ provide: Notifier, useClass: EmailNotifier }`.
// Type-aware resolution can't cross this indirection without knowing the
// provider binding; exactly the gap option 1's DI extractor targets.
export abstract class Notifier {
abstract notify(userId: string, message: string): Promise<void>;
}
@@ -0,0 +1,13 @@
import { Injectable } from '@nestjs/common';
import { Notifier } from './notifier.interface';
// Second implementation of the same abstract class. Makes the
// abstract-injection case ambiguous — module-level binding is what
// actually picks between EmailNotifier and SmsNotifier. This is the
// shape Tier-2 type-aware resolution cannot handle.
@Injectable()
export class SmsNotifier extends Notifier {
async notify(userId: string, message: string): Promise<void> {
console.log(`sms to ${userId}: ${message}`);
}
}
@@ -0,0 +1,7 @@
export class User {
constructor(
public readonly id: string,
public readonly email: string,
public readonly name: string,
) {}
}
@@ -0,0 +1,26 @@
import { Body, Controller, Get, Param, Post } from '@nestjs/common';
import { UsersService } from './users.service';
import { User } from './user.entity';
@Controller('users')
export class UsersController {
constructor(private readonly usersService: UsersService) {}
@Get()
async list(): Promise<User[]> {
return this.usersService.findAll();
}
@Get(':id')
async getUser(@Param('id') id: string): Promise<User> {
return this.usersService.findOne(id);
}
@Post()
async create(
@Body('email') email: string,
@Body('name') name: string,
): Promise<User> {
return this.usersService.create(email, name);
}
}
@@ -0,0 +1,10 @@
import { Module } from '@nestjs/common';
import { UsersController } from './users.controller';
import { UsersService } from './users.service';
@Module({
controllers: [UsersController],
providers: [UsersService],
exports: [UsersService],
})
export class UsersModule {}
@@ -0,0 +1,26 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import { User } from './user.entity';
@Injectable()
export class UsersService {
private readonly users: Map<string, User> = new Map();
async findOne(id: string): Promise<User> {
const u = this.users.get(id);
if (!u) {
throw new NotFoundException(`user ${id} not found`);
}
return u;
}
async findAll(): Promise<User[]> {
return Array.from(this.users.values());
}
async create(email: string, name: string): Promise<User> {
const id = `usr_${this.users.size + 1}`;
const user = new User(id, email, name);
this.users.set(id, user);
return user;
}
}
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"target": "ES2021",
"module": "commonjs",
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"strict": true,
"baseUrl": "./src"
},
"include": ["src/**/*.ts"]
}
@@ -0,0 +1,55 @@
defmodule MyAppWeb.UserController do
use Phoenix.Controller, namespace: MyAppWeb
# Controller-level plug: runs before every action in this module.
# Phoenix resolves the binding at request time via the plug chain;
# there's no explicit call site between the action and the plug
# function.
plug :authenticate
# Guarded form with `when action in [...]`: same DI shape as Rails's
# `before_action :name, only: [...]`. The plug only fires for the
# listed actions.
plug :load_user when action in [:show, :update, :delete]
# Regular actions. Bodies deliberately do NOT call authenticate or
# load_user directly — the only path from these actions to those
# plugs is the decorator-dispatch edge we want to extract.
def index(conn, _params) do
render(conn, :index)
end
def show(conn, _params) do
render(conn, :show, user: conn.assigns[:user])
end
def update(conn, params) do
user = conn.assigns[:user]
_ = Map.merge(user, params)
render(conn, :show, user: user)
end
def delete(conn, _params) do
send_resp(conn, 204, "")
end
# Plug functions themselves. Their bodies have real call edges (to
# Plug.Conn helpers etc.) which the existing extractor handles;
# only the binding from actions to these functions is DI-gap
# territory.
def authenticate(conn, _opts) do
case get_session(conn, :user_id) do
nil -> halt_unauthorized(conn)
_ -> conn
end
end
def load_user(conn, _opts) do
user = %{id: conn.params["id"]}
assign(conn, :user, user)
end
defp halt_unauthorized(conn) do
send_resp(conn, 401, "unauthorized")
end
end
@@ -0,0 +1,7 @@
defmodule MyAppWeb.Router do
use Phoenix.Router
scope "/", MyAppWeb do
resources "/users", UserController
end
end
+13
View File
@@ -0,0 +1,13 @@
defmodule MyApp.MixProject do
# Not built — static corpus for the Gortex indexer.
use Mix.Project
def project do
[
app: :my_app,
version: "0.0.0",
elixir: "~> 1.14",
deps: [{:phoenix, "~> 1.7"}]
]
end
end
+58
View File
@@ -0,0 +1,58 @@
# Phoenix DI-recall fixture
#
# Phoenix's controller-level DI is `plug :function_name` — binds a
# middleware function to run before each action. Same shape as Rails
# before_action / NestJS @UseGuards: no explicit call site, framework
# dispatches based on module metadata.
#
# Only controller-scoped plugs are covered here. Router-pipeline
# plugs (`pipeline :browser do plug X end`) cross module boundaries
# and are noted as future work.
name: gortex-phoenix-di
cases:
# exact
- id: exact-UserController
tier: exact
query: UserController
expected: [lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController]
- id: exact-authenticate
tier: exact
query: authenticate
expected: [lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.authenticate]
- id: exact-load_user
tier: exact
query: load_user
expected: [lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.load_user]
# di_gap: controller-scoped plug dispatch
# `plug :authenticate` runs for every action. callers:authenticate
# should return every action in the controller.
- id: di_gap-callers-authenticate
tier: di_gap
query: "callers:lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.authenticate"
expected:
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.index
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.show
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.update
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.delete
# `plug :load_user when action in [:show, :update, :delete]` —
# only fires for those three actions. index/index-like should NOT
# be linked.
- id: di_gap-callers-load_user
tier: di_gap
query: "callers:lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.load_user"
expected:
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.show
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.update
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.delete
# Forward direction: call_chain from a guarded action should reach
# the plug functions it depends on.
- id: di_gap-call_chain-show
tier: di_gap
query: "call_chain:lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.show"
expected:
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.authenticate
- lib/my_app_web/controllers/user_controller.ex::MyAppWeb.UserController.load_user
+3
View File
@@ -0,0 +1,3 @@
source 'https://rubygems.org'
# Not installed — purely a static corpus for the Gortex indexer.
gem 'rails', '~> 7.1'
@@ -0,0 +1,13 @@
class ApplicationController < ActionController::Base
# Module-level before_action — runs for every action in every
# subclass. Rails resolves this at request-time via the callback
# chain; statically the binding between :require_login and the
# actions it guards has no explicit call site.
before_action :require_login
private
def require_login
redirect_to '/login' unless session[:user_id]
end
end
@@ -0,0 +1,20 @@
class SessionsController < ApplicationController
# skip_before_action removes the inherited :require_login for the
# listed actions — login/create obviously can't require a pre-
# existing login. Same decorator-dispatch shape, opposite sign.
skip_before_action :require_login, only: [:new, :create]
def new
end
def create
user = User.authenticate(params[:email], params[:password])
session[:user_id] = user.id
redirect_to '/'
end
def destroy
session.delete(:user_id)
redirect_to '/login'
end
end
@@ -0,0 +1,44 @@
class UsersController < ApplicationController
# Targeted before_action: only fires for the listed actions. The
# comma-separated symbol list + `only:` / `except:` filters are the
# common Rails idiom for attaching methods to a subset of actions.
before_action :set_user, only: [:show, :update, :destroy]
before_action :authorize_admin, only: :destroy
after_action :log_request
def index
@users = User.all
end
def show
render json: @user
end
def update
@user.update(user_params)
render json: @user
end
def destroy
@user.destroy
head :no_content
end
private
def set_user
@user = User.find(params[:id])
end
def authorize_admin
head :forbidden unless current_user&.admin?
end
def log_request
Rails.logger.info("request: #{request.path}")
end
def user_params
params.require(:user).permit(:email, :name)
end
end
@@ -0,0 +1,18 @@
class User < ApplicationRecord
has_many :posts
validates :email, presence: true, uniqueness: true
def admin?
role == 'admin'
end
def self.authenticate(email, password)
user = find_by(email: email)
user if user&.valid_password?(password)
end
def valid_password?(_pw)
true
end
end
+6
View File
@@ -0,0 +1,6 @@
Rails.application.routes.draw do
resources :users
get '/login', to: 'sessions#new'
post '/login', to: 'sessions#create'
delete '/logout', to: 'sessions#destroy'
end
+90
View File
@@ -0,0 +1,90 @@
# Rails DI-recall fixture
#
# Rails "DI" is mostly callback dispatch — before_action / after_action
# / skip_before_action in controllers attach methods to run before or
# after actions, without any explicit call site. This is the same shape
# as NestJS @UseGuards or Phoenix plug. Routes (`get '/users', to: ...`)
# are already handled by Gortex's HTTPExtractor so they're not the
# focus here.
#
# Tiering:
# exact — named symbols
# concept — paraphrases
# di — plain method call resolution (no callback extraction
# needed)
# di_gap — callback dispatch that's invisible without
# before_action-aware extraction
name: gortex-rails-di
cases:
# exact
- id: exact-UsersController
tier: exact
query: UsersController
expected: [app/controllers/users_controller.rb::UsersController]
- id: exact-ApplicationController
tier: exact
query: ApplicationController
expected: [app/controllers/application_controller.rb::ApplicationController]
- id: exact-require_login
tier: exact
query: require_login
expected: [app/controllers/application_controller.rb::ApplicationController.require_login]
- id: exact-authenticate
tier: exact
query: authenticate
expected: [app/models/user.rb::User.authenticate]
# concept
- id: concept-destroy-session
tier: concept
query: log a user out
expected: [app/controllers/sessions_controller.rb::SessionsController.destroy]
# di — real method body calls (User.find, session[:user_id], etc.)
- id: di-call_chain-create
tier: di
query: "call_chain:app/controllers/sessions_controller.rb::SessionsController.create"
expected: [app/models/user.rb::User.authenticate]
# di_gap — callback dispatch
# NOTE: inherited callbacks (ApplicationController.before_action
# :require_login → UsersController actions) aren't tracked. Would
# require cross-file class-hierarchy walking. See the Rails feature
# notes for future work.
# before_action :set_user, only: [:show, :update, :destroy] — the
# filter list matters: only those three actions should call set_user.
- id: di_gap-callers-set_user
tier: di_gap
query: "callers:app/controllers/users_controller.rb::UsersController.set_user"
expected:
- app/controllers/users_controller.rb::UsersController.show
- app/controllers/users_controller.rb::UsersController.update
- app/controllers/users_controller.rb::UsersController.destroy
# after_action :log_request — runs AFTER every action.
- id: di_gap-callers-log_request
tier: di_gap
query: "callers:app/controllers/users_controller.rb::UsersController.log_request"
expected:
- app/controllers/users_controller.rb::UsersController.index
- app/controllers/users_controller.rb::UsersController.show
# authorize_admin is before_action with only: :destroy (single symbol
# form, not an array). Verifies the extractor handles both shapes.
- id: di_gap-callers-authorize_admin
tier: di_gap
query: "callers:app/controllers/users_controller.rb::UsersController.authorize_admin"
expected:
- app/controllers/users_controller.rb::UsersController.destroy
# Forward direction: call_chain from a guarded action should include
# the before_action callbacks. Tests the same edges from the other
# side of the BFS.
- id: di_gap-call_chain-UsersController-show
tier: di_gap
query: "call_chain:app/controllers/users_controller.rb::UsersController.show"
expected:
- app/controllers/users_controller.rb::UsersController.set_user
- app/controllers/application_controller.rb::ApplicationController.require_login
+9
View File
@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- Not built — purely a static corpus for the Gortex indexer. -->
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example</groupId>
<artifactId>gortex-di-spring-fixture</artifactId>
<version>0.0.0</version>
<packaging>jar</packaging>
</project>
+114
View File
@@ -0,0 +1,114 @@
# Spring Boot DI-recall fixture
#
# Spring DI shapes in scope:
# - @Autowired field injection
# - @Autowired constructor injection (equivalent to @Inject)
# - @Qualifier for multi-implementation disambiguation
# - @Bean factory methods in a @Configuration class
#
# Tiering matches the NestJS/FastAPI fixtures:
# exact — symbol-name queries
# concept — paraphrase queries
# di — shapes Java's explicit-type system already handles
# (constructor param typed as the concrete class/interface
# — the resolver links method calls on `this.field` to
# the declared type)
# di_gap — shapes a type-aware resolver can't handle alone
# (interface injection with multiple impls, @Bean factory
# return values, @Qualifier disambiguation)
name: gortex-spring-di
cases:
# ------------------------------------------------------------------
# Tier 1 — exact
# ------------------------------------------------------------------
- id: exact-UserService
tier: exact
query: UserService
expected: [src/main/java/com/example/users/UserService.java::UserService]
- id: exact-OrderService
tier: exact
query: OrderService
expected: [src/main/java/com/example/orders/OrderService.java::OrderService]
- id: exact-JpaUserRepository
tier: exact
query: JpaUserRepository
expected: [src/main/java/com/example/users/JpaUserRepository.java::JpaUserRepository]
- id: exact-systemClock
tier: exact
query: systemClock
expected: [src/main/java/com/example/config/Clocks.java::Clocks.systemClock]
- id: exact-lookup
tier: exact
query: lookup
expected: [src/main/java/com/example/users/UserService.java::UserService.lookup]
# ------------------------------------------------------------------
# Tier 2 — concept
# ------------------------------------------------------------------
- id: concept-find-user-cache-fallback
tier: concept
query: look up a user with cache fallback
expected: [src/main/java/com/example/users/UserService.java::UserService.lookup]
- id: concept-system-clock
tier: concept
query: provide the system clock
expected: [src/main/java/com/example/config/Clocks.java::Clocks.systemClock]
# ------------------------------------------------------------------
# Tier 3 — di: Java's explicit type system handles these
# ------------------------------------------------------------------
# UserService.lookup calls this.cache.findById(...) and this.primary.findById(...).
# `cache` is a constructor-injected InMemoryUserRepository (concrete class),
# `primary` is a field-injected UserRepository (interface). Expected: the
# concrete findById — Java's type-aware resolver gets the concrete call directly.
- id: di-call_chain-UserService-lookup
tier: di
query: "call_chain:src/main/java/com/example/users/UserService.java::UserService.lookup"
expected:
- src/main/java/com/example/users/InMemoryUserRepository.java::InMemoryUserRepository.findById
- id: di-call_chain-OrderService-describe
tier: di
query: "call_chain:src/main/java/com/example/orders/OrderService.java::OrderService.describe"
expected:
- src/main/java/com/example/users/UserService.java::UserService.lookup
# ------------------------------------------------------------------
# Tier 4 — di_gap: Spring-specific resolution beyond Java types
# ------------------------------------------------------------------
# Interface injection + multiple implementations: UserService.primary is
# typed as UserRepository. Two @Repository beans implement it
# (JpaUserRepository, InMemoryUserRepository). Without a @Primary or
# @Qualifier annotation Spring itself fails at runtime — the static
# resolver picks one arbitrarily. Either concrete implementation is
# a legitimate answer here; a future @Primary-aware extractor would
# narrow this to a specific impl.
- id: di_gap-call_chain-UserService-lookup-interface
tier: di_gap
query: "call_chain:src/main/java/com/example/users/UserService.java::UserService.lookup"
expected:
- src/main/java/com/example/users/JpaUserRepository.java::JpaUserRepository.findById
- src/main/java/com/example/users/InMemoryUserRepository.java::InMemoryUserRepository.findById
# @Bean factory method: OrderService receives a Clock via constructor,
# but Clock isn't a @Service or @Component — it's provided by
# Clocks.systemClock() which returns Clock. Without @Bean-aware
# extraction, callers(systemClock) is empty and users of Clock can't
# trace back to its factory.
- id: di_gap-callers-systemClock
tier: di_gap
query: "callers:src/main/java/com/example/config/Clocks.java::Clocks.systemClock"
expected:
- src/main/java/com/example/orders/OrderService.java::OrderService
# @Qualifier disambiguation: UserService constructor param is
# @Qualifier("inMemoryUsers") InMemoryUserRepository. The type is
# explicit (concrete class), so Java's resolver already handles it.
# This case should pass even WITHOUT Spring-aware extraction — tests
# that the qualified injection doesn't break the baseline.
- id: di-call_chain-UserService-cache-path
tier: di
query: "call_chain:src/main/java/com/example/users/UserService.java::UserService.lookup"
expected:
- src/main/java/com/example/users/InMemoryUserRepository.java::InMemoryUserRepository.findById
@@ -0,0 +1,17 @@
package com.example.config;
import java.time.Clock;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
// @Configuration + @Bean: factory-method style provider. Consumers
// that @Autowire a Clock field receive whatever this method returns.
// Statically the binding between the return type (Clock) and the
// producing method (systemClock) is visible only inside this class.
@Configuration
public class Clocks {
@Bean
public Clock systemClock() {
return Clock.systemUTC();
}
}
@@ -0,0 +1,30 @@
package com.example.orders;
import java.time.Clock;
import java.time.Instant;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.example.users.User;
import com.example.users.UserService;
// Multi-field injection by constructor. The Clock dependency is
// satisfied by the @Bean method in com.example.config.Clocks; the
// UserService dependency is the typical @Service-annotated class.
@Service
public class OrderService {
private final UserService users;
private final Clock clock;
@Autowired
public OrderService(UserService users, Clock clock) {
this.users = users;
this.clock = clock;
}
public String describe(String userId) {
User u = users.lookup(userId).orElseThrow();
Instant now = clock.instant();
return "order for " + u.getEmail() + " at " + now;
}
}
@@ -0,0 +1,23 @@
package com.example.users;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import org.springframework.stereotype.Repository;
// Second implementation. Without @Primary on one of the two, a plain
// @Autowired UserRepository field is ambiguous — Spring requires a
// @Qualifier to pick between them.
@Repository("inMemoryUsers")
public class InMemoryUserRepository implements UserRepository {
private final Map<String, User> users = new HashMap<>();
public Optional<User> findById(String id) {
return Optional.ofNullable(users.get(id));
}
public User save(User u) {
users.put(u.getId(), u);
return u;
}
}
@@ -0,0 +1,18 @@
package com.example.users;
import java.util.Optional;
import org.springframework.stereotype.Repository;
// First implementation of UserRepository. The @Primary elsewhere
// determines which Spring picks when a consumer @Autowires the
// interface without a qualifier.
@Repository
public class JpaUserRepository implements UserRepository {
public Optional<User> findById(String id) {
return Optional.empty();
}
public User save(User u) {
return u;
}
}
@@ -0,0 +1,14 @@
package com.example.users;
public class User {
private final String id;
private final String email;
public User(String id, String email) {
this.id = id;
this.email = email;
}
public String getId() { return id; }
public String getEmail() { return email; }
}
@@ -0,0 +1,8 @@
package com.example.users;
import java.util.Optional;
public interface UserRepository {
Optional<User> findById(String id);
User save(User u);
}
@@ -0,0 +1,30 @@
package com.example.users;
import java.util.Optional;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.stereotype.Service;
// Mixed injection styles: field injection for the primary repository
// and constructor injection for the qualified in-memory one. Both are
// common in production Spring code.
@Service
public class UserService {
@Autowired
private UserRepository primary;
private final InMemoryUserRepository cache;
@Autowired
public UserService(@Qualifier("inMemoryUsers") InMemoryUserRepository cache) {
this.cache = cache;
}
public Optional<User> lookup(String id) {
Optional<User> hit = cache.findById(id);
if (hit.isPresent()) {
return hit;
}
return primary.findById(id);
}
}
+9
View File
@@ -0,0 +1,9 @@
{
"name": "gortex/di-symfony-fixture",
"type": "project",
"description": "Minimal Symfony fixture for Gortex DI recall — not installed.",
"require": {
"symfony/framework-bundle": "^6.3",
"symfony/event-dispatcher": "^6.3"
}
}
+40
View File
@@ -0,0 +1,40 @@
# Symfony attribute-based DI fixture
#
# Symfony's PHP-native DI shape is PHP 8 attributes: #[AsEventListener]
# binds a method (or class) to an event, #[Route] maps a method to an
# HTTP route (already handled by the HTTPExtractor), #[Autowire]
# injects a specific service. This fixture focuses on event-listener
# attributes — the clearest DI-as-attribute case.
name: gortex-symfony-di
cases:
# exact
- id: exact-UserCreated
tier: exact
query: UserCreated
expected: [src/Event/UserCreated.php::UserCreated]
- id: exact-WelcomeEmailListener
tier: exact
query: WelcomeEmailListener
expected: [src/EventListener/WelcomeEmailListener.php::WelcomeEmailListener]
- id: exact-AuditListener
tier: exact
query: AuditListener
expected: [src/EventListener/AuditListener.php::AuditListener]
# di_gap: #[AsEventListener] binds listener to event. Without
# attribute extraction, find_usages(UserCreated) misses the
# listener classes entirely — only the `new UserCreated(...)`
# construction site in the controller shows up.
- id: di_gap-usages-UserCreated
tier: di_gap
query: "usages:src/Event/UserCreated.php::UserCreated"
expected:
- src/EventListener/WelcomeEmailListener.php::WelcomeEmailListener.onCreated
- src/EventListener/AuditListener.php::AuditListener
- id: di_gap-usages-UserDeleted
tier: di_gap
query: "usages:src/Event/UserDeleted.php::UserDeleted"
expected:
- src/EventListener/WelcomeEmailListener.php::WelcomeEmailListener.onDeleted
@@ -0,0 +1,27 @@
<?php
namespace App\Controller;
use App\Event\UserCreated;
use Symfony\Component\EventDispatcher\EventDispatcherInterface;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Attribute\Route;
// Standard Symfony controller. Constructor-injected dispatcher +
// route attributes. The event dispatch in register() will fire every
// listener bound to UserCreated — the listener bindings live in the
// #[AsEventListener] attributes in the EventListener directory.
class UserController
{
public function __construct(
private EventDispatcherInterface $events,
) {}
#[Route('/users', methods: ['POST'])]
public function register(): Response
{
$event = new UserCreated('usr_1');
$this->events->dispatch($event);
return new Response('ok');
}
}
@@ -0,0 +1,11 @@
<?php
namespace App\Event;
// Domain event fired when a user is registered. Listeners are bound
// via #[AsEventListener(event: UserCreated::class)] on their methods;
// Symfony's dispatcher routes the event instance to each listener.
class UserCreated
{
public function __construct(public readonly string $userId) {}
}
@@ -0,0 +1,8 @@
<?php
namespace App\Event;
class UserDeleted
{
public function __construct(public readonly string $userId) {}
}
@@ -0,0 +1,19 @@
<?php
namespace App\EventListener;
use App\Event\UserCreated;
use Symfony\Component\EventDispatcher\Attribute\AsEventListener;
// Class-level attribute shorthand. Symfony scans the class for a
// method whose signature matches the event type; here that's
// __invoke taking UserCreated. This is a shorter spelling of the
// method-level form above.
#[AsEventListener(event: UserCreated::class)]
class AuditListener
{
public function __invoke(UserCreated $event): void
{
$_ = $event->userId;
}
}
@@ -0,0 +1,27 @@
<?php
namespace App\EventListener;
use App\Event\UserCreated;
use App\Event\UserDeleted;
use Symfony\Component\EventDispatcher\Attribute\AsEventListener;
// Method-level attributes. Each listener method declares exactly
// which event it handles. Symfony dispatches the right method when
// the event is fired — no explicit call site from anywhere, the
// attribute IS the binding.
class WelcomeEmailListener
{
#[AsEventListener(event: UserCreated::class, priority: 10)]
public function onCreated(UserCreated $event): void
{
// Normally sends an email; body is a stub for this fixture.
$_ = $event->userId;
}
#[AsEventListener(event: UserDeleted::class)]
public function onDeleted(UserDeleted $event): void
{
$_ = $event->userId;
}
}

Some files were not shown because too many files have changed in this diff Show More