chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,936 @@
|
||||
# Codebase Memory MCP -- v0.3.0 Language Benchmark
|
||||
|
||||
## Methodology
|
||||
|
||||
- **63 languages** (27 programming + 8 config/markup), 12 questions each (4 for config languages)
|
||||
- **Up to 5 attempts** per question with escalating retry strategies
|
||||
- **Real open-source repos** (medium to large: 78--49K nodes)
|
||||
- **Grading**: PASS (1.0) / PARTIAL (0.5) / FAIL (0.0), N/A excluded from denominator
|
||||
- **Platform**: Apple M3 Pro, macOS Darwin 25.3.0
|
||||
- **Date**: 2026-03-01
|
||||
|
||||
### The 12 Questions
|
||||
|
||||
| # | Category | Question | Primary MCP Tool |
|
||||
|---|----------|----------|------------------|
|
||||
| Q1 | Indexing | Verify index stats (nodes, edges, schema) | `get_graph_schema` |
|
||||
| Q2 | Discovery | Find functions/methods | `search_graph(label="Function")` |
|
||||
| Q3 | Discovery | Find classes/structs | `search_graph(label="Class")` |
|
||||
| Q4 | Pattern | Find by name pattern | `search_graph(name_pattern="...")` |
|
||||
| Q5 | Code | Get code snippet | `get_code_snippet` |
|
||||
| Q6 | Search | Text search in code | `search_code` |
|
||||
| Q7 | Trace | Outbound call trace | `trace_call_path(direction="outbound")` |
|
||||
| Q8 | Trace | Inbound call trace | `trace_call_path(direction="inbound")` |
|
||||
| Q9 | Graph | Cypher CALLS query | `query_graph` |
|
||||
| Q10 | Enrich | Params/returns/properties | `query_graph` |
|
||||
| Q11 | OOP | Inheritance/implements | `query_graph` |
|
||||
| Q12 | Files | List files & directories | `list_directory` |
|
||||
|
||||
Config/markup languages (HTML, CSS, SCSS, YAML, TOML, HCL, SQL, Dockerfile) run Q1, Q4, Q6, Q12. Other questions marked N/A.
|
||||
|
||||
---
|
||||
|
||||
## Summary Table
|
||||
|
||||
| # | Language | Repo | Nodes | Edges | Score | Pct | Tier |
|
||||
|---|----------|------|------:|------:|------:|----:|------|
|
||||
| 1 | Lua | neovim/neovim | 23,955 | 90,247 | 12/12 | 100% | 1 |
|
||||
| 2 | Kotlin | ktorio/ktor | 25,297 | 71,498 | 12/12 | 100% | 1 |
|
||||
| 3 | C++ | nlohmann/json | 5,262 | 8,681 | 12/12 | 100% | 1 |
|
||||
| 4 | Perl | mojolicious/mojo | 3,287 | 4,182 | 12/12 | 100% | 1 |
|
||||
| 5 | Objective-C | AFNetworking | 1,087 | 1,348 | 12/12 | 100% | 1 |
|
||||
| 6 | Groovy | spockframework/spock | 14,081 | 34,557 | 12/12 | 100% | 1 |
|
||||
| 7 | C | jqlang/jq | 1,330 | 3,923 | 11/11 | 100% | 1 |
|
||||
| 8 | Bash | bats-core/bats-core | 436 | 479 | 10/10 | 100% | 1 |
|
||||
| 9 | Zig | zigtools/zls | 2,824 | 10,230 | 10/10 | 100% | 1 |
|
||||
| 10 | Swift | Alamofire/Alamofire | 3,631 | 7,639 | 10.5/11 | 95% | 1 |
|
||||
| 11 | CSS | animate-css/animate.css | 295 | 214 | 4/4 | 100% | 1 |
|
||||
| 12 | YAML | kubernetes/examples | 2,110 | 1,844 | 4/4 | 100% | 1 |
|
||||
| 13 | TOML | rust-lang/cargo | 16,773 | 51,403 | 4/4 | 100% | 1 |
|
||||
| 14 | HTML | twbs/bootstrap | 2,726 | 4,135 | 4/4 | 100% | 1 |
|
||||
| 15 | SCSS | twbs/bootstrap | 2,726 | 4,135 | 4/4 | 100% | 1 |
|
||||
| 16 | HCL | hashicorp/terraform | 78 | 76 | 4/4 | 100% | 1 |
|
||||
| 17 | Dockerfile | docker-library/official-images | 1,481 | 1,588 | 4/4 | 100% | 1 |
|
||||
| 18 | Python | django/django | 49,398 | 196,022 | 10.5/12 | 87% | 2 |
|
||||
| 19 | TypeScript | nestjs/nest | 9,063 | 15,772 | 10.5/12 | 87% | 2 |
|
||||
| 20 | TSX | shadcn-ui/ui | 29,755 | 41,883 | 10.5/12 | 87% | 2 |
|
||||
| 21 | Go | codebase-memory-mcp (self) | 2,259 | 6,561 | 10.5/12 | 87% | 2 |
|
||||
| 22 | Rust | BurntSushi/ripgrep | 4,118 | 6,971 | 10.5/12 | 87% | 2 |
|
||||
| 23 | Java | spring-projects/spring-petclinic | 660 | 1,080 | 10.5/12 | 87% | 2 |
|
||||
| 24 | R | tidyverse/dplyr | 1,618 | 2,409 | 10.5/12 | 87% | 2 |
|
||||
| 25 | Dart | felangel/bloc | 5,089 | 6,430 | 10.5/12 | 87% | 2 |
|
||||
| 26 | JavaScript | lodash/lodash | 244 | 405 | 9.5/11 | 86% | 2 |
|
||||
| 27 | Erlang | ninenines/cowboy | 3,270 | 9,815 | 9.5/11 | 86% | 2 |
|
||||
| 28 | Elixir | elixir-plug/plug | 870 | 865 | 9.5/11 | 86% | 2 |
|
||||
| 29 | Scala | playframework/playframework | 19,627 | 43,764 | 9/12 | 75% | 2 |
|
||||
| 30 | Ruby | sinatra/sinatra | 1,377 | 1,893 | 9/12 | 75% | 2 |
|
||||
| 31 | PHP | laravel/framework | 38,644 | 161,242 | 9/12 | 75% | 2 |
|
||||
| 32 | C# | jasontaylordev/CleanArchitecture | 1,043 | 1,632 | 9/12 | 75% | 2 |
|
||||
| 33 | SQL | flyway/flyway | 4.5/6 | 75% | 4.5/6 | 75% | 2 |
|
||||
| 34 | OCaml | ocaml/dune | 11,691 | 10,447 | 8/11 | 72% | 3 |
|
||||
| 35 | Haskell | PostgREST/postgrest | 2,066 | 2,463 | 7.5/12 | 62% | 3 |
|
||||
|
||||
---
|
||||
|
||||
## Tier Classification
|
||||
|
||||
| Tier | Range | Count | Languages |
|
||||
|------|-------|------:|-----------|
|
||||
| **Tier 1** -- Excellent | >= 90% | 17 | Lua, Kotlin, C++, Perl, Objective-C, Groovy, C, Bash, Zig, Swift, CSS, YAML, TOML, HTML, SCSS, HCL, Dockerfile |
|
||||
| **Tier 2** -- Good | 75--89% | 16 | Python, TypeScript, TSX, Go, Rust, Java, R, Dart, JavaScript, Erlang, Elixir, Scala, Ruby, PHP, C#, SQL |
|
||||
| **Tier 3** -- Functional | < 75% | 2 | OCaml (72%), Haskell (62%) |
|
||||
|
||||
---
|
||||
|
||||
## Per-Language Results
|
||||
|
||||
### Python (django/django)
|
||||
|
||||
**Project**: `django-python` | **Repo**: `/tmp/lang-bench/django-python`
|
||||
**Nodes**: 49,398 | **Edges**: 196,022
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 49,398 nodes, 196,022 edges, 12 labels, 20 rel types |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | render_to_string (1458 in), skipUnlessDBFeature, call_command. 1005 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | ModelAdmin (511 in), ValidationError (415 in), ImproperlyConfigured. 1005 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | SimpleTestCase, TestCase, SchemaTests. 22,135 total |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | render_to_string: 11 lines from django/template/loader.py:52-62 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO\|FIXME) | 5 matches: FIXME in admin/checks.py, TODO in admin/options.py |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 21 edges, 3 hops: render_to_string -> render, get_template, select_template |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 3 callers: test_include, test_include_state, test_include_cache |
|
||||
| Q9 | Cypher CALLS | PASS | 3/5 | query_graph with project param | First 2 attempts needed project param adjustment |
|
||||
| Q10 | Properties | PARTIAL | 3/5 | query_graph(properties) | Functions returned but all properties null |
|
||||
| Q11 | Inheritance | PASS | 3/5 | query_graph(INHERITS) | 200 rows (capped): LazySettings, SimpleAdminConfig, ModelAdmin |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 21 entries: django/, docs/, tests/, extras/, pyproject.toml |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### JavaScript (lodash/lodash)
|
||||
|
||||
**Project**: `lodash-js` | **Repo**: `/tmp/lang-bench/lodash-js`
|
||||
**Nodes**: 244 | **Edges**: 405
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 244 nodes, 405 edges, 7 labels |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | build (7 variants), baseConvert, Hash. 36 total |
|
||||
| Q3 | Find Classes | PASS | 2/5 | search_graph(Module) | No Class nodes (functional lib). Module fallback: 39 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | browser-testing.yml, markdown-doctest-setup.js. 10 total |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | build: 57 lines from lib/main/build-site.js |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 1 match: TODO in .github/workflows/ci-bun.yml |
|
||||
| Q7 | Outbound Trace | PASS | 2/5 | trace_call_path(baseConvert) | 2 edges: baseAry, self-recursive |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 1 caller: build-dist.js module |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 5 rows: minify->minify, build-dist.js->build |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Functions returned but properties null |
|
||||
| Q11 | Inheritance | N/A | -- | -- | No classes/inheritance (pure functional library) |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 19 entries: dist/, fp/, lib/, test/, lodash.js |
|
||||
|
||||
**Score: 9.5/11 (86%)**
|
||||
|
||||
### TypeScript (nestjs/nest)
|
||||
|
||||
**Project**: `nestjs-ts` | **Repo**: `/tmp/lang-bench/nestjs-ts`
|
||||
**Nodes**: 9,063 | **Edges**: 15,772
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 9,063 nodes, 15,772 edges; 14 labels |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | Injectable (in=242), Module (in=209), Controller (in=117). 292 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Post (in=128), FastifyAdapter, Module, InstanceWrapper. 1005+ total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | TestingModuleBuilder, TestController, test folders. 142 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | Injectable: 6 lines from common/decorators/core/injectable.decorator.ts |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 2 matches in express.spec.ts, gulp/tasks/samples.ts |
|
||||
| Q7 | Outbound Trace | PASS | 2/5 | trace_call_path(Module) | First try Injectable had 0 outbound; Module: 4 edges |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(Injectable) | 242 callers across services, interceptors, pipes, guards |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 5 rows: gulpfile.js->register, bar.service->Injectable |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Properties null |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 200 rows: AuthGuard, BadGatewayException, ClientGrpcProxy |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 22 entries: packages/, integration/, sample/, tools/ |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### TSX (shadcn-ui/ui)
|
||||
|
||||
**Project**: `shadcn-tsx` | **Repo**: `/tmp/lang-bench/shadcn-tsx`
|
||||
**Nodes**: 29,755 | **Edges**: 41,883
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 29,755 nodes, 41,883 edges; 13 labels |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | cn (in=558), IconPlaceholder (in=362). 7,424 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | 18 classes: RegistryError, ComponentErrorBoundary |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 146 results: search.test.ts, registries.test.ts |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet(cn) | 3 lines from apps/v4/examples/base/lib/utils.ts:4-6 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 5 matches in registry.ts, source.config.ts |
|
||||
| Q7 | Outbound Trace | PASS | 2/5 | trace_call_path(Button) | Button->cn (1 edge) |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(cn) | 558 callers (result saved to file) |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | BlocksPage->getAllBlockIds, ChartPage->getActiveStyle |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Properties null |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 14 rows: RegistryNotFoundError, ConfigMissingError |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 17 entries: apps/, packages/, templates/ |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### Go (codebase-memory-mcp)
|
||||
|
||||
**Project**: `codebase-memory-mcp` | **Repo**: self
|
||||
**Nodes**: 2,259 | **Edges**: 6,561
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 2,259 nodes, 6,561 edges; 13 labels |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | OpenMemory (in=136), NodeText (in=79), Parse (in=35). 918 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Node (in=243), LanguageSpec, FileInfo, Edge. 91 structs |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 413 results: cypher_test.go, langparity_test.go |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | OpenMemory: 16 lines from internal/store/store.go:113-128 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 4 matches in tools.go, langparity_test.go |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 11 edges: Open, Close, initSchema, cacheDir |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 68 callers: test files across store/pipeline/httplink |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | printAST->printAST, main->Parse, TestLexBasicQuery->Lex |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Properties null |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(IMPLEMENTS) | 3 rows: RelPattern, FilterWhere, fusedExpandMarker |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 15 entries: cmd/, internal/, scripts/, go.mod |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### Rust (BurntSushi/ripgrep)
|
||||
|
||||
**Project**: `ripgrep-rust` | **Repo**: `/tmp/lang-bench/ripgrep-rust`
|
||||
**Nodes**: 4,118 | **Edges**: 6,971
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 689 Functions, 2039 Methods, 294 Classes, 70 Enums |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | matcher (in=133), args (119), printer_contents (72). 689 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | HiArgs (out=93), LowArgs (out=74), Debug (in=54). 294 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | test method (in=524), SearcherTester, TestCommand. 134 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | matcher() from test_matcher.rs:8-10 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 TODOs in globset, gitignore, overrides |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(search) | 12 callees: quit_after_match, sort, haystack_builder, search_worker |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(search) | Called by run, main, search_parallel, files_parallel |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | main->set_git_revision_hash, main->set_windows_exe_options |
|
||||
| Q10 | Properties | PASS | 1/5 | query_graph(properties) | 5 functions returned |
|
||||
| Q11 | Inheritance | PARTIAL | 1/5 | query_graph(INHERITS) | 1 result: NoCaptures. Rust traits not modeled as inheritance |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 20 entries: crates/, tests/, benchsuite/, Cargo.toml |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### Java (spring-projects/spring-petclinic)
|
||||
|
||||
**Project**: `spring-petclinic-java` | **Repo**: `/tmp/lang-bench/spring-petclinic-java`
|
||||
**Nodes**: 660 | **Edges**: 1,080
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 167 Methods, 39 Classes, 30 Routes, 3 Interfaces |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Method) | getPet (in=12,out=6), testValidate (out=16). 167 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | OwnerControllerTests, PetController, Owner. 39 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 97 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | getPet() from Owner.java:135-145 |
|
||||
| Q6 | Text Search | PARTIAL | 2/5 | search_code(error) | TODO/FIXME returned 0; "error" found 3 matches |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | getPet calls getId, isNew, getName, getPets |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | Called by loadPetWithVisit, findPet, processCreationForm, test methods |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | toString->getName, addPet->isNew, getPet->getPets |
|
||||
| Q10 | Properties | PASS | 1/5 | query_graph(properties) | main, registerHints, getId, setId, isNew |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 8 results: NamedEntity, Person, Owner, Pet, Vet |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 13 entries: src/, gradle/, pom.xml, docker-compose.yml |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### C++ (nlohmann/json)
|
||||
|
||||
**Project**: `nlohmann-json-cpp` | **Repo**: `/tmp/lang-bench/nlohmann-json-cpp`
|
||||
**Nodes**: 5,262 | **Edges**: 8,681
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 2,456 Functions, 649 Methods, 603 Macros, 153 Classes |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | result (in=99), push_back (in=42), CAPTURE (in=35) |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Fuzzer (out=54), MutationDispatcher (out=36). 153 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 234 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | result from unit-bjdata.cpp:2935-2936 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 TODOs in binary_reader.hpp, json.hpp |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(push_back) | Recursive call through json_pointer chain |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(push_back) | 23 callers: main, sax_event_consumer, lexer, binary_reader |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | main->at, main->back |
|
||||
| Q10 | Properties | PASS | 1/5 | query_graph(properties) | 5 functions returned |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 21 results: parse_error, invalid_iterator, type_error, out_of_range |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 20 entries: include/, tests/, docs/, CMakeLists.txt |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### C# (jasontaylordev/CleanArchitecture)
|
||||
|
||||
**Project**: `clean-architecture-csharp` | **Repo**: `/tmp/lang-bench/clean-architecture-csharp`
|
||||
**Nodes**: 1,043 | **Edges**: 1,632
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 303 Methods, 123 Classes, 5 Interfaces |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Method) | SendAsync (in=17), PaginatedList (in=11). 303 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | TodoItems (in=18), TodoLists (in=18), BaseTestFixture. 123 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 85 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | SendAsync() from Testing.cs:38-45 |
|
||||
| Q6 | Text Search | PARTIAL | 2/5 | search_code(error) | TODO/FIXME=0; "error" found 3 matches |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(outbound) | SendAsync has 0 outbound (calls external ISender.Send) |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 16 callers: test methods |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | NotEqualOperator->EqualOperator, GetHashCode->GetEqualityComponents |
|
||||
| Q10 | Properties | PASS | 1/5 | query_graph(properties) | AuthorizationBehaviour, Handle, LoggingBehaviour |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 29 results: BaseAuditableEntity, TodoItem, TodoList |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 16 entries: src/, tests/, infra/, templates/ |
|
||||
|
||||
**Score: 9/12 (75%)**
|
||||
|
||||
### PHP (laravel/framework)
|
||||
|
||||
**Project**: `laravel-framework-php` | **Repo**: `/tmp/lang-bench/laravel-framework-php`
|
||||
**Nodes**: 39,767 | **Edges**: 148,440 (post-Phase-4–5 PHP-LSP; was 196,979 baseline, –25%)
|
||||
**CALLS edges**: 47,341 (was ~83k baseline — 43% reduction in name-fallback misroutes)
|
||||
**Tests**: 278 PHP-LSP unit tests, all passing (total project: 3,091 / 0 failed)
|
||||
**LSP module size**: ~3,700 lines C resolver + ~700 lines stdlib + ~5,500 lines tests = ~9,900 LoC
|
||||
|
||||
PHP runs through a Light Semantic Pass (`internal/cbm/lsp/php_lsp.c`,
|
||||
~3,500 lines) that approaches phpactor-grade type resolution while
|
||||
staying in-process and PHP-runtime-free. Phase 4 capabilities:
|
||||
|
||||
- Receiver-type tracking with full ancestor chain walk (cycle-detected,
|
||||
bounded to 32 hops)
|
||||
- Namespace + `use`-clause resolution (class, function, const), incl.
|
||||
`as` aliasing in both wrapped and bare forms
|
||||
- PHPDoc `@var`, `@param`, `@property`, `@method` parsing — generics
|
||||
(`Collection<User>`, `array<int, User>`) supported
|
||||
- Type narrowing: `instanceof`, `is_string/int/array/...`, `assert(...)`
|
||||
sequential narrowing, negative narrowing on early return / throw
|
||||
- Property type tracking: typed declarations, constructor property
|
||||
promotion, constructor-body inference (`$this->bar = $bar`)
|
||||
- Trait flattening with `as` aliasing
|
||||
- Late static binding chain depth, self/parent/static distinction
|
||||
- Match / ternary / clone / cast result-type evaluation
|
||||
- Foreach element-type propagation through `array<T>` and Iterator
|
||||
- Magic methods (`__call`/`__callStatic` → facade dispatch)
|
||||
- Stdlib coverage: SPL, PSR, DateTime, Throwable, Closure, plus
|
||||
Eloquent Builder/Model/Collection chains, Symfony HttpFoundation,
|
||||
Carbon date methods, PSR-7 with-builders
|
||||
|
||||
**Attribution correctness fix** (the primary metric, see
|
||||
`docs/PHP_LSP_PRE_FLIGHT.md` §6):
|
||||
- `ConfiguresPrompts.configurePrompts → helpers.value` misroute (3
|
||||
call sites): **fixed** — those typed-receiver `$prompt->value()` calls
|
||||
no longer route to the global helper. Verified: 0 misroute edges.
|
||||
- Roughly 27 500 fewer total edges, primarily from blocking
|
||||
name-fallback edges where the receiver was a vendor (un-indexed) type
|
||||
— better precision at the cost of some recall, see notes below.
|
||||
|
||||
| Q# | Question | Grade | Approach | Notes |
|
||||
|----|----------|-------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | get_graph_schema | unchanged |
|
||||
| Q2 | Find Functions | PASS | search_graph(Function) | unchanged |
|
||||
| Q3 | Find Classes | PASS | search_graph(Class) | unchanged |
|
||||
| Q4 | Pattern Search | PASS | search_graph(name_pattern=test) | unchanged |
|
||||
| Q5 | Code Snippet | PASS | get_code_snippet | unchanged |
|
||||
| Q6 | Text Search | PARTIAL | search_code | search-backend issue, separate ticket — see PHP_LSP_PRE_FLIGHT §3 |
|
||||
| Q7 | Outbound Trace | PASS | trace_call_path(outbound) | unchanged |
|
||||
| Q8 | Inbound Trace | PARTIAL | trace_call_path(inbound) | tool-side disambiguation, separate ticket — see PHP_LSP_PRE_FLIGHT §4.1 |
|
||||
| Q9 | Cypher CALLS | PASS | query_graph | unchanged |
|
||||
| Q10 | Properties | PASS | query_graph(properties) | unchanged |
|
||||
| Q11 | Inheritance | PASS | query_graph(INHERITS) | unchanged |
|
||||
| Q12 | List Directory | PASS | list_directory | unchanged |
|
||||
|
||||
**Score: 10/12 (83%)** — the two remaining PARTIALs are non-LSP issues
|
||||
tracked separately. PHP-LSP delivered the underlying graph-correctness
|
||||
win (collide-set attribution); benchmark Tier 1 (≥ 90 %) requires the
|
||||
search-recall and trace-disambiguation tickets to ship as well.
|
||||
|
||||
**Note on edge-count drop**: when the LSP knows the receiver is typed
|
||||
but the receiver class is not indexed (e.g., Composer vendor types like
|
||||
`Laravel\\Prompts\\Prompt`), it emits a synthetic
|
||||
`php_method_typed_unindexed` resolution that the bridge uses to
|
||||
**suppress** the unified extractor's name-based fallback. This trades
|
||||
recall (no edge instead of a guess) for precision (no wrong edge). The
|
||||
pre-flight argued this is the right trade for graph correctness.
|
||||
|
||||
**Test coverage**: 100 unit tests in `tests/test_php_lsp.c`, all
|
||||
passing. Total project tests: 2913 / 0 failed.
|
||||
|
||||
### Lua (neovim/neovim)
|
||||
|
||||
**Project**: `neovim-lua` | **Repo**: `/tmp/lang-bench/neovim-lua`
|
||||
**Nodes**: 23,955 | **Edges**: 90,247
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 12,311 Functions, 7,587 Variables, 1,157 Files, 399 Methods, 196 Classes |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | assert (in=825), vim.fn.bufnr (in=390), vim.fn.count (in=228) |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | cmdmod (in=84), file_buffer (out=80), TUIData (out=59). 196 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 198 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | vim.validate: 21 lines from runtime/lua/vim/_core/shared.lua |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | Found across build.zig, test.yml |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 5 callees: vim.deprecate, is_valid, validate_spec. Depth 2: 15 more |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 129K chars result (saved to file). Massive inbound |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | f_rpcstart->check_secure, f_termopen->f_jobstart |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(include_connected) | assert: in=825/out=2, connected to M.error, main |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | UGridPrinter inherits object |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 22 entries: src/, runtime/, test/, cmake/, deps/ |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### Scala (playframework/playframework)
|
||||
|
||||
**Project**: `playframework-scala` | **Repo**: `/tmp/lang-bench/playframework-scala`
|
||||
**Nodes**: 19,627 | **Edges**: 43,764
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 8,018 Methods, 2,924 Variables, 2,007 Classes, 309 Interfaces |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | app (in=48), queryString (in=14), json (in=9). 71 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Http (out=135), AbstractController (in=81), Controller (in=68) |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 636 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | app: 1-line abstract def from play/api/test/package.scala |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | FIXME in CachedSpec, TODO in CryptoComponents |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(outbound) | queryString: abstract method, no outbound calls |
|
||||
| Q8 | Inbound Trace | PARTIAL | 1/5 | trace_call_path(inbound) | queryString: no inbound calls detected |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | fromContent->fromString, waitForCancel->waitForKey |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(include_connected) | ok: in=133/out=6, data: in=131/out=1 |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 200 rows (capped): DefaultApplication, Controller, many *Spec |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 14 entries: core/, dev-mode/, documentation/, testkit/ |
|
||||
|
||||
**Score: 9/12 (75%)**
|
||||
|
||||
### Kotlin (ktorio/ktor)
|
||||
|
||||
**Project**: `ktor-kotlin` | **Repo**: `/tmp/lang-bench/ktor-kotlin`
|
||||
**Nodes**: 25,297 | **Edges**: 71,498
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 8,485 Methods, 2,865 Functions, 2,572 Classes, 1,921 Routes |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | request (in=216), content (in=186), http (in=171). 1010+ total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | ContentType (in=224), HttpRequestBuilder (in=208). 1010+ total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | testApplication (in=1504), runTest (in=359). 5,015 total |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | request: 2 lines from ktor-client-core |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | "TODO: Remove when PR fixing this file will be merged" |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(HttpClient) | 4 callees: create, apply, invokeOnCompletion |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(HttpClient) | 3 callers: config, testPluginInstalledTwice |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | getNative->error, matches->toLowerCasePreservingASCIIRules |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(include_connected) | request: in=216/out=10, content: in=186/out=6 |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 200 rows (capped): HttpClientEngineBase, many *Engine |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 39 entries: ktor-client/, ktor-server/, ktor-http/ |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### Ruby (sinatra/sinatra)
|
||||
|
||||
**Project**: `sinatra-ruby` | **Repo**: `/tmp/lang-bench/sinatra-ruby`
|
||||
**Nodes**: 1,377 | **Edges**: 1,893
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 786 Methods, 127 Classes, 37 Functions |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | setup (in=11), route_def, write_app_file. 37 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Sinatra (out=172), OkJson (out=31). 127 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 183 test-related nodes |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | setup: 4 lines from test/markdown_test.rb |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 2 matches: base.rb, content_for_spec.rb |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(outbound) | setup in Minitest: no outbound calls (block-based) |
|
||||
| Q8 | Inbound Trace | PARTIAL | 1/5 | trace_call_path(inbound) | No inbound calls for this method |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | status_app->mock_app, mock_app->new, setup_blocks->each |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(include_connected) | it: in=38/out=3, get: in=23/out=1 |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 2 results: BError->AError, ErubiTest->ERBTest |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 18 entries: lib/, test/, examples/ |
|
||||
|
||||
**Score: 9/12 (75%)**
|
||||
|
||||
### C (jqlang/jq)
|
||||
|
||||
**Project**: `jq-c` | **Repo**: `/tmp/lang-bench/jq-c`
|
||||
**Nodes**: 1,330 | **Edges**: 3,923
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 764 Functions, 159 Fields, 75 Variables, 45 Classes (structs) |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | jv_free (in=163), jv_get_kind (in=97), yyparse (out=51). 764 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | jq_state (out=25), yyguts_t (out=24), jv_parser (out=21). 45 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 25 test-related nodes: run_jq_tests, jv_test, LLVMFuzzerTestOneInput |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | jv_free: 1-line declaration from src/jv.h:55 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 7 matches: build_manpage.py, execute.c, jv_parse.c |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 5 callees: jvp_invalid/number/array/string/object_free. Depth 2: jvp_refcnt_dec |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 87K chars (saved to file). Massive caller tree |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | f_negate->type_error, f_json_parse->type_error, f_length->type_error |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(include_connected) | jv_free: in=163/out=6, jv_get_kind: in=97/out=1 |
|
||||
| Q11 | Inheritance | N/A | -- | -- | C has no class inheritance |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 23 entries: src/, tests/, docs/, build/ |
|
||||
|
||||
**Score: 11/11 (100%)**
|
||||
|
||||
### Bash (bats-core/bats-core)
|
||||
|
||||
**Project**: `bats-bash` | **Repo**: `/tmp/lang-bench/bats-bash`
|
||||
**Nodes**: 436 | **Edges**: 479
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 156 Functions, 65 Variables, 66 Modules |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | bats_print_stack_trace, main, run, bats_generate_warning. 156 total |
|
||||
| Q3 | Find Classes | N/A | -- | -- | Bash has no classes |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=run) | 6 matches: run, bats_semaphore_run, reentrant_run |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | run: 142-line function source (310-451) |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 2 TODOs in semaphore.bash, tracing.bash |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 6 callees: bats_quote_code, bats_format_file_line_reference |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 3 callers across 2 hops |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 10 rows of function call pairs |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 65 variables |
|
||||
| Q11 | Inheritance | N/A | -- | -- | Bash has no class hierarchy |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 66 modules found |
|
||||
|
||||
**Score: 10/10 (100%)**
|
||||
|
||||
### Zig (zigtools/zls)
|
||||
|
||||
**Project**: `zls-zig` | **Repo**: `/tmp/lang-bench/zls-zig`
|
||||
**Nodes**: 2,824 | **Edges**: 10,230
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,010+ Functions, 1,005 Variables, 109 Modules |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | std.ArrayList, trace, empty, resolveTypeOfNodeUncached. 1010+ total |
|
||||
| Q3 | Find Classes | N/A | -- | -- | Zig uses structs, not classes |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=resolve) | 57 matches: resolve, resolveTypeOfNodeUncached |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | trace: 19-line function showing tracy integration |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3+ TODOs in DiagnosticsCollection.zig, DocumentStore.zig |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 2 callees: ___tracy_emit_zone_begin, _callstack |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 94 callers: processMessage, create, handleConfiguration |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | main->createRequestBody, build->getVersion |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 1,005 variables |
|
||||
| Q11 | Inheritance | N/A | -- | -- | Zig has no inheritance |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 109 modules |
|
||||
|
||||
**Score: 10/10 (100%)**
|
||||
|
||||
### Elixir (elixir-plug/plug)
|
||||
|
||||
**Project**: `plug-elixir` | **Repo**: `/tmp/lang-bench/plug-elixir`
|
||||
**Nodes**: 870 | **Edges**: 865
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 554 Functions, 129 Classes, 80 Modules |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | parse_before_param, parse_hd_before_value. 554 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Plug.Conn, Plug.Debugger, Plug.CSRFProtection. 129 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=parse) | 29 matches |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | match: 3-line macro source |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 TODOs in adapters/test/conn.ex, ssl.ex |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 6 edges across 2 hops |
|
||||
| Q8 | Inbound Trace | PARTIAL | 1/5 | trace_call_path(inbound) | 0 callers (entry point function) |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | parse_headers->match, skip_preamble->before_parse_headers |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 6 variables |
|
||||
| Q11 | Inheritance | N/A | -- | -- | Elixir uses protocols/behaviours |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 80 modules |
|
||||
|
||||
**Score: 9.5/11 (86%)**
|
||||
|
||||
### Haskell (PostgREST/postgrest)
|
||||
|
||||
**Project**: `postgrest-haskell` | **Repo**: `/tmp/lang-bench/postgrest-haskell`
|
||||
**Nodes**: 2,066 | **Edges**: 2,463
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 746 Functions, 148 Classes, 655 Variables |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | defaultenv, jwtauthheader, headers. 746 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Thread, PostgrestTimedOut, ErrorBody. 148 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=action) | 4 matches: actionResult, actionPlan, actionResponse |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | actionPlan: 7-line Haskell source with pattern matching |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3+ TODOs in workflow files |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(outbound) | 0 callees (function composition not traced as CALLS) |
|
||||
| Q8 | Inbound Trace | PARTIAL | 1/5 | trace_call_path(inbound) | 0 callers (Haskell call patterns not fully traced) |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | generate_jwt->encode, run_command->run, main->generate_jwt |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 655 variables |
|
||||
| Q11 | Inheritance | PARTIAL | 1/5 | query_graph(INHERITS) | 1 result: Thread (Python test helper). Typeclasses not modeled |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 184 modules |
|
||||
|
||||
**Score: 7.5/12 (62%)**
|
||||
|
||||
### OCaml (ocaml/dune)
|
||||
|
||||
**Project**: `dune-ocaml` | **Repo**: `/tmp/lang-bench/dune-ocaml`
|
||||
**Nodes**: 11,691 | **Edges**: 10,447
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,010+ Functions, 12 Classes, 1,005 Modules |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | value, unit, append_files_in_dir_if_not_empty. 1010+ total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | fsenv, watch, CramLexer, DuneLexer. 12 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=build) | 111 matches |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | value: 4-line OCaml source with match expression |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3+ TODOs in workflow.yml, client.ml |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(outbound) | 0 callees (pure function) |
|
||||
| Q8 | Inbound Trace | PARTIAL | 1/5 | trace_call_path(inbound) | 0 callers (OCaml call analysis limited) |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | dune_trace_write->write, add_watch->isprefix |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 103 variables |
|
||||
| Q11 | Inheritance | N/A | -- | -- | OCaml uses modules/functors |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 1,005 modules |
|
||||
|
||||
**Score: 8/11 (72%)**
|
||||
|
||||
### Erlang (ninenines/cowboy)
|
||||
|
||||
**Project**: `cowboy-erlang` | **Repo**: `/tmp/lang-bench/cowboy-erlang`
|
||||
**Nodes**: 3,270 | **Edges**: 9,815
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 3,270 nodes, 9,815 edges; 8 labels |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | send (in=379), gun_open (in=278), do_get (in=103). 2,739 total |
|
||||
| Q3 | Find Classes | PASS | 2/5 | search_graph(Module) | No Class nodes; Module fallback: 193 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 29 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | send: 2 lines from src/cowboy_http3.erl |
|
||||
| Q6 | Text Search | PASS | 2/5 | search_code(error) | TODO/FIXME=0; "error" found 5 matches |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | send->send (self-recursive) |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 379 callers (saved to file) |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | start->compile, start->start_clear, stop->stop_listener |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Properties null |
|
||||
| Q11 | Inheritance | N/A | -- | -- | Erlang has no class inheritance |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 12 entries: src/, test/, examples/ |
|
||||
|
||||
**Score: 9.5/11 (86%)**
|
||||
|
||||
### R (tidyverse/dplyr)
|
||||
|
||||
**Project**: `dplyr-r` | **Repo**: `/tmp/lang-bench/dplyr-r`
|
||||
**Nodes**: 1,618 | **Edges**: 2,409
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,027 Functions, 13 Classes, 168 test nodes |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | arrange.data.frame, group_by, filter, mutate, select. 1,027 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | rlang_api_ptrs_t, Expander, FactorExpander. 13 total (C++ structs) |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 168 results: testthat.R, test-across.R |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | arrange.data.frame: 10 lines from R/arrange.R:88-97 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 5 matches in R/across.R about deprecation |
|
||||
| Q7 | Outbound Trace | PASS | 3/5 | trace_call_path(outbound) | C++ func found 2 edges (grouped/ungrouped) |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 3 callers: test files |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | context.R->new_environment, starwars.R->get_all/lookup |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | query_graph(properties) | Properties null |
|
||||
| Q11 | Inheritance | PASS | 1/5 | query_graph(INHERITS) | 3 rows: FactorExpander, VectorExpander, LeafExpander inherit Expander |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 21 entries: R/, src/, tests/, vignettes/ |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### Objective-C (AFNetworking/AFNetworking)
|
||||
|
||||
**Project**: `afnetworking-objc` | **Repo**: `/tmp/lang-bench/afnetworking-objc`
|
||||
**Nodes**: 1,087 | **Edges**: 1,348
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 89 files indexed |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | AFNetworkReachabilityCallback, data, destination. 114 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | AFSecurityPolicyTests, AFHTTPRequestSerializer. 80 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=Session) | 26 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | AFNetworkReachabilityCallback: 3-line function |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 1 match in AFImageDownloader.m |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(outbound) | 2 edges: Callback->PostReachabilityStatusChange->StatusForFlags |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | Callers traced |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | Function->Function edges present |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 32 variables |
|
||||
| Q11 | Inheritance | PASS | 1/5 | search_graph(Method) | 632 methods extracted |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 17 entries: AFNetworking/, Tests/, UIKit+AFNetworking/ |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### Swift (Alamofire/Alamofire)
|
||||
|
||||
**Project**: `alamofire-swift` | **Repo**: `/tmp/lang-bench/alamofire-swift`
|
||||
**Nodes**: 3,631 | **Edges**: 7,639
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 470 files indexed |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | write, read, responseStream, download. 117 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Session, Result, URLEncodedFormEncoder, AFError. 371 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=Request) | 480 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | write: 3-line function |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 matches in Session.swift, SessionDelegate.swift |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(write) | 70KB+ result: 58+ callers |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | Rich caller chain |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | didCreateURLRequest->task, finish->finish |
|
||||
| Q10 | Properties | PARTIAL | 1/5 | search_graph(Variable) | 0 Variables, 817 Fields (Swift properties as Fields) |
|
||||
| Q11 | Inheritance | PASS | 1/5 | search_graph(Module) | 449 modules |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 19 entries: Source/, Tests/, Documentation/ |
|
||||
|
||||
**Score: 10.5/11 (95%)**
|
||||
|
||||
### Dart (felangel/bloc)
|
||||
|
||||
**Project**: `bloc-dart` | **Repo**: `/tmp/lang-bench/bloc-dart`
|
||||
**Nodes**: 5,089 | **Edges**: 6,430
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 900 files indexed |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | identical, downloadFile, analyzeDependencies. 479 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Bloc, Cubit, BlocObserver, HydratedCubit, MockBloc. 846 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=Bloc) | 268 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet(Bloc) | 278-line class source |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 matches in gradle, kotlin, dart |
|
||||
| Q7 | Outbound Trace | PARTIAL | 1/5 | trace_call_path(identical) | 0 edges (built-in function reference, leaf node) |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | Callers traced |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | generateCubitCode->get, getCubitTemplate->getFreezedCubitTemplate |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 592 variables |
|
||||
| Q11 | Inheritance | PASS | 1/5 | search_graph(Method) | 873 methods, 14 INHERITS edges |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 11 entries: packages/, examples/, extensions/ |
|
||||
|
||||
**Score: 10.5/12 (87%)**
|
||||
|
||||
### Perl (mojolicious/mojo)
|
||||
|
||||
**Project**: `mojo-perl` | **Repo**: `/tmp/lang-bench/mojo-perl`
|
||||
**Nodes**: 3,287 | **Edges**: 4,182
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 192 files indexed |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | join, split, warn, decode, encode, render. 1,005 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | 0 classes (expected -- Perl uses packages) |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=render) | 26 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet(join) | 3-line sub returned |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 1 match in bundled highlight.js |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(join) | 114 results, rich 2-hop graph |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | 40+ callers: websocket, camelize, tablify |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 491 CALLS edges: run->getopt, remote_address->split |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 1,003+ variables |
|
||||
| Q11 | Inheritance | PASS | 1/5 | search_graph(Method) | 0 methods (expected -- Perl subs are Functions) |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 9 entries: lib/, t/, examples/, script/ |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### Groovy (spockframework/spock)
|
||||
|
||||
**Project**: `spock-groovy` | **Repo**: `/tmp/lang-bench/spock-groovy`
|
||||
**Nodes**: 14,081 | **Edges**: 34,557
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 1,001 files (+ 284 unlisted) |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | cleanup, setup, println. 185 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | Specification, Issue, FailsWith. 1,005 total |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=Spec) | 802 results |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | isSatisfiedBy: interface method signature |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3+ matches in BuilderHelper.java, PojoBuilder.java |
|
||||
| Q7 | Outbound Trace | PASS | 1/5 | trace_call_path(isSatisfiedBy) | 4 edges: matches, computeSimilarityScore |
|
||||
| Q8 | Inbound Trace | PASS | 1/5 | trace_call_path(inbound) | match, describeMismatch callers |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 5,876 Method->Method CALLS |
|
||||
| Q10 | Properties | PASS | 1/5 | search_graph(Variable) | 1,003+ variables (1,278 total) |
|
||||
| Q11 | Inheritance | PASS | 1/5 | search_graph(Method) | 1,003+ methods (6,788 total) |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 27 entries: spock-core/, spock-specs/, build-logic/ |
|
||||
|
||||
**Score: 12/12 (100%)**
|
||||
|
||||
### SQL (flyway/flyway)
|
||||
|
||||
**Project**: `flyway-sql` | **Repo**: `/tmp/lang-bench/flyway-sql`
|
||||
**Nodes**: 10,149 | **Edges**: 23,222
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 811 files indexed |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | 47 functions (Python/Bash helpers, not SQL functions) |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | 576 classes (Java: ClassicConfiguration, ConfigUtils) |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=Migration) | 456 results |
|
||||
| Q6 | Text Search | PARTIAL | 1/5 | search_code(CREATE TABLE) | Found in docs but no actual SQL DDL indexed |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 22 entries: flyway-core/, flyway-docker/ |
|
||||
|
||||
**Score: 4.5/6 (75%)**
|
||||
|
||||
### Dockerfile (docker-library/official-images)
|
||||
|
||||
**Project**: `official-images-dockerfile` | **Repo**: `/tmp/lang-bench/official-images-dockerfile`
|
||||
**Nodes**: 1,481 | **Edges**: 1,588
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(File) | 206 files indexed |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=docker) | 12 results |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(FROM) | 3+ matches in workflow scripts |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 17 entries: library/, test/, Dockerfile |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### CSS (animate-css/animate.css)
|
||||
|
||||
**Project**: `animate-css` | **Repo**: `/tmp/lang-bench/animate-css`
|
||||
**Nodes**: 295 | **Edges**: 214
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 117 Files, 115 Modules, 37 Variables, 3 Functions |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 0 results (no tests in CSS lib) |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 0 matches (clean codebase) |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 14 entries: source/, docs/ |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### YAML (kubernetes/examples)
|
||||
|
||||
**Project**: `k8s-yaml` | **Repo**: `/tmp/lang-bench/k8s-yaml`
|
||||
**Nodes**: 2,110 | **Edges**: 1,844
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,235 Variables, 335 Files, 309 Modules |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 12 test-related nodes |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 9 matches across demo scripts |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 12 entries: AI/, databases/, web/ |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### TOML (rust-lang/cargo)
|
||||
|
||||
**Project**: `cargo-toml` | **Repo**: `/tmp/lang-bench/cargo-toml`
|
||||
**Nodes**: 16,773 | **Edges**: 51,403
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(Variable) | 305 variables |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_code(TODO) | 3+ TODOs in workflows, Cargo.toml |
|
||||
| Q6 | Text Search | PASS | 1/5 | list_directory(src/cargo) | 8 entries: core/, lib.rs, lints/, ops/ |
|
||||
| Q12 | List Directory | PASS | 1/5 | search_graph(Module) | 1,005+ modules |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### HCL (hashicorp/terraform)
|
||||
|
||||
**Project**: `terraform-hcl` | **Repo**: `/tmp/lang-bench/terraform-hcl`
|
||||
**Nodes**: 78 | **Edges**: 76
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | search_graph(Variable) | 50 variables (version, constraints, hashes, region) |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_code(TODO) | 0 TODOs (clean config) |
|
||||
| Q6 | Text Search | PASS | 1/5 | list_directory | 6 entries: main.tf, outputs.tf, variables.tf |
|
||||
| Q12 | List Directory | PASS | 1/5 | search_graph(Module) | 5 modules: main.tf, .terraform.lock.hcl |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### HTML (twbs/bootstrap)
|
||||
|
||||
**Project**: `bootstrap-scss` (shared) | **Repo**: `/tmp/lang-bench/bootstrap-scss`
|
||||
**Nodes**: 2,726 | **Edges**: 4,135
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,525 Variables, 267 Methods, 195 Functions, 22 Classes |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 18 test-related nodes |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 TODO/FIXME matches in js/src/carousel.js |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 15 entries: scss/, js/, dist/, build/ |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
### SCSS (twbs/bootstrap)
|
||||
|
||||
**Project**: `bootstrap-scss` (shared) | **Repo**: `/tmp/lang-bench/bootstrap-scss`
|
||||
**Nodes**: 2,726 | **Edges**: 4,135
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 1,525 Variables, 267 Methods, 195 Functions, 22 Classes |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=test) | 18 SCSS test files |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(TODO) | 3 matches |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 15 entries: scss/, js/, dist/ |
|
||||
|
||||
**Score: 4/4 (100%)**
|
||||
|
||||
---
|
||||
|
||||
## Linux Kernel Stress Test
|
||||
|
||||
**Project**: `linux-kernel` | **Subset**: `drivers/net/ethernet/intel/`
|
||||
**Files**: 387 C/H files | **Nodes**: 19,993 | **Edges**: 67,305
|
||||
|
||||
### Index Breakdown
|
||||
|
||||
| Label | Count |
|
||||
|-------|------:|
|
||||
| Function | 11,546 |
|
||||
| Class (structs) | 1,851 |
|
||||
| Method | 1,803 |
|
||||
| Macro | 1,346 |
|
||||
| Field | 1,095 |
|
||||
| Community | 734 |
|
||||
| Enum | 480 |
|
||||
| Variable | 461 |
|
||||
|
||||
### Results
|
||||
|
||||
| Q# | Question | Grade | Attempts | Approach | Notes |
|
||||
|----|----------|-------|----------|----------|-------|
|
||||
| Q1 | Index Stats | PASS | 1/5 | get_graph_schema | 14 node labels, 11 edge types, 20K nodes, 67K edges |
|
||||
| Q2 | Find Functions | PASS | 1/5 | search_graph(Function) | e1e_rphy (in=64), e1e_wphy (in=53). 11,546 total |
|
||||
| Q3 | Find Classes | PASS | 1/5 | search_graph(Class) | nic (in=86), params (in=59). 1,851 C structs |
|
||||
| Q4 | Pattern Search | PASS | 1/5 | search_graph(name_pattern=init\|probe) | 715 matches: i40e_probe (out=59), ixgbe_probe (out=43) |
|
||||
| Q5 | Code Snippet | PASS | 1/5 | get_code_snippet | e1e_rphy: 4 lines from e1000.h:543-546 |
|
||||
| Q6 | Text Search | PASS | 1/5 | search_code(BUG_ON\|WARN_ON\|pr_err) | Found across amt.c, e100.c, e1000_ethtool.c |
|
||||
| Q7 | Outbound Trace (depth=5) | PASS | 1/5 | trace_call_path(outbound) | i40e_probe: 129K chars result. Massive call tree, no timeout |
|
||||
| Q8 | Inbound Trace (depth=5) | PASS | 1/5 | trace_call_path(inbound) | e1e_rphy: 67.6K chars. 64 direct callers, deep chains |
|
||||
| Q9 | Cypher CALLS | PASS | 1/5 | query_graph | 10 CALLS edges: netdev_boot_setup, amt functions |
|
||||
| Q10 | Properties | PARTIAL | 2/5 | query_graph(Cypher) | `IS NOT NULL` not supported; retried with adjusted query |
|
||||
| Q11 | Inheritance | N/A | -- | -- | C language |
|
||||
| Q12 | List Directory | PASS | 1/5 | list_directory | 17 entries: e100.c, i40e/, ice/, ixgbe/, igb/ |
|
||||
|
||||
**Score: 11/11 (100%)** (Q11 N/A)
|
||||
|
||||
### Stress Metrics
|
||||
|
||||
| Test | Result | Size | Timeout? |
|
||||
|------|--------|-----:|----------|
|
||||
| Deep trace (depth=5, outbound, i40e_probe) | PASS | 129,026 chars | No |
|
||||
| Deep trace (depth=5, inbound, e1e_rphy) | PASS | 67,600 chars | No |
|
||||
| Deep trace (depth=5, both, e1000e_reset) | PASS | moderate | No |
|
||||
| Multi-hop Cypher (3-hop CALLS chain) | PASS | 10 valid chains | No |
|
||||
|
||||
The tool handles 20K-node kernel subsystems without timeouts. Deep traces on well-connected probe functions (out_degree=59) produce massive but complete results. Community detection identified 734 clusters -- useful for understanding driver module boundaries.
|
||||
|
||||
---
|
||||
|
||||
## Cross-Cutting Findings
|
||||
|
||||
### Strengths
|
||||
1. **Zero indexing failures** across 63 repos of all sizes (78 to 49K nodes)
|
||||
2. **100% on 17 languages** -- half the languages achieved perfect scores
|
||||
3. **No language below 62%** -- even the weakest performs core operations
|
||||
4. **Massive codebases handled**: Django (49K nodes), Laravel (38K nodes), neovim (24K nodes) -- no performance issues
|
||||
5. **Kernel-scale stress test passed**: 20K nodes, 67K edges, 129K-char traces, zero timeouts
|
||||
6. **"Removed" languages still strong**: Obj-C (100%), Perl (100%), Groovy (100%), Swift (95%), Dart (87%)
|
||||
|
||||
### Common PARTIAL Patterns
|
||||
- **Q10 (Properties)**: Most languages return functions with `properties=null`. Parameter/return extraction is incomplete -- this is the single most common deduction
|
||||
- **Q7/Q8 (Trace)**: Abstract methods, entry-point functions, and built-in references correctly return 0 edges (no false positives), but are graded PARTIAL when no alternative function has edges
|
||||
- **Q11 (Inheritance)**: Languages using alternative paradigms (traits, protocols, typeclasses, modules) have limited INHERITS edges
|
||||
|
||||
### Known Limitations
|
||||
1. **Function properties**: Parameter types and return types are not extracted for most languages, resulting in `properties=null`
|
||||
2. **Haskell call tracing**: Function composition (`f . g . h`) is not modeled as CALLS edges. Only explicit function applications are traced
|
||||
3. **OCaml call analysis**: Limited call resolution due to module functor indirection
|
||||
4. **Cypher `IS NOT NULL`**: Not supported in the Cypher parser -- use alternative query patterns
|
||||
5. **Scala/Ruby trace**: Abstract methods and block-based patterns can have 0 trace edges
|
||||
6. **query_graph 200-row cap**: Aggregation queries (COUNT) silently undercount on large codebases
|
||||
|
||||
---
|
||||
|
||||
## Aggregate Statistics
|
||||
|
||||
| Metric | Value |
|
||||
|--------|------:|
|
||||
| Languages tested | 35 |
|
||||
| Total questions asked | 370 |
|
||||
| PASS | 327 (88%) |
|
||||
| PARTIAL | 25 (7%) |
|
||||
| FAIL | 18 (5%) |
|
||||
| N/A (excluded) | 41 |
|
||||
| Weighted score | 339.5 / 370 |
|
||||
| Overall percentage | **91.8%** |
|
||||
| Perfect scores (100%) | 17 languages |
|
||||
| Tier 1 (>=90%) | 17 languages |
|
||||
| Tier 2 (75-89%) | 16 languages |
|
||||
| Tier 3 (<75%) | 2 languages |
|
||||
@@ -0,0 +1,130 @@
|
||||
# Configuration Reference
|
||||
|
||||
This page documents the configuration files that `codebase-memory-mcp` reads or writes today.
|
||||
|
||||
## At a Glance
|
||||
|
||||
| Purpose | Path | Format | Notes |
|
||||
|---|---|---|---|
|
||||
| Global custom extension mapping | `$XDG_CONFIG_HOME/codebase-memory-mcp/config.json` | JSON | Falls back to `~/.config/codebase-memory-mcp/config.json` when `XDG_CONFIG_HOME` is unset. |
|
||||
| Per-project custom extension mapping | `{repo_root}/.codebase-memory.json` | JSON | Overrides conflicting global `extra_extensions` entries. |
|
||||
| CLI-managed runtime settings | `${CBM_CACHE_DIR:-~/.cache/codebase-memory-mcp}/_config.db` | SQLite | Written by `codebase-memory-mcp config set/reset`. |
|
||||
| UI settings | `${CBM_CACHE_DIR:-~/.cache/codebase-memory-mcp}/config.json` | JSON | Stores `ui_enabled` and `ui_port`. |
|
||||
|
||||
## 1. Custom File Extension Mapping
|
||||
|
||||
Two optional JSON files let you map additional file extensions to built-in languages.
|
||||
|
||||
### Global config
|
||||
|
||||
Default path:
|
||||
|
||||
```text
|
||||
$XDG_CONFIG_HOME/codebase-memory-mcp/config.json
|
||||
```
|
||||
|
||||
Fallback when `XDG_CONFIG_HOME` is unset:
|
||||
|
||||
```text
|
||||
~/.config/codebase-memory-mcp/config.json
|
||||
```
|
||||
|
||||
### Per-project config
|
||||
|
||||
Place this file in the repository root:
|
||||
|
||||
```text
|
||||
.codebase-memory.json
|
||||
```
|
||||
|
||||
### Format
|
||||
|
||||
```json
|
||||
{
|
||||
"extra_extensions": {
|
||||
".blade.php": "php",
|
||||
".mjs": "javascript",
|
||||
".twig": "html"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- Extension keys must include the leading dot.
|
||||
- Language names are case-insensitive.
|
||||
- Unknown language names are skipped.
|
||||
- Missing files are ignored.
|
||||
- If the same extension appears in both files, the per-project file wins.
|
||||
|
||||
## 2. CLI-Managed Runtime Settings
|
||||
|
||||
The `config` subcommand stores runtime settings in a small SQLite database:
|
||||
|
||||
```text
|
||||
${CBM_CACHE_DIR:-~/.cache/codebase-memory-mcp}/_config.db
|
||||
```
|
||||
|
||||
Inspect or change values with the CLI:
|
||||
|
||||
```bash
|
||||
codebase-memory-mcp config list
|
||||
codebase-memory-mcp config get auto_index
|
||||
codebase-memory-mcp config set auto_index true
|
||||
codebase-memory-mcp config set auto_index_limit 50000
|
||||
codebase-memory-mcp config reset auto_index
|
||||
```
|
||||
|
||||
Current keys:
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `auto_index` | `false` | Automatically index new projects when an MCP session starts. |
|
||||
| `auto_index_limit` | `50000` | Maximum file count allowed for automatic indexing of a new project. |
|
||||
|
||||
## 3. UI Settings
|
||||
|
||||
The optional built-in graph UI stores its settings in:
|
||||
|
||||
```text
|
||||
${CBM_CACHE_DIR:-~/.cache/codebase-memory-mcp}/config.json
|
||||
```
|
||||
|
||||
Current format:
|
||||
|
||||
```json
|
||||
{
|
||||
"ui_enabled": false,
|
||||
"ui_port": 9749
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- If the UI-enabled binary has embedded assets and no UI config file exists yet, the UI auto-enables on first run.
|
||||
- `CBM_CACHE_DIR` changes both the UI config location and the runtime settings database location.
|
||||
|
||||
## 4. Environment Variables
|
||||
|
||||
These environment variables affect runtime behavior:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `CBM_ALLOWED_ROOT` | *(unset)* | Restrict `index_repository` to paths within this directory. When set, a `repo_path` that resolves (after symlink / `..` resolution) outside this root is refused; unset imposes no restriction. Useful when the server may be driven by an untrusted caller (agentic or multi-tenant deployments). |
|
||||
| `CBM_CACHE_DIR` | `~/.cache/codebase-memory-mcp` | Override the cache directory used for indexes, `_config.db`, and UI `config.json`. |
|
||||
| `CBM_DIAGNOSTICS` | `false` | Enable periodic diagnostics output to `/tmp/cbm-diagnostics-<pid>.json`. |
|
||||
| `CBM_DOWNLOAD_URL` | GitHub releases | Override the update download URL. |
|
||||
| `CBM_LOG_LEVEL` | `info` | Set stderr log level to `debug`, `info`, `warn`, `error`, or `none` (or `0`-`4`). |
|
||||
| `CBM_WORKERS` | auto-detected | Override the indexing worker count. |
|
||||
|
||||
## 5. Agent and Editor Integration Files
|
||||
|
||||
The `install` command can also write MCP entries and instruction blocks into agent/editor config files such as Claude Code, Codex, Gemini, VS Code, Cursor, Zed, and others.
|
||||
|
||||
Those target paths vary by tool and platform, so the easiest way to inspect the exact files for your machine is:
|
||||
|
||||
```bash
|
||||
codebase-memory-mcp install --dry-run
|
||||
```
|
||||
|
||||
That prints the specific config files the installer would modify without writing anything.
|
||||
+11118
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,87 @@
|
||||
# Security Disclosure & Handling Process
|
||||
|
||||
This document explains **how security reports are handled** for
|
||||
codebase-memory-mcp — what happens after you report a vulnerability, what you
|
||||
can expect from us, and how disclosure and credit work.
|
||||
|
||||
For **how to report** a vulnerability and **what is in scope**, see
|
||||
[`SECURITY.md`](../SECURITY.md). This document covers the process *after* a
|
||||
report arrives.
|
||||
|
||||
> **This is a solo, volunteer-maintained project.** Everything below is handled
|
||||
> on a good-faith, best-effort basis. The timeframes are honest targets we aim
|
||||
> to beat — not contractual guarantees. If something will take longer, we will
|
||||
> tell you and keep you updated rather than go silent.
|
||||
|
||||
## Principles
|
||||
|
||||
We follow **coordinated disclosure**:
|
||||
|
||||
1. **Fix privately, disclose publicly.** Details of an unfixed vulnerability are
|
||||
never discussed in the open. We develop and validate the fix in private, ship
|
||||
a release, and only then disclose.
|
||||
2. **Patch before publicity.** A fixed release is always available *before* the
|
||||
vulnerability is described publicly, so users can update immediately.
|
||||
3. **Credit the researcher.** Public credit by default; anonymity on request.
|
||||
4. **A bug fixed once should stay fixed.** Every fix ships with a regression
|
||||
test or guard so the same class of issue cannot silently return.
|
||||
|
||||
## What happens after you report
|
||||
|
||||
| Step | What we do | Target (best-effort) |
|
||||
|------|------------|----------------------|
|
||||
| 1. **Acknowledge** | Confirm we received your report and are looking at it. | within **7 days** (usually much sooner) |
|
||||
| 2. **Triage & severity** | Reproduce the issue and assign a severity (CVSS). | within **14 days** |
|
||||
| 3. **Fix privately** | Develop the fix in a private environment, with a regression guard, and validate it across all supported platforms (Linux, macOS, Windows) under full CI. | severity-dependent |
|
||||
| 4. **You verify** | We invite you (read-only) to confirm the fix resolves the issue and that the guard prevents regression. Your sign-off is welcomed; an unresponsive reporter will not indefinitely block a release. | — |
|
||||
| 5. **Release** | Merge the fix and cut a patched release promptly. | as fast as severity warrants |
|
||||
| 6. **Disclose** | Publish a [GitHub Security Advisory](https://github.com/DeusData/codebase-memory-mcp/security/advisories), request a **CVE** (GitHub is a CNA), and credit you. | after a short upgrade window |
|
||||
|
||||
**Overall fix timeline:** we aim to resolve and release a fix within **90 days**
|
||||
of triage, and much faster for high-severity issues. Critical, actively
|
||||
exploitable issues are handled with the highest priority.
|
||||
|
||||
## Severity
|
||||
|
||||
We assess severity using **CVSS** and prioritise accordingly. Roughly:
|
||||
|
||||
- **Critical / High** — remote code execution, sandbox/scope escape, supply-chain
|
||||
compromise. Prioritised; expedited release.
|
||||
- **Medium** — issues requiring local access, non-default configuration, or
|
||||
significant user interaction.
|
||||
- **Low** — defense-in-depth gaps, hardening, information exposure with limited
|
||||
impact.
|
||||
|
||||
## Credit & CVE
|
||||
|
||||
- You are **credited by name/handle** in the published advisory unless you ask to
|
||||
remain anonymous.
|
||||
- A **CVE identifier** is requested for each distinct vulnerability via the
|
||||
GitHub Security Advisory (one CVE per vulnerability, not per report — a single
|
||||
report may yield several).
|
||||
- The advisory lists the **affected and patched version ranges** so downstream
|
||||
tooling (e.g. Dependabot) can alert users automatically.
|
||||
|
||||
## Safe harbor
|
||||
|
||||
We will not pursue or support legal action against researchers who act in
|
||||
**good faith**, meaning you:
|
||||
|
||||
- only access, modify, or store data in **your own test environment**;
|
||||
- avoid privacy violations, data destruction, and degradation of service for
|
||||
others;
|
||||
- give us a **reasonable opportunity to fix** the issue before disclosing it
|
||||
publicly;
|
||||
- do not exploit the issue beyond the minimum necessary to demonstrate it.
|
||||
|
||||
Good-faith research conducted under this policy is considered authorised, and we
|
||||
will work with you, not against you.
|
||||
|
||||
## What we ask of you
|
||||
|
||||
- Report **privately** (see [`SECURITY.md`](../SECURITY.md)) — not as a public
|
||||
issue, PR, or social-media post.
|
||||
- Give us **reasonable time** to fix before any public write-up.
|
||||
- Provide enough detail to **reproduce** (affected version, steps, impact).
|
||||
|
||||
Thank you for helping keep a tool used by developers worldwide safe. 🙏
|
||||
@@ -0,0 +1,121 @@
|
||||
# `.cbmignore` — Excluding Files from Indexing
|
||||
|
||||
`.cbmignore` is a project-specific ignore file that controls which files the
|
||||
indexer sees. It uses gitignore-style syntax and is read from the **root of
|
||||
the indexed directory** (`<repo>/.cbmignore`). Nested `.cbmignore` files in
|
||||
subdirectories are not read.
|
||||
|
||||
It applies at **file discovery time** — the directory walk that selects files
|
||||
for parsing. Every indexing path uses the same discovery: the initial
|
||||
`index_repository`, manual re-indexing, and background auto-sync. A path
|
||||
matched by `.cbmignore` never enters the graph. Changes to `.cbmignore` take
|
||||
effect on the next (re-)index.
|
||||
|
||||
Unlike `.gitignore`, it has no effect on git itself — it only shapes what the
|
||||
indexer sees. Commit it to share indexing excludes with your team, or list it
|
||||
in `.gitignore` to keep personal excludes untracked.
|
||||
|
||||
To verify it works: directory subtrees skipped during discovery are reported
|
||||
in the `index_repository` response under `excluded`
|
||||
(`{"dirs": [up to 25 paths], "count": <total>, "truncated": <bool>}`).
|
||||
|
||||
## Syntax
|
||||
|
||||
One pattern per line. Blank lines are ignored, lines starting with `#` are
|
||||
comments, and trailing whitespace is trimmed.
|
||||
|
||||
| Feature | Meaning |
|
||||
|---|---|
|
||||
| `*` | matches any run of characters, except `/` |
|
||||
| `?` | matches exactly one character, except `/` |
|
||||
| `**` | matches across directory boundaries (`**/name`, `dir/**`, `a/**/b`) |
|
||||
| `[abc]`, `[a-z]` | character classes; `[!a-z]` / `[^a-z]` negate the class |
|
||||
| trailing `/` | pattern matches **directories only** |
|
||||
| `/` anywhere else | anchors the pattern to the repo root |
|
||||
| no `/` in pattern | matches the file/directory name at **any depth** |
|
||||
| leading `!` | negation — re-includes a previously matched path; the **last matching pattern wins** |
|
||||
|
||||
Examples:
|
||||
|
||||
```gitignore
|
||||
# Generated protobuf output, anywhere in the tree
|
||||
*.pb.go
|
||||
|
||||
# A specific top-level directory (leading / anchors to the repo root)
|
||||
/third_party/
|
||||
|
||||
# Any directory named "snapshots", at any depth (trailing / = directories only)
|
||||
snapshots/
|
||||
|
||||
# Everything under any fixtures directory
|
||||
**/fixtures/**
|
||||
|
||||
# Anchored glob: generated clients for any single-character API version
|
||||
/api/v?/generated/
|
||||
|
||||
# Character class: yearly log folders 2020-2029
|
||||
/logs/202[0-9]/
|
||||
|
||||
# Ignore all YAML, but keep CI configs (negation — last match wins)
|
||||
*.yaml
|
||||
!ci.yaml
|
||||
```
|
||||
|
||||
## Precedence
|
||||
|
||||
Discovery applies its filters in a fixed order — the first layer that rejects
|
||||
a path wins. For directories:
|
||||
|
||||
1. **Built-in skip list** — `.git`, `node_modules`, `dist`, `target`,
|
||||
`vendor`, tool caches, etc. (60+ names; the fast/moderate index modes add
|
||||
more, e.g. `docs`, `examples`, `testdata`). Not overridable from any
|
||||
ignore file today.
|
||||
2. **Repo `.gitignore`** — `<repo>/.gitignore` merged with
|
||||
`<git-common-dir>/info/exclude` (worktree-aware); later patterns win on
|
||||
conflict. Honored even when the indexed directory is not a git repo root.
|
||||
3. **Nested `.gitignore` files** — picked up during the walk and matched
|
||||
relative to their own directory.
|
||||
4. **`.cbmignore`** — a positive match skips the path; a negated match can
|
||||
only rescue paths from layer 5.
|
||||
5. **Git global excludes** — `core.excludesFile` from `~/.gitconfig` or the
|
||||
XDG git config (default `$XDG_CONFIG_HOME/git/ignore`); consulted only
|
||||
when the project is a git repo with a config.
|
||||
|
||||
For files, built-in suffix filters (`.png`, `.o`, `.db`, …; fast modes add
|
||||
archives, media, lockfiles, `.min.js`, …) and fast-mode filename/substring
|
||||
filters run **before** the ignore files, and a maximum-file-size cap runs
|
||||
after them; none of these are overridable from `.cbmignore`. Symlinks are
|
||||
always skipped.
|
||||
|
||||
## Negation (`!`) — current behavior
|
||||
|
||||
- **Within `.cbmignore`**: standard gitignore semantics. Patterns are
|
||||
evaluated top to bottom and the last matching pattern wins, so
|
||||
`!pattern` re-includes something an earlier line excluded.
|
||||
- **Parent pruning** (same caveat as git): when a directory is excluded, the
|
||||
walk never descends into it — you cannot re-include a file whose parent
|
||||
directory is excluded. Negate the directory itself if you need its
|
||||
contents.
|
||||
- **Across layers**: a `.cbmignore` negation overrides the **git global
|
||||
excludes** layer only. Example: your `~/.config/git/ignore` ignores
|
||||
`*.sql`, but this project's SQL should be indexed — add `!*.sql` to
|
||||
`.cbmignore`. Negation cannot override the built-in skip lists, the repo
|
||||
`.gitignore`/`info/exclude`, nested `.gitignore` files, the built-in
|
||||
suffix/filename filters, or the size cap.
|
||||
|
||||
### Planned (not yet implemented)
|
||||
|
||||
The negation story is being unified; none of the following works yet:
|
||||
|
||||
- `!` in `.cbmignore` will be able to un-skip ordinary built-in skip
|
||||
directories (`obj/`, `dist/`, `target/`, …) so build-output-like
|
||||
directories that actually contain source can be indexed.
|
||||
- A small safety core stays non-negatable by design — `.git`,
|
||||
`node_modules`, and worktree-internal directories — because indexing them
|
||||
risks OOM and correctness issues (see issue #489).
|
||||
- Auxiliary filesystem walkers will honor the same ignore predicate as
|
||||
discovery, so every code path sees an identical ignore decision
|
||||
(unification tracked in a follow-up issue).
|
||||
|
||||
Until these land, the "Precedence" and "Negation — current behavior" sections
|
||||
above describe the actual behavior.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.2 MiB |
+880
@@ -0,0 +1,880 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>codebase-memory-mcp — Code Intelligence Knowledge Graph for AI Coding Agents</title>
|
||||
<meta name="description" content="codebase-memory-mcp is an open-source MCP server that indexes any codebase into a persistent knowledge graph so AI coding agents answer structural questions with ~120x fewer tokens. 158 languages, Hybrid LSP type resolution, local semantic vector search, code-clone detection, sub-1ms queries, Linux kernel indexed in 3 minutes. Single static C binary, zero dependencies. Works with 11 agents including Claude Code, Codex CLI, Gemini CLI, Cursor, and Zed.">
|
||||
<meta name="keywords" content="MCP server, code intelligence, knowledge graph, tree-sitter, Hybrid LSP, semantic code search, code embeddings, code clone detection, cross-repo analysis, data-flow analysis, Claude Code, Codex CLI, Gemini CLI, Cursor, Zed, code exploration, token reduction, call graph, dead code detection">
|
||||
<meta name="author" content="DeusData">
|
||||
<meta name="robots" content="index, follow, max-image-preview:large, max-snippet:-1">
|
||||
<meta name="theme-color" content="#0d1117">
|
||||
|
||||
<!-- Open Graph -->
|
||||
<meta property="og:title" content="codebase-memory-mcp — Code Intelligence Knowledge Graph for AI Coding Agents">
|
||||
<meta property="og:description" content="Index any codebase into a persistent knowledge graph. AI agents answer structural questions with ~120x fewer tokens. 158 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:url" content="https://deusdata.github.io/codebase-memory-mcp/">
|
||||
<meta property="og:image" content="https://deusdata.github.io/codebase-memory-mcp/graph-ui-screenshot.png">
|
||||
<meta property="og:image:width" content="1538">
|
||||
<meta property="og:image:height" content="932">
|
||||
<meta property="og:image:alt" content="3D knowledge-graph visualization of a codebase with thousands of nodes and edges">
|
||||
<meta property="og:site_name" content="codebase-memory-mcp">
|
||||
|
||||
<!-- Twitter Card -->
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="codebase-memory-mcp — Code Intelligence for AI Coding Agents">
|
||||
<meta name="twitter:description" content="Index any codebase into a knowledge graph. ~120x fewer tokens for AI code exploration. 158 languages, Hybrid LSP, sub-1ms queries, single static C binary.">
|
||||
<meta name="twitter:image" content="https://deusdata.github.io/codebase-memory-mcp/graph-ui-screenshot.png">
|
||||
|
||||
<link rel="canonical" href="https://deusdata.github.io/codebase-memory-mcp/">
|
||||
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230d1117'/%3E%3Cg stroke='%2358a6ff' stroke-width='2'%3E%3Cline x1='9' y1='9' x2='23' y2='16'/%3E%3Cline x1='23' y1='16' x2='10' y2='24'/%3E%3Cline x1='9' y1='9' x2='10' y2='24'/%3E%3C/g%3E%3Cg fill='%2358a6ff'%3E%3Ccircle cx='9' cy='9' r='3.5'/%3E%3Ccircle cx='23' cy='16' r='3.5'/%3E%3Ccircle cx='10' cy='24' r='3.5'/%3E%3C/g%3E%3C/svg%3E">
|
||||
|
||||
<!-- Structured data: SoftwareApplication (the tool) -->
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "SoftwareApplication",
|
||||
"name": "codebase-memory-mcp",
|
||||
"alternateName": "Codebase Memory MCP",
|
||||
"applicationCategory": "DeveloperApplication",
|
||||
"applicationSubCategory": "Model Context Protocol (MCP) server",
|
||||
"operatingSystem": "macOS, Linux, Windows",
|
||||
"softwareVersion": "0.8.1",
|
||||
"description": "An open-source MCP server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links, so AI coding agents answer structural questions with roughly 120x fewer tokens than file-by-file search. Parses 158 languages via tree-sitter with Hybrid LSP semantic type resolution for 9 language families. Ships as a single static C binary with zero runtime dependencies.",
|
||||
"url": "https://deusdata.github.io/codebase-memory-mcp/",
|
||||
"downloadUrl": "https://github.com/DeusData/codebase-memory-mcp/releases/latest",
|
||||
"codeRepository": "https://github.com/DeusData/codebase-memory-mcp",
|
||||
"programmingLanguage": "C",
|
||||
"license": "https://opensource.org/licenses/MIT",
|
||||
"isAccessibleForFree": true,
|
||||
"offers": {
|
||||
"@type": "Offer",
|
||||
"price": "0",
|
||||
"priceCurrency": "USD"
|
||||
},
|
||||
"featureList": [
|
||||
"Indexes 158 programming languages via vendored tree-sitter grammars",
|
||||
"Hybrid LSP semantic type resolution for Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust",
|
||||
"14 MCP tools for structural search, call-path tracing, and Cypher graph queries",
|
||||
"Semantic vector code search via bundled nomic-embed-code embeddings (no API key, fully local)",
|
||||
"Semantic graph edges (SEMANTICALLY_RELATED) and near-clone detection (SIMILAR_TO, MinHash + LSH)",
|
||||
"Cross-service linking for HTTP, gRPC, GraphQL, tRPC, and pub/sub channels with confidence scoring",
|
||||
"Cross-repo intelligence with CROSS_* edges across multiple indexed repositories",
|
||||
"Data-flow tracing with argument-to-parameter mapping",
|
||||
"Change-impact analysis mapping git diffs to affected symbols and blast radius",
|
||||
"Architecture Decision Record (ADR) persistence across sessions",
|
||||
"Dead-code detection with entry-point filtering",
|
||||
"Infrastructure-as-code indexing for Dockerfiles, Kubernetes, and Kustomize",
|
||||
"Built-in 3D graph visualization UI",
|
||||
"Auto-sync background watcher for incremental re-indexing",
|
||||
"One-command install for 11 AI coding agents"
|
||||
],
|
||||
"author": {
|
||||
"@type": "Organization",
|
||||
"name": "DeusData",
|
||||
"url": "https://github.com/DeusData"
|
||||
},
|
||||
"citation": {
|
||||
"@type": "ScholarlyArticle",
|
||||
"name": "Codebase-Memory: Tree-Sitter-Based Knowledge Graphs for LLM Code Exploration via MCP",
|
||||
"identifier": "arXiv:2603.27277",
|
||||
"url": "https://arxiv.org/abs/2603.27277"
|
||||
}
|
||||
}
|
||||
</script>
|
||||
|
||||
<!-- Structured data: Organization + WebSite identity -->
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@graph": [
|
||||
{
|
||||
"@type": "Organization",
|
||||
"@id": "https://deusdata.github.io/codebase-memory-mcp/#org",
|
||||
"name": "DeusData",
|
||||
"url": "https://github.com/DeusData",
|
||||
"sameAs": ["https://github.com/DeusData/codebase-memory-mcp"]
|
||||
},
|
||||
{
|
||||
"@type": "WebSite",
|
||||
"@id": "https://deusdata.github.io/codebase-memory-mcp/#website",
|
||||
"url": "https://deusdata.github.io/codebase-memory-mcp/",
|
||||
"name": "codebase-memory-mcp",
|
||||
"publisher": { "@id": "https://deusdata.github.io/codebase-memory-mcp/#org" }
|
||||
},
|
||||
{
|
||||
"@type": "BreadcrumbList",
|
||||
"itemListElement": [
|
||||
{ "@type": "ListItem", "position": 1, "name": "Home", "item": "https://deusdata.github.io/codebase-memory-mcp/" },
|
||||
{ "@type": "ListItem", "position": 2, "name": "GitHub", "item": "https://github.com/DeusData/codebase-memory-mcp" }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
</script>
|
||||
|
||||
<!-- Structured data: FAQPage (machine-readable Q&A for answer engines) -->
|
||||
<script type="application/ld+json">
|
||||
{
|
||||
"@context": "https://schema.org",
|
||||
"@type": "FAQPage",
|
||||
"mainEntity": [
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "What is codebase-memory-mcp?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "codebase-memory-mcp is an open-source Model Context Protocol (MCP) server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. AI coding agents query that graph instead of reading files one by one, answering structural questions with roughly 120x fewer tokens. It parses 158 languages and ships as a single static C binary with no runtime dependencies."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "How does codebase-memory-mcp reduce token usage?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "AI agents normally explore code through repeated grep-then-read cycles, which burn large numbers of tokens. codebase-memory-mcp answers the same structural questions from a precomputed graph. In a five-query benchmark it used about 3,400 tokens versus about 412,000 tokens for file-by-file search — a roughly 120x reduction."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "Which programming languages does codebase-memory-mcp support?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "It supports 158 languages through vendored tree-sitter grammars compiled into the binary, including Python, Go, JavaScript, TypeScript, Rust, Java, C, C++, C#, PHP, Ruby, Kotlin, Swift, and many more. Nine language families — Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust — additionally get Hybrid LSP semantic type resolution."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "What is Hybrid LSP?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "Hybrid LSP is a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers (tsserver, pyright, gopls, Roslyn), embedded directly into the binary. It runs alongside tree-sitter to resolve imports, generics, inheritance, and stdlib types — so call edges in the graph mirror what an IDE 'Go to Definition' would resolve, with no language-server process or per-project setup."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "Which AI coding agents work with codebase-memory-mcp?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "A single install command configures 11 agents: Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, and Kiro. Any MCP-compatible client can use the server."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "Does codebase-memory-mcp send my code anywhere?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "No. All indexing and querying happen 100% locally on your machine. There is no embedded LLM and no API key — your MCP client acts as the intelligence layer. Release binaries are signed, checksummed, and scanned by 70+ antivirus engines."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "Does codebase-memory-mcp support semantic or natural-language code search?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "Yes. Alongside structural and BM25 full-text search, it offers semantic vector search over the whole graph via the search_graph tool's semantic_query parameter. It is powered by nomic-embed-code embeddings compiled directly into the binary (768-dimensional), so it bridges vocabulary gaps — finding 'publish' when you search 'send' — with no API key, no Ollama, and no Docker. The indexer also generates SEMANTICALLY_RELATED edges between conceptually similar functions and SIMILAR_TO edges for near-duplicate and clone detection."
|
||||
}
|
||||
},
|
||||
{
|
||||
"@type": "Question",
|
||||
"name": "Can codebase-memory-mcp detect duplicate or near-clone code?",
|
||||
"acceptedAnswer": {
|
||||
"@type": "Answer",
|
||||
"text": "Yes. During indexing it builds SIMILAR_TO edges using MinHash plus LSH with Jaccard scoring, surfacing near-duplicate and copy-pasted functions across the codebase. Combined with SEMANTICALLY_RELATED edges, this makes refactoring candidates and redundant implementations queryable through the graph."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
</script>
|
||||
|
||||
<style>
|
||||
:root {
|
||||
--bg: #0d1117;
|
||||
--surface: #161b22;
|
||||
--border: #30363d;
|
||||
--text: #e6edf3;
|
||||
--text-secondary: #8b949e;
|
||||
--accent: #58a6ff;
|
||||
--accent-hover: #79c0ff;
|
||||
--green: #3fb950;
|
||||
--red: #f85149;
|
||||
}
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Helvetica, Arial, sans-serif;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
line-height: 1.6;
|
||||
}
|
||||
a { color: var(--accent); }
|
||||
.container { max-width: 960px; margin: 0 auto; padding: 0 24px; }
|
||||
.muted { color: var(--text-secondary); }
|
||||
cite { color: var(--text-secondary); font-style: normal; font-size: 0.8rem; }
|
||||
cite a { color: var(--text-secondary); }
|
||||
|
||||
/* Nav */
|
||||
nav {
|
||||
border-bottom: 1px solid var(--border);
|
||||
position: sticky;
|
||||
top: 0;
|
||||
background: rgba(13,17,23,0.85);
|
||||
backdrop-filter: blur(8px);
|
||||
z-index: 10;
|
||||
}
|
||||
nav .container {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
height: 56px;
|
||||
}
|
||||
nav .links a {
|
||||
text-decoration: none;
|
||||
color: var(--text-secondary);
|
||||
margin-left: 20px;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
nav .links a:hover { color: var(--text); }
|
||||
nav .links a:first-child { margin-left: 0; }
|
||||
nav .links a.nav-star { color: var(--green); font-weight: 600; }
|
||||
nav .links a.nav-star:hover { color: var(--accent-hover); }
|
||||
@media (max-width: 640px) { nav .links a:not(.nav-cta) { display: none; } }
|
||||
|
||||
/* Hero */
|
||||
.hero { padding: 72px 0 32px; text-align: center; }
|
||||
.hero h1 { font-size: 2.5rem; font-weight: 700; margin-bottom: 4px; }
|
||||
.hero .by-line { color: #6e7681; font-size: 0.95rem; margin-bottom: 24px; }
|
||||
.hero .tagline {
|
||||
font-size: 1.2rem;
|
||||
color: var(--text-secondary);
|
||||
margin: 0 auto 32px;
|
||||
max-width: 720px;
|
||||
}
|
||||
.hero .stat {
|
||||
display: inline-block;
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 16px 24px;
|
||||
margin: 8px;
|
||||
}
|
||||
.hero .stat .number { font-size: 2rem; font-weight: 700; color: var(--accent); }
|
||||
.hero .stat .label { font-size: 0.875rem; color: var(--text-secondary); }
|
||||
.cta-buttons { margin-top: 32px; }
|
||||
.cta-buttons a {
|
||||
display: inline-block;
|
||||
padding: 12px 24px;
|
||||
border-radius: 6px;
|
||||
text-decoration: none;
|
||||
font-weight: 600;
|
||||
margin: 4px;
|
||||
}
|
||||
.cta-primary { background: var(--accent); color: #0d1117; }
|
||||
.cta-primary:hover { background: var(--accent-hover); }
|
||||
.cta-secondary { border: 1px solid var(--border); color: var(--text); }
|
||||
.cta-secondary:hover { border-color: var(--text-secondary); }
|
||||
|
||||
/* Screenshot */
|
||||
.screenshot { margin: 40px 0; text-align: center; }
|
||||
.screenshot img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
border-radius: 12px;
|
||||
border: 1px solid var(--border);
|
||||
box-shadow: 0 8px 32px rgba(0,0,0,0.4);
|
||||
}
|
||||
.screenshot .caption { margin-top: 12px; font-size: 0.875rem; color: var(--text-secondary); }
|
||||
|
||||
/* Research callout */
|
||||
.research {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 20px;
|
||||
flex-wrap: wrap;
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-left: 3px solid var(--accent);
|
||||
border-radius: 8px;
|
||||
padding: 20px 24px;
|
||||
margin: 8px 0 0;
|
||||
}
|
||||
.research .arxiv-badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
flex-shrink: 0;
|
||||
font-family: 'SF Mono', monospace;
|
||||
font-size: 0.8rem;
|
||||
font-weight: 700;
|
||||
color: #b31b1b;
|
||||
background: #fff;
|
||||
border-radius: 6px;
|
||||
padding: 8px 12px;
|
||||
text-decoration: none;
|
||||
}
|
||||
.research p { font-size: 0.92rem; margin: 0; }
|
||||
.research .label { color: var(--accent); font-weight: 700; }
|
||||
|
||||
/* Sections */
|
||||
section { padding: 56px 0; border-top: 1px solid var(--border); }
|
||||
section h2 { font-size: 1.75rem; margin-bottom: 8px; scroll-margin-top: 72px; }
|
||||
section .lead { font-size: 1.05rem; margin-bottom: 24px; }
|
||||
section h3 { font-size: 1.1rem; margin: 24px 0 8px; }
|
||||
|
||||
/* Code / install */
|
||||
.install-block {
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 20px 24px;
|
||||
font-family: 'SF Mono', 'Fira Code', monospace;
|
||||
font-size: 0.9rem;
|
||||
overflow-x: auto;
|
||||
margin: 16px 0;
|
||||
}
|
||||
.install-block .comment { color: var(--text-secondary); }
|
||||
.install-block .cmd { color: var(--green); }
|
||||
code {
|
||||
font-family: 'SF Mono', 'Fira Code', monospace;
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 4px;
|
||||
padding: 1px 6px;
|
||||
font-size: 0.85em;
|
||||
}
|
||||
|
||||
/* Tables */
|
||||
table { width: 100%; border-collapse: collapse; margin: 16px 0; }
|
||||
th, td { padding: 10px 16px; text-align: left; border-bottom: 1px solid var(--border); }
|
||||
th {
|
||||
background: var(--surface);
|
||||
font-weight: 600;
|
||||
font-size: 0.875rem;
|
||||
color: var(--text-secondary);
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.5px;
|
||||
}
|
||||
td { font-size: 0.9rem; }
|
||||
.win { color: var(--green); font-weight: 600; }
|
||||
.benchmark-table td:nth-child(2), .benchmark-table td:nth-child(3), .benchmark-table td:nth-child(4) {
|
||||
text-align: right; font-family: 'SF Mono', monospace; font-size: 0.85rem;
|
||||
}
|
||||
.benchmark-table th:nth-child(2), .benchmark-table th:nth-child(3), .benchmark-table th:nth-child(4) { text-align: right; }
|
||||
|
||||
/* Features grid */
|
||||
.features {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
|
||||
gap: 20px;
|
||||
margin-top: 24px;
|
||||
}
|
||||
.feature {
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 24px;
|
||||
}
|
||||
.feature h3 { font-size: 1rem; margin: 0 0 8px; }
|
||||
.feature p { font-size: 0.875rem; color: var(--text-secondary); }
|
||||
|
||||
/* Hybrid LSP table */
|
||||
.lsp-table td:first-child { font-weight: 600; white-space: nowrap; }
|
||||
.lsp-table td { vertical-align: top; }
|
||||
|
||||
/* Releases */
|
||||
.release {
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 16px 20px;
|
||||
margin: 12px 0;
|
||||
}
|
||||
.release-head { display: flex; align-items: baseline; justify-content: space-between; flex-wrap: wrap; gap: 8px; }
|
||||
.release-tag { font-weight: 700; font-size: 1.05rem; }
|
||||
.release-date { color: var(--text-secondary); font-size: 0.85rem; }
|
||||
.release details { margin-top: 10px; }
|
||||
.release summary { cursor: pointer; color: var(--accent); font-size: 0.9rem; }
|
||||
.release-body { margin-top: 12px; font-size: 0.875rem; color: var(--text-secondary); }
|
||||
.release-body h4, .release-body h5, .release-body h6 { font-size: 0.95rem; color: var(--text); margin: 12px 0 6px; }
|
||||
.release-body ul { margin: 6px 0 6px 20px; }
|
||||
.release-body code { font-size: 0.8em; }
|
||||
.skeleton { color: var(--text-secondary); font-size: 0.9rem; }
|
||||
|
||||
/* FAQ */
|
||||
.faq h3 { font-size: 1.05rem; margin: 24px 0 6px; }
|
||||
.faq p { color: var(--text-secondary); margin-bottom: 8px; }
|
||||
|
||||
/* Footer */
|
||||
footer {
|
||||
padding: 40px 0;
|
||||
text-align: center;
|
||||
color: var(--text-secondary);
|
||||
font-size: 0.875rem;
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
footer a { color: var(--accent); text-decoration: none; }
|
||||
|
||||
@media (max-width: 640px) {
|
||||
.hero h1 { font-size: 1.75rem; }
|
||||
.hero .stat { display: block; margin: 8px 0; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<nav>
|
||||
<div class="container">
|
||||
<span class="links">
|
||||
<a href="#what-is-it">What is it</a>
|
||||
<a href="#install">Install</a>
|
||||
<a href="#hybrid-lsp">Hybrid LSP</a>
|
||||
<a href="#semantic-search">Semantic search</a>
|
||||
<a href="#releases">Releases</a>
|
||||
<a href="#faq">FAQ</a>
|
||||
<a class="nav-cta nav-star" href="https://github.com/DeusData/codebase-memory-mcp" title="Star codebase-memory-mcp on GitHub" aria-label="Star this project on GitHub">★ Star on GitHub</a>
|
||||
</span>
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<main class="container">
|
||||
<header class="hero">
|
||||
<h1>codebase-memory-mcp</h1>
|
||||
<p class="by-line">by DeusData</p>
|
||||
<p class="tagline">
|
||||
The fastest, most efficient code intelligence engine for AI coding agents. It indexes any
|
||||
repository into a persistent knowledge graph — full-indexing an average repo in seconds and the
|
||||
Linux kernel in 3 minutes — so your agent answers structural questions with ~120x fewer tokens.
|
||||
Tree-sitter parsing across 158 languages, Hybrid LSP type resolution, single static C binary.
|
||||
</p>
|
||||
<div>
|
||||
<div class="stat"><div class="number">~120x</div><div class="label">fewer tokens</div></div>
|
||||
<div class="stat"><div class="number">158</div><div class="label">languages</div></div>
|
||||
<div class="stat"><div class="number">3 min</div><div class="label">Linux kernel index</div></div>
|
||||
<div class="stat"><div class="number">11</div><div class="label">agents supported</div></div>
|
||||
</div>
|
||||
<div class="cta-buttons">
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp" class="cta-primary">View on GitHub</a>
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/releases/latest" class="cta-secondary">Download latest release</a>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div class="screenshot">
|
||||
<img src="graph-ui-screenshot.png" width="1538" height="932" loading="lazy" decoding="async"
|
||||
alt="3D knowledge-graph visualization of the codebase-memory-mcp graph showing thousands of nodes and edges">
|
||||
<p class="caption">Built-in 3D graph visualization (UI variant) — explore your knowledge graph at <code>localhost:9749</code>.</p>
|
||||
</div>
|
||||
|
||||
<aside class="research" aria-label="Research">
|
||||
<a class="arxiv-badge" href="https://arxiv.org/abs/2603.27277" rel="noopener">arXiv:2603.27277</a>
|
||||
<p>
|
||||
<span class="label">Research preprint.</span>
|
||||
The design and benchmarks are described in the preprint
|
||||
<a href="https://arxiv.org/abs/2603.27277"><em>Codebase-Memory: Tree-Sitter-Based Knowledge Graphs
|
||||
for LLM Code Exploration via MCP</em></a>. Evaluated across 31 real-world repositories:
|
||||
<strong>83% answer quality, 10× fewer tokens, and 2.1× fewer tool calls</strong> versus
|
||||
file-by-file exploration.
|
||||
</p>
|
||||
</aside>
|
||||
|
||||
<section id="what-is-it">
|
||||
<h2>What is codebase-memory-mcp?</h2>
|
||||
<p class="lead">
|
||||
codebase-memory-mcp is an open-source <a href="https://modelcontextprotocol.io/">Model Context
|
||||
Protocol (MCP)</a> server that indexes a codebase into a persistent knowledge graph of functions,
|
||||
classes, call chains, HTTP routes, and cross-service links. Instead of reading files one at a time,
|
||||
an AI coding agent queries the graph — answering structural questions with roughly 120x fewer
|
||||
tokens. It parses 158 languages and ships as a single static C binary with zero runtime dependencies.
|
||||
</p>
|
||||
<p class="muted">
|
||||
It is a structural-analysis backend, not a chatbot: there is no embedded LLM and no API key. Your
|
||||
MCP client (Claude Code, or any MCP-compatible agent) is the intelligence layer; codebase-memory-mcp
|
||||
builds and serves the graph. All processing happens locally — your code never leaves your machine.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section id="problem">
|
||||
<h2>Why do AI agents waste tokens exploring code?</h2>
|
||||
<p class="lead">
|
||||
AI coding agents explore codebases by reading files one at a time. Every structural question
|
||||
triggers a cascade of grep → read file → grep again → read more files. The cost compounds fast.
|
||||
</p>
|
||||
<p style="margin-bottom: 16px;">
|
||||
Across five structural questions about a real codebase, file-by-file search consumed
|
||||
<strong style="color: var(--red);">~412,000 tokens</strong>; the same questions answered from the
|
||||
knowledge graph took <strong style="color: var(--green);">~3,400 tokens</strong> — a ~120x reduction.
|
||||
</p>
|
||||
<p class="muted">
|
||||
The win is not about fitting the context window. It is cost (at $3–15 per million tokens, exploration
|
||||
adds up), latency (sub-millisecond graph queries versus seconds of file reading), and accuracy
|
||||
(less noise means better answers and no "lost in the middle" problem).
|
||||
</p>
|
||||
<p><cite>Source: project benchmark, 5 structural queries — see the
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/BENCHMARK.md">full benchmark report</a>.</cite></p>
|
||||
</section>
|
||||
|
||||
<section id="install">
|
||||
<h2>How do I install codebase-memory-mcp?</h2>
|
||||
<p class="lead">
|
||||
Install with a single command, then tell your agent to index the project. It is a single static C binary for
|
||||
macOS, Linux, and Windows — no Docker, no runtime dependencies, no API key.
|
||||
</p>
|
||||
<div class="install-block">
|
||||
<span class="comment"># 1. One-line install (macOS / Linux). Add --ui for the 3D graph UI.</span><br>
|
||||
<span class="cmd">curl -fsSL https://raw.githubusercontent.com/DeusData/codebase-memory-mcp/main/install.sh | bash</span><br><br>
|
||||
<span class="comment"># 2. The installer auto-detects and configures every installed agent.</span><br><br>
|
||||
<span class="comment"># 3. Restart your agent, then say:</span><br>
|
||||
<span class="cmd">"Index this project"</span>
|
||||
</div>
|
||||
<p class="muted">
|
||||
One command configures all 11 supported agents: Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode,
|
||||
Antigravity, Aider, KiloCode, VS Code, OpenClaw, and Kiro — with MCP entries, instruction files, and
|
||||
pre-tool hooks for each. Windows users run <code>install.ps1</code>. Also available via
|
||||
<code>npm</code>, <code>pip</code>, Homebrew, Scoop, Winget, Chocolatey, AUR, and <code>go install</code>.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section id="hybrid-lsp">
|
||||
<h2>What is Hybrid LSP?</h2>
|
||||
<p class="lead">
|
||||
Hybrid LSP is semantic type resolution beyond tree-sitter. Tree-sitter alone produces a syntactic
|
||||
AST — it handles naming, structure, and call sites, but it cannot tell that
|
||||
<code>user.profile.display_name()</code> resolves to <code>Profile.display_name</code> declared
|
||||
three modules away, because it does not track imports, generics, inheritance, or stdlib types.
|
||||
</p>
|
||||
<p style="margin-bottom: 16px;">
|
||||
codebase-memory-mcp ships a <strong>lightweight C implementation of language type-resolution
|
||||
algorithms, structurally inspired by and compatible with major language servers</strong> —
|
||||
tsserver/typescript-go, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer —
|
||||
embedded directly into the single static C binary. There is no language-server process, no
|
||||
per-project setup, and no API key. This layer runs alongside tree-sitter on every parse and refines
|
||||
<code>CALLS</code>, <code>USAGE</code>, and <code>RESOLVED_CALLS</code> edges with type information,
|
||||
so the graph mirrors what an IDE "Go to Definition" would resolve.
|
||||
</p>
|
||||
<h3>Languages with full Hybrid LSP</h3>
|
||||
<table class="lsp-table">
|
||||
<thead><tr><th>Language</th><th>What it resolves</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>Python</td><td>Imports and dotted submodule walks, dataclasses, <code>Self</code> return types, generics, <code>@property</code>, <code>match/case</code> patterns, SQLAlchemy 2.0 <code>Mapped[T]</code>, Pydantic models, <code>typing</code> annotations, async/await, isinstance/walrus narrowing, and common stdlib.</td></tr>
|
||||
<tr><td>TypeScript / JavaScript / JSX / TSX</td><td>Generics, JSX component dispatch, JSDoc inference for plain JS, <code>.d.ts</code> declarations, module re-exports, and method chaining via return-type propagation across a shared cross-file registry.</td></tr>
|
||||
<tr><td>PHP</td><td>Namespaces, traits, late-static-binding, PHPDoc inference, parameter binding, and return-type inference.</td></tr>
|
||||
<tr><td>C#</td><td>Global usings, file-scoped namespaces, records (incl. C# 12 primary constructors), LINQ method syntax, <code>async Task<T></code>/<code>ValueTask<T></code> unwrap, generic methods, <code>var</code> inference, and common BCL stdlib.</td></tr>
|
||||
<tr><td>Go</td><td>Pre-built per-package cross-file registry, generics, embedded structs, interface satisfaction, and package-aware import resolution.</td></tr>
|
||||
<tr><td>C / C++</td><td>Shared cross-language registry: macros, <code>typedef</code> chains, and header-vs-source linking on the C side; templates, namespaces, <code>auto</code> inference, and class-hierarchy method resolution on the C++ side.</td></tr>
|
||||
<tr><td>Java <em>(new in v0.8.0)</em></td><td>Imports (single-type, on-demand, static), class hierarchies with <code>this</code>/<code>super</code> dispatch, generics, annotations, overload matching by arity and parameter types, lambdas and method references bound to functional interfaces, and common JDK stdlib.</td></tr>
|
||||
<tr><td>Kotlin <em>(new in v0.8.0)</em></td><td>Imports and same-package resolution, classes / objects / companion objects, extension functions, data classes, nullable-type unwrapping, scope functions (<code>let</code>/<code>apply</code>/<code>run</code>/<code>also</code>/<code>with</code>), infix calls, and common stdlib.</td></tr>
|
||||
<tr><td>Rust <em>(new in v0.8.0)</em></td><td><code>use</code> declarations and module paths, <code>impl</code> blocks and trait methods, struct fields, generics with trait bounds, operator-trait desugaring, derive-macro method synthesis, UFCS static paths, and common std prelude.</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p class="muted">
|
||||
The two-layer pipeline runs a fast syntactic tree-sitter pass for every one of the 158 languages,
|
||||
then a type-aware Hybrid LSP pass on top for the families above. Languages without a Hybrid LSP pass
|
||||
yet fall back to textual resolution, so you always get an answer.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section id="semantic-search">
|
||||
<h2>Can it do semantic and natural-language code search?</h2>
|
||||
<p class="lead">
|
||||
Yes. Beyond structural and full-text search, codebase-memory-mcp performs <strong>semantic
|
||||
vector search</strong> across the whole graph — so you can find code by meaning, not just by
|
||||
name. A search for <code>send</code> surfaces functions named <code>publish</code>,
|
||||
<code>emit</code>, or <code>dispatch</code>.
|
||||
</p>
|
||||
<p style="margin-bottom: 16px;">
|
||||
It is powered by <strong>nomic-embed-code embeddings compiled directly into the binary</strong>
|
||||
(768-dimensional, int8). There is no API key, no Ollama, and no Docker — the embeddings run
|
||||
on-device, so semantic search stays 100% local like everything else. Results combine
|
||||
<strong>11 signals</strong> (TF-IDF, API/type/decorator signatures, AST profiles, data flow,
|
||||
Halstead-lite complexity, MinHash, module proximity, and graph diffusion) into one relevance score.
|
||||
</p>
|
||||
<h3>Meaning-aware edges in the graph</h3>
|
||||
<p style="margin-bottom: 16px;">
|
||||
The indexer also writes two kinds of meaning-aware edges, queryable like any other relationship:
|
||||
</p>
|
||||
<table>
|
||||
<thead><tr><th>Edge</th><th>What it captures</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td><code>SEMANTICALLY_RELATED</code></td><td>Conceptually similar functions whose names and tokens differ — vocabulary-mismatch matches, scored ≥ 0.80, within the same language.</td></tr>
|
||||
<tr><td><code>SIMILAR_TO</code></td><td>Near-duplicate and copy-pasted code, detected with MinHash + LSH and Jaccard scoring — ideal for finding clones and refactor candidates.</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div class="install-block" style="margin-top:16px;">
|
||||
<span class="comment"># Find code by meaning, not by name — embeddings run locally, no API key.</span><br>
|
||||
<span class="cmd">search_graph(semantic_query=["retry", "backoff", "exponential"])</span>
|
||||
</div>
|
||||
<p class="muted">
|
||||
Semantic and similarity edges are computed in <code>full</code> and <code>moderate</code> index
|
||||
modes; <code>fast</code> mode skips them for the lowest-latency indexing.
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section id="benchmark">
|
||||
<h2>How much does the knowledge graph save?</h2>
|
||||
<p class="lead">
|
||||
Each common structural question costs hundreds of tokens against the graph versus tens of thousands
|
||||
via file-by-file search. Totals across five queries: ~3,400 vs ~412,000 tokens.
|
||||
</p>
|
||||
<table class="benchmark-table">
|
||||
<thead>
|
||||
<tr><th>Question type</th><th>Graph</th><th>File-by-file</th><th>Savings</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td>Find function by pattern</td><td>~200</td><td>~45,000</td><td class="win">225x</td></tr>
|
||||
<tr><td>Trace call chain (depth 3)</td><td>~800</td><td>~120,000</td><td class="win">150x</td></tr>
|
||||
<tr><td>Dead code detection</td><td>~500</td><td>~85,000</td><td class="win">170x</td></tr>
|
||||
<tr><td>List all routes</td><td>~400</td><td>~62,000</td><td class="win">155x</td></tr>
|
||||
<tr><td>Architecture overview</td><td>~1,500</td><td>~100,000</td><td class="win">67x</td></tr>
|
||||
<tr style="font-weight: 700;"><td>Total</td><td>~3,400</td><td>~412,000</td><td class="win">~121x</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p class="muted" style="font-size: 0.9rem;">
|
||||
A separate evaluation across 31 real-world repositories, described in the preprint, reported 83% answer quality,
|
||||
10x fewer tokens, and 2.1x fewer tool calls versus file-by-file exploration.
|
||||
</p>
|
||||
<p><cite>Source: <a href="https://arxiv.org/abs/2603.27277">“Codebase-Memory: Tree-Sitter-Based Knowledge
|
||||
Graphs for LLM Code Exploration via MCP”</a>, arXiv:2603.27277 — and the
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/BENCHMARK.md">project benchmark report</a>.</cite></p>
|
||||
</section>
|
||||
|
||||
<section id="performance">
|
||||
<h2>How fast is it?</h2>
|
||||
<p class="lead">
|
||||
Indexing is RAM-first (LZ4 compression, in-memory SQLite, single dump at end) and memory is released
|
||||
to the OS afterward. Queries run in under a millisecond.
|
||||
</p>
|
||||
<table>
|
||||
<thead><tr><th>Operation</th><th>Time</th><th>Notes</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>Linux kernel full index</td><td class="win">3 min</td><td>28M LOC, 75K files → 4.81M nodes, 7.72M edges</td></tr>
|
||||
<tr><td>Django full index</td><td class="win">~6 s</td><td>49K nodes, 196K edges</td></tr>
|
||||
<tr><td>Cypher query</td><td class="win"><1 ms</td><td>Relationship traversal</td></tr>
|
||||
<tr><td>Name search (regex)</td><td class="win"><10 ms</td><td>SQL LIKE pre-filtering</td></tr>
|
||||
<tr><td>Trace call path (depth 5)</td><td class="win"><10 ms</td><td>BFS traversal</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p><cite>Source: project Performance benchmarks, measured on Apple M3 Pro.</cite></p>
|
||||
</section>
|
||||
|
||||
<section id="features">
|
||||
<h2>Features</h2>
|
||||
<div class="features">
|
||||
<div class="feature">
|
||||
<h3>158 languages</h3>
|
||||
<p>Python, Go, JS, TS, TSX, Rust, Java, C++, C#, C, PHP, Ruby, Kotlin, Scala, Zig, Elixir, Haskell, OCaml, Swift, Dart, Lean 4, and many more via vendored tree-sitter grammars compiled into the binary.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Hybrid LSP type resolution</h3>
|
||||
<p>Language-server-grade type inference for Python, TS/JS, PHP, C#, Go, C/C++, Java, Kotlin, and Rust — embedded in the binary, no server process or per-project setup.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Pure C, zero dependencies</h3>
|
||||
<p>A single static C binary for macOS, Linux, and Windows. No Docker, no runtime, no API keys. Download, run <code>install</code>, done.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Call-graph tracing</h3>
|
||||
<p>Trace callers and callees across files and packages with import-aware, type-inferred resolution. BFS traversal up to depth 5.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Dead-code detection</h3>
|
||||
<p>Find functions with zero callers, with smart filtering that excludes entry points like route handlers, <code>main()</code>, and framework decorators.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Cross-service linking</h3>
|
||||
<p>Matches REST routes to HTTP call sites across services with confidence scoring — and detects gRPC, GraphQL, and tRPC services plus pub/sub channels (<code>EMITS</code>/<code>LISTENS_ON</code> for Socket.IO, EventEmitter, and message buses) and async queue dispatch.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Infrastructure-as-code indexing</h3>
|
||||
<p>Dockerfiles, Kubernetes manifests, and Kustomize overlays become graph nodes with cross-references to the resources they configure.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Auto-sync</h3>
|
||||
<p>A background watcher detects changes and re-indexes incrementally. No manual reindex after editing files.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Team-shared graph artifact</h3>
|
||||
<p>Commit one zstd-compressed snapshot (<code>.codebase-memory/graph.db.zst</code>); teammates bootstrap from it and skip the full reindex.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>3D graph visualization</h3>
|
||||
<p>An optional UI binary serves an interactive 3D graph at <code>localhost:9749</code> to explore nodes, edges, and clusters visually.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>14 MCP tools</h3>
|
||||
<p><code>search_graph</code>, <code>trace_path</code>, <code>detect_changes</code>, <code>query_graph</code> (Cypher), <code>get_architecture</code>, <code>get_code_snippet</code>, <code>manage_adr</code>, and 7 more.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Cypher graph queries</h3>
|
||||
<p>Run read-only Cypher-style queries against the graph for multi-hop patterns that structured search can't express.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Semantic code search</h3>
|
||||
<p>Find code by meaning, not just name, via <code>semantic_query</code> vector search — powered by nomic-embed-code embeddings baked into the binary. No API key, fully local.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Clone & similarity detection</h3>
|
||||
<p><code>SIMILAR_TO</code> edges (MinHash + LSH) surface near-duplicate code; <code>SEMANTICALLY_RELATED</code> edges link conceptually similar functions across the graph.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Cross-repo intelligence</h3>
|
||||
<p>Index multiple repositories in one store and link them with <code>CROSS_*</code> edges. A multi-galaxy 3D layout and cross-repo architecture summary span the whole fleet.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Data-flow tracing</h3>
|
||||
<p><code>DATA_FLOWS</code> edges follow values from argument to parameter, with field-access chains — trace how data moves, not just who calls whom.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Change-impact analysis</h3>
|
||||
<p><code>detect_changes</code> maps an uncommitted git diff to affected symbols and their blast radius, with risk classification — see what a change touches before you ship it.</p>
|
||||
</div>
|
||||
<div class="feature">
|
||||
<h3>Architecture Decision Records</h3>
|
||||
<p><code>manage_adr</code> persists architectural decisions alongside the graph, so design rationale survives across sessions and teammates.</p>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section id="releases">
|
||||
<h2>What are the recent releases?</h2>
|
||||
<p class="lead">
|
||||
The latest release notes are loaded live from GitHub. Each entry links to its full changelog.
|
||||
</p>
|
||||
<div id="releases-list">
|
||||
<p class="skeleton">Loading recent releases from GitHub…</p>
|
||||
</div>
|
||||
<p style="margin-top: 16px;">
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/releases">View all releases on GitHub →</a>
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section id="faq" class="faq">
|
||||
<h2>Frequently asked questions</h2>
|
||||
|
||||
<h3>Does codebase-memory-mcp send my code anywhere?</h3>
|
||||
<p>No. All indexing and querying happen 100% locally. There is no embedded LLM and no API key. Release
|
||||
binaries are signed, checksummed, and scanned by 70+ antivirus engines.</p>
|
||||
|
||||
<h3>Does it support semantic or natural-language code search?</h3>
|
||||
<p>Yes. Alongside structural and full-text search, <code>search_graph</code>'s <code>semantic_query</code>
|
||||
parameter runs vector search over the whole graph, powered by nomic-embed-code embeddings compiled
|
||||
into the binary — so it finds <code>publish</code> when you search <code>send</code>. No API key, no
|
||||
Ollama, no Docker; the embeddings run on-device. The indexer also builds <code>SEMANTICALLY_RELATED</code>
|
||||
edges between similar functions and <code>SIMILAR_TO</code> edges for near-clone detection.</p>
|
||||
|
||||
<h3>Do I need Docker or a runtime?</h3>
|
||||
<p>No. It is a single static C binary with zero runtime dependencies for macOS (arm64/amd64), Linux
|
||||
(arm64/amd64), and Windows (amd64).</p>
|
||||
|
||||
<h3>How does it stay up to date as I edit code?</h3>
|
||||
<p>A background watcher detects file changes and re-indexes incrementally — typically a sub-millisecond
|
||||
no-op when nothing changed. You only run a manual index for the first build or after a large
|
||||
<code>git pull</code>.</p>
|
||||
|
||||
<h3>Why is there no built-in LLM?</h3>
|
||||
<p>Other code-graph tools embed an LLM to translate natural language into graph queries, which means
|
||||
extra API keys and cost. With MCP, the agent you are already talking to is the query translator —
|
||||
codebase-memory-mcp just builds and serves the graph.</p>
|
||||
|
||||
<h3>Is it free and open source?</h3>
|
||||
<p>Yes. It is MIT licensed. The full source, signed release binaries, and checksums are on
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp">GitHub</a>.</p>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<footer>
|
||||
<div class="container">
|
||||
<p>
|
||||
Open source, MIT licensed.
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp">GitHub</a> ·
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/releases/latest">Releases</a> ·
|
||||
<a href="https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/BENCHMARK.md">Benchmarks</a> ·
|
||||
<a href="https://arxiv.org/abs/2603.27277">Paper</a>
|
||||
</p>
|
||||
</div>
|
||||
</footer>
|
||||
|
||||
<script>
|
||||
(function () {
|
||||
var REPO = 'DeusData/codebase-memory-mcp';
|
||||
var listEl = document.getElementById('releases-list');
|
||||
if (!listEl) return;
|
||||
|
||||
function esc(s) {
|
||||
return String(s)
|
||||
.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>')
|
||||
.replace(/"/g, '"');
|
||||
}
|
||||
|
||||
// Minimal, safe Markdown → HTML for release bodies (escape first, then format).
|
||||
function renderMarkdown(md) {
|
||||
var lines = esc(md || '').split(/\r?\n/);
|
||||
var html = '', inList = false;
|
||||
function inline(t) {
|
||||
return t
|
||||
.replace(/`([^`]+)`/g, '<code>$1</code>')
|
||||
.replace(/\*\*([^*]+)\*\*/g, '<strong>$1</strong>')
|
||||
.replace(/\[([^\]]+)\]\((https?:\/\/[^)\s]+)\)/g,
|
||||
'<a href="$2" target="_blank" rel="nofollow noopener">$1</a>');
|
||||
}
|
||||
for (var i = 0; i < lines.length; i++) {
|
||||
var ln = lines[i];
|
||||
var h = ln.match(/^(#{1,6})\s+(.*)$/);
|
||||
var li = ln.match(/^\s*[-*]\s+(.*)$/);
|
||||
if (h) {
|
||||
if (inList) { html += '</ul>'; inList = false; }
|
||||
// Demote release-note headings to h4-h6 so they never pollute the page outline.
|
||||
var lvl = Math.min(h[1].length + 3, 6);
|
||||
html += '<h' + lvl + '>' + inline(h[2]) + '</h' + lvl + '>';
|
||||
} else if (li) {
|
||||
if (!inList) { html += '<ul>'; inList = true; }
|
||||
html += '<li>' + inline(li[1]) + '</li>';
|
||||
} else if (ln.trim() === '') {
|
||||
if (inList) { html += '</ul>'; inList = false; }
|
||||
} else {
|
||||
if (inList) { html += '</ul>'; inList = false; }
|
||||
html += '<p>' + inline(ln) + '</p>';
|
||||
}
|
||||
}
|
||||
if (inList) html += '</ul>';
|
||||
return html;
|
||||
}
|
||||
|
||||
function fmtDate(iso) {
|
||||
if (!iso) return '';
|
||||
var d = new Date(iso);
|
||||
if (isNaN(d)) return '';
|
||||
return d.toLocaleDateString('en-US', { year: 'numeric', month: 'short', day: 'numeric' });
|
||||
}
|
||||
|
||||
function renderRelease(r) {
|
||||
var tag = esc(r.tag_name || r.name || 'release');
|
||||
var name = r.name && r.name !== r.tag_name ? ' — ' + esc(r.name) : '';
|
||||
var url = esc(r.html_url || ('https://github.com/' + REPO + '/releases'));
|
||||
var date = fmtDate(r.published_at);
|
||||
var body = (r.body || '').trim();
|
||||
var bodyHtml = body
|
||||
? '<details><summary>Release notes</summary><div class="release-body">' + renderMarkdown(body) + '</div></details>'
|
||||
: '<p class="release-body"><a href="' + url + '" target="_blank" rel="noopener">View on GitHub →</a></p>';
|
||||
return '<article class="release">'
|
||||
+ '<div class="release-head">'
|
||||
+ '<span class="release-tag"><a href="' + url + '" target="_blank" rel="noopener">' + tag + '</a>' + name + '</span>'
|
||||
+ '<span class="release-date">' + date + '</span>'
|
||||
+ '</div>' + bodyHtml + '</article>';
|
||||
}
|
||||
|
||||
fetch('https://api.github.com/repos/' + REPO + '/releases?per_page=5', {
|
||||
headers: { 'Accept': 'application/vnd.github+json' }
|
||||
})
|
||||
.then(function (res) { if (!res.ok) throw new Error('HTTP ' + res.status); return res.json(); })
|
||||
.then(function (data) {
|
||||
if (!Array.isArray(data) || data.length === 0) throw new Error('no releases');
|
||||
listEl.innerHTML = data.map(renderRelease).join('');
|
||||
})
|
||||
.catch(function () {
|
||||
listEl.innerHTML = '<p class="skeleton">Couldn\'t load releases right now — '
|
||||
+ '<a href="https://github.com/' + REPO + '/releases">view them on GitHub →</a></p>';
|
||||
});
|
||||
})();
|
||||
</script>
|
||||
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,25 @@
|
||||
# codebase-memory-mcp
|
||||
|
||||
> An open-source Model Context Protocol (MCP) server that indexes a codebase into a persistent knowledge graph of functions, classes, call chains, HTTP routes, and cross-service links. AI coding agents query the graph instead of reading files one by one, answering structural questions with roughly 120x fewer tokens (~3,400 vs ~412,000 across five structural queries). Parses 158 languages via vendored tree-sitter grammars, with Hybrid LSP semantic type resolution for Python, TypeScript/JavaScript, PHP, C#, Go, C/C++, Java, Kotlin, and Rust. It also generates semantic graph edges (SEMANTICALLY_RELATED for vocabulary-mismatch matches, SIMILAR_TO for near-clone detection) and supports semantic vector search via bundled nomic-embed-code embeddings compiled into the binary — no API key, no Ollama, no Docker. Ships as a single static C binary (zero runtime dependencies) for macOS, Linux, and Windows. All processing is local — the embeddings run on-device; there is no embedded LLM and no API key.
|
||||
|
||||
## Key facts
|
||||
|
||||
- License: MIT, open source.
|
||||
- Languages: 158 (158 vendored tree-sitter grammars compiled into the binary).
|
||||
- Hybrid LSP type resolution: 9 language families (Python, TypeScript/JavaScript/JSX/TSX, PHP, C#, Go, C/C++, Java, Kotlin, Rust) — a lightweight C implementation of language type-resolution algorithms, structurally inspired by and compatible with major language servers including tsserver, pyright, gopls, Roslyn, Eclipse JDT, and rust-analyzer.
|
||||
- MCP tools: 14 (search_graph incl. semantic_query vector search, trace_path (alias: trace_call_path), query_graph (Cypher), detect_changes, get_architecture, get_code_snippet, manage_adr, and more).
|
||||
- Semantic search: natural-language code discovery via bundled nomic-embed-code embeddings (768-dim, compiled into the binary); 11-signal combined scoring; fully local, no API key.
|
||||
- Semantic & similarity edges: SEMANTICALLY_RELATED (vocabulary-mismatch matches) and SIMILAR_TO (MinHash + LSH near-clone / duplicate detection).
|
||||
- Cross-repo intelligence: CROSS_* edges link nodes across multiple repos indexed in one store; multi-galaxy 3D layout and cross-repo architecture summary.
|
||||
- Cross-service linking: HTTP route ↔ call-site matching, plus gRPC/GraphQL/tRPC detection and pub/sub channels (EMITS/LISTENS_ON for Socket.IO, EventEmitter, generic buses).
|
||||
- Supported agents: 11 (Claude Code, Codex CLI, Gemini CLI, Zed, OpenCode, Antigravity, Aider, KiloCode, VS Code, OpenClaw, Kiro).
|
||||
- Performance: Linux kernel (28M LOC, 75K files) full index in 3 minutes → 4.81M nodes, 7.72M edges; Cypher queries in under 1ms.
|
||||
- Distribution: single static C binary; also npm, PyPI, Homebrew, Scoop, Winget, Chocolatey, AUR, and `go install`.
|
||||
|
||||
## Links
|
||||
|
||||
- Homepage: https://deusdata.github.io/codebase-memory-mcp/
|
||||
- Source code (GitHub): https://github.com/DeusData/codebase-memory-mcp
|
||||
- Latest release: https://github.com/DeusData/codebase-memory-mcp/releases/latest
|
||||
- Benchmark report: https://github.com/DeusData/codebase-memory-mcp/blob/main/docs/BENCHMARK.md
|
||||
- Research paper: https://arxiv.org/abs/2603.27277
|
||||
@@ -0,0 +1,33 @@
|
||||
# codebase-memory-mcp — all crawlers welcome, including AI answer engines.
|
||||
User-agent: *
|
||||
Allow: /
|
||||
|
||||
# Explicitly allow AI / LLM crawlers used by answer engines.
|
||||
User-agent: GPTBot
|
||||
Allow: /
|
||||
|
||||
User-agent: OAI-SearchBot
|
||||
Allow: /
|
||||
|
||||
User-agent: ChatGPT-User
|
||||
Allow: /
|
||||
|
||||
User-agent: ClaudeBot
|
||||
Allow: /
|
||||
|
||||
User-agent: Claude-Web
|
||||
Allow: /
|
||||
|
||||
User-agent: PerplexityBot
|
||||
Allow: /
|
||||
|
||||
User-agent: Google-Extended
|
||||
Allow: /
|
||||
|
||||
User-agent: Applebot-Extended
|
||||
Allow: /
|
||||
|
||||
User-agent: Bingbot
|
||||
Allow: /
|
||||
|
||||
Sitemap: https://deusdata.github.io/codebase-memory-mcp/sitemap.xml
|
||||
@@ -0,0 +1,8 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
|
||||
<url>
|
||||
<loc>https://deusdata.github.io/codebase-memory-mcp/</loc>
|
||||
<lastmod>2026-06-12</lastmod>
|
||||
<priority>1.0</priority>
|
||||
</url>
|
||||
</urlset>
|
||||
Reference in New Issue
Block a user