# tree-sitter-vbnet.wasm — provenance & rebuild

`src/extraction/wasm/tree-sitter-vbnet.wasm` is built from
[govindbanura/tree-sitter-vbnet](https://github.com/govindbanura/tree-sitter-vbnet)
(MIT) at commit `538b7087bf80e86004531b392fe1186379c0a2b5` with the patch in
`tree-sitter-vbnet.patch` applied. The patch carries two files: `grammar.js`
(edits) and `src/scanner.c` (a new external scanner; upstream has none). The
upstream repo checks in no generated `src/`, so everything else is produced by
`tree-sitter generate`.

Alternatives considered: `CodeAnt-AI/tree-sitter-vb-dotnet` (22★) has **no
license file** and its git history stopped in July 2025 — unusable for
vendoring; `gabriel-gubert/tree-sitter-vbnet` is a 470-line VBScript-flavored
toy. The Roslyn-based approach (PR #627) was withdrawn by its author in favor
of tree-sitter — a Roslyn sidecar would add a .NET runtime dependency to a
local-first npm tool.

## What the patch adds

Upstream parses textbook VB.NET but fails on the constructs that dominate real
codebases (measured: 3–18% of files parsed clean across PolicyPlus, CompactGUI,
and staxrip before patching). Each item below was found by parse-error census
on those repos plus SCrawler and PCL:

1. **Generic type arguments in dotted names** — `System.Collections.Generic.
   Dictionary(Of K, V)`, `Implements IRepository(Of Invoice)`, and
   method-level `Implements I(Of T).Member` (generic segments were only
   accepted unqualified). Open generic types (`GetType(LoaderTask(Of ,))`)
   parse too.
2. **Interpolated strings** `$"… {expr[,align][:fmt]} …"` with `""`/`{{`/`}}`
   escapes — including multi-line bodies and content pieces that begin with an
   apostrophe: the pieces carry lexical precedence 101 (above `comment`'s 100)
   because the comment **extra** otherwise fires *inside* the string rule and
   eats the rest of the line, closing quote included.
3. **Date/time literals** `#1/15/2020#` — previously lexed as a preprocessor
   directive that swallowed to end-of-line. Directives are now constrained to
   `#` + letter (`#If`, `#Region`, …), which real directives always satisfy.
4. **VB 14 multi-line string literals** (a `"…"` literal may span lines since
   VS 2015) and single-token `string_literal`/`character_literal` (`"["c`) —
   the old multi-token form let extras interleave mid-string.
5. **Numeric literal forms** — hex/octal/binary (`&HFF`, `&O777`, `&B1010`),
   digit separators (`1_000`), type characters (`6.0!`, `50.0#`, `1.5@`,
   `123&`, `7%`), and lowercase `f/r/d` suffixes. WinForms `.Designer.vb`
   files are full of `6.0!`.
6. **Identifier type characters and Unicode identifiers** — `Dim i% = 0`,
   `Dim r$ = …` (classic VB style, pervasive in SCrawler) and full Unicode
   identifiers (`CrashReason.Java虚拟机参数有误` — PCL is written in Chinese).
   The identifier token is now `[\p{L}\p{Nl}_][\p{L}\p{Nl}\p{Nd}\p{Mn}\p{Pc}]*
   [%$&!#@]?` with the `u` regex flag. **The `u` flag requires
   tree-sitter-cli ≥ 0.25** — 0.24.x silently drops the `\p{…}` classes.
7. **`As New T(args)` initializer clauses** — `as_clause` embeds a full
   `object_creation_expression` for the `As New` form, so `Dim x As New
   StringBuilder` / `Property P As New List(Of String)` produce instantiation
   nodes. `Dim x? = expr` nullable declarators parse as well.
8. **Statement separators and single-line forms** — `:` as a statement
   terminator and block opener (`Class X : Inherits Y`, `Case 1 : Return "X"`),
   single-line `If … Then stmt Else stmt` (via terminator-less inline statement
   variants, aliased to the normal statement node names), inline `RaiseEvent`,
   and optional `Then` on block `If` and `ElseIf` (legal VB, used in staxrip).
9. **Multi-line lambdas** — `Sub(…) … End Sub` / `Function(…) … End Function`
   bodies (upstream had a statement-block body with no `End` closer, so every
   block lambda broke its surrounding argument list), `Async`/`Iterator`
   lambda modifiers, `ByVal`/`ByRef` lambda parameters, and single-line
   `Sub() If cond Then …` statement bodies.
10. **Member declarations** — `Declare [Auto|Ansi|Unicode] Sub/Function … Lib
    "dll" [Alias "…"]` P/Invoke declarations, `Custom Event … AddHandler/
    RemoveHandler/RaiseEvent … End Event`, stacked attribute lines above one
    member, property `= initializer` before `Implements`, type-less
    auto-properties, and **`MustOverride` body-less methods and properties**:
    `MustOverride` lexes as a dedicated token (removed from the
    `member_modifier` alternation) that only `abstract_method_declaration` /
    `abstract_property_declaration` accept, making the body-less parse
    deterministic. (A GLR body-less alternative on `method_declaration` was
    tried first and measurably poisoned error recovery — 100%→60% clean on
    PolicyPlus — before being replaced with the token split.)
11. **Expressions** — VB 15 tuple literals `(a, b)`, array literals
    `{1, 2, 3}` (plus nested `{{k, v}, …}` dictionary groups, replacing the
    ambiguous upstream `dictionary_initializer`), omitted argument slots
    (`f(a,, b)` — Optional parameters passed positionally), `TypeOf x IsNot T`,
    generic method calls without parens (`items.OfType(Of Panel)`),
    null-conditional indexing `x?(0)`, and `Global.`-qualified type names.
12. **LINQ queries** — query expressions no longer require a trailing
    `Select`/`Group` clause, `Aggregate`-led queries, and
    `Distinct`/`Skip`/`Take` clauses.

### External scanner (`src/scanner.c`, new)

Two constructs are not LR(1)-parseable with tree-sitter's newline-as-extra
treatment; both get external tokens:

- **`QUERY_CLAUSE_CONTINUATION`** — multi-line LINQ (`From x In xs` ↵
  `Where …`). At a clause boundary the newline alone cannot distinguish
  "query continues on the next line" from "statement ends here". The scanner
  looks past the newline run at the next word and emits the continuation
  token only when it is a query-clause keyword (with a `Select Case`
  guard), so the decision is made by the lexer instead of the LR table.
- **`XML_LITERAL`** — whole VB XML literals (`<Tags><Tag/></Tags>`) consumed
  as one opaque token: element nesting, attributes, comments, CDATA,
  processing instructions, and **nested** `<%= … %>` embedded expressions
  (the staxrip `WriteTagfile` shape). Valid only where a literal can begin an
  expression, so a relational `<` (which always *follows* an expression)
  never collides. The scanner never skips a leading newline (it must remain
  available as a statement terminator).

The scanner is stateless (serialize/deserialize are no-ops).

The `_eof` hack upstream (a literal-`$` token) cannot match a real
end-of-file, so files whose last line has no trailing newline would end with a
MISSING-newline error; the extractor's `preParse` appends a trailing newline
instead of patching that in the grammar.

## Measured parse health (at vendoring time)

| Corpus | Clean parses |
|---|---|
| Fleex255/PolicyPlus (94 `.vb`) | 94/94 (100%) — upstream: 3/94 |
| IridiumIO/CompactGUI (66) | 66/66 (100%) — upstream: 12/66 |
| staxrip/staxrip (145) | 138/145 (95.2%) — upstream: 22/145 |
| AAndyProgram/SCrawler (320) | 279/320 (87.2%) |
| Meloong-Git/PCL (112, Chinese identifiers) | 98/112 (87.5%) |

Known remaining gap (localized ERROR regions, deliberately unpatched):

- **Column-0 GoTo labels** (`Recheck:` at the start of a line inside indented
  code — the classic VB label style, used heavily in PCL). The `word:`
  keyword-extraction token interacts badly with a newline immediately followed
  by a word at column 0, consuming the newline and dropping the previous
  statement's terminator. Removing `word:` fixes labels but reintroduces
  keyword-prefix identifier bugs corpus-wide (measured: staxrip 95%→28%), so
  `word:` stays and column-0 labels keep a localized error; indented labels
  parse fine. Worth an upstream tree-sitter investigation eventually.

## Rebuild

```bash
git clone https://github.com/govindbanura/tree-sitter-vbnet
cd tree-sitter-vbnet
git checkout 538b7087bf80e86004531b392fe1186379c0a2b5
git apply path/to/tree-sitter-vbnet.patch   # patches grammar.js, adds src/scanner.c
# tree-sitter needs a tree-sitter.json (upstream ships none); grammar name is
# `vbnet` (C symbols tree_sitter_vbnet*):
cat > tree-sitter.json <<'JSON'
{
  "grammars": [
    { "name": "vbnet", "camelcase": "Vbnet", "scope": "source.vbnet",
      "path": ".", "file-types": ["vb"] }
  ],
  "metadata": { "version": "0.1.0", "license": "MIT",
    "description": "VB.NET grammar for tree-sitter",
    "links": { "repository": "https://github.com/govindbanura/tree-sitter-vbnet" } }
}
JSON
npm install tree-sitter-cli@0.25.10   # ≥0.25 REQUIRED: the /u regex flag (Unicode
                                      # identifiers) is dropped silently by 0.24.x
npx tree-sitter generate              # src/scanner.c from the patch is picked up
npx tree-sitter build --wasm -o tree-sitter-vbnet.wasm   # needs emscripten or Docker
```

Upstream's checked-in `test/corpus` expectations predate its own grammar.js
(every corpus test fails at the pinned commit, before any patching), so the
five-repo parse-health sweep above — plus 16 construct repros and the
`__tests__/extraction.test.ts` VB.NET block — is the regression baseline.

## Upstreaming

Not yet sent. The patch is one large, coherent "parse real-world VB.NET"
change; if upstream shows signs of life it can be offered as a PR the same way
the COBOL patch was ([tree-sitter-cobol#41](https://github.com/yutaro-sakamoto/tree-sitter-cobol/pull/41)),
with the corpus numbers above as the motivation. Until then,
`git apply tree-sitter-vbnet.patch` on upstream commit `538b708` reproduces
the vendored grammar exactly.