Files
wehub-resource-sync bcbd1bdb22
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:30:44 +08:00

138 lines
4.8 KiB
Plaintext

---
title: LanceDB
description: Mount a LanceDB table as a Mirage filesystem, with label folders, multimodal blobs, and a semantic search command for Python agents.
icon: database
---
The LanceDB resource exposes a LanceDB table as a virtual filesystem mounted at
some prefix such as `/fashion/`. Group-by columns become nested folders, each
row becomes a card plus an optional blob file, and semantic search is the
`search` command, which returns ranked rows as canonical file paths.
For connection setup (LanceDB OSS, object storage, Cloud, Enterprise), see
[LanceDB Setup](/python/setup/lancedb).
## Config
```python
from mirage import MountMode, Workspace
from mirage.resource.lancedb import LanceDBConfig, LanceDBResource
config = LanceDBConfig(
uri="/data/fashion.lancedb",
table="fashion",
group_by=["gender", "articleType", "baseColour"],
id_column="id",
title_column="productDisplayName",
blob_column="image_bytes",
blob_ext="jpg",
vector_column="vector",
search_limit=5,
)
resource = LanceDBResource(config)
ws = Workspace({"/fashion/": resource}, mode=MountMode.READ)
```
The mapping is config-driven; nothing about the dataset is hardcoded. Point
`group_by` at different columns and the folder tree changes. See the full
[config reference](/python/setup/lancedb#config-reference).
## Filesystem layout
Every path is translated into a LanceDB query. Descending a folder adds one
`WHERE` clause; the leaf level lists rows.
```text
/ # list tables (omitted when `table` is pinned)
/<table>/ # distinct group_by[0] values
<v1>/ # distinct group_by[1] WHERE group_by[0]=v1
.../<vN>/ # all group-by columns bound -> row files
<id>.md # rendered card (text)
<id>.<ext> # raw blob / image bytes
```
When `table` is set the table level is elided, so the mount root is that table:
```text
/fashion/
Men/
Shoes/
White/
3.md
3.jpg
```
### Row cards
A `<id>.md` card renders the row's columns as readable text and points at its
blob. The vector and blob columns are omitted from the card body.
```text
# Nike Men White Running Sneakers
id: 3
gender: Men
articleType: Shoes
baseColour: White
productDisplayName: Nike Men White Running Sneakers
blob: 3.jpg
```
## Semantic search
Search is a command, not a path. It returns each ranked row as its **canonical
file path** (the same `<id>.md` you would `cat` while browsing) annotated with
the vector distance, followed by the card body. Results point back at the real
files, so search composes with `cat`, pipes, and `wc`:
```text
$ search "white running sneakers" /fashion
/fashion/Men/Shoes/White/3.md:0.2679
# Nike Men White Running Sneakers
id: 3
gender: Men
articleType: Shoes
baseColour: White
productDisplayName: Nike Men White Running Sneakers
blob: 3.jpg
...
```
Flags: `--top-k <n>` (default `search_limit`), `--threshold <max-distance>`,
`--method semantic` (the only supported method; `grep`/`rg` stay lexical).
## Supported commands
All commands delegate to Mirage's shared implementations.
| Command | Behaviour on a LanceDB mount |
| ------------- | ------------------------------------------------------------- |
| `ls` | list tables, label folders, or row files |
| `cd` | navigate (each level narrows the filter) |
| `tree` | render the label hierarchy |
| `cat` | print a row card, or dump raw blob/image bytes |
| `stat` | directory vs file, blob size, image mime type |
| `find` | walk the tree (e.g. `find /fashion -name '*.md'`) |
| `grep` / `rg` | lexical search over the rendered cards |
| `search` | semantic (vector) search -> ranked canonical paths + score |
| `head` / `tail` | first/last lines of a card |
| `wc` | count lines/bytes of a card |
`grep`/`rg` stay lexical (literal/regex). `search` is the semantic path: it
auto-embeds the query via the table's embedding function and returns ranked
rows as canonical file paths, which compose with `cat`, `wc`, and pipes.
## Access pattern
The mount is **read-only** (`MountMode.READ`); writes are not supported. The two
read modes are:
- **Browse** by label folders: pure metadata `WHERE` filters, no embedding.
- **Search** by meaning: `search "<query>" <path>` runs vector search using the
table's embedding function and returns canonical row paths.
Folder listings scan one column with `SELECT DISTINCT` and are capped by
`max_rows`, so very large tables should keep `group_by` to low-cardinality
columns.