chore: import upstream snapshot with attribution

examples/python/droid_semantic_search/.gitignore

@@ -0,0 +1,10 @@
+# Local vector index built by `ingest.py` (LanceDB / Qdrant backends)
+droid_lancedb/
+droid_qdrant/
+# DROID episodes downloaded by `prepare_dataset.py` (Hugging Face path)
+data/
+# Optimized (keyframe-fixed) episode copies written by `prepare_dataset.py`
+optimized/
+# uv-managed virtualenv for this isolated example
+.venv/
+__pycache__/

examples/python/droid_semantic_search/README.md

@@ -0,0 +1,162 @@
+<!--[metadata]
+title = "DROID semantic frame search"
+tags = ["Robotics", "Semantic search", "Embeddings", "SigLIP", "LanceDB", "Qdrant", "Rerun Hub"]
+thumbnail = "https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/480w.png"
+thumbnail_dimensions = [480, 320]
+include_in_manifest = true
+-->
+
+Find moments in robot demonstrations by describing them in plain language, and jump straight to the matching frame in the Rerun viewer.
+
+<picture>
+  <source media="(max-width: 480px)" srcset="https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/480w.png">
+  <source media="(max-width: 768px)" srcset="https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/768w.png">
+  <source media="(max-width: 1024px)" srcset="https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/1024w.png">
+  <source media="(max-width: 1200px)" srcset="https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/1200w.png">
+  <img src="https://static.rerun.io/droid_semantic_search/fd265e110c9d05c9fdd018c68dd83561aa836c52/full.png" alt="DROID semantic frame search screenshot">
+</picture>
+
+This pulls frame embeddings from a Rerun dataset, indexes them in an external vector store, and resolves text queries back into deep-links that open the relevant frames in the viewer.
+It ships two interchangeable backends — [LanceDB](https://lancedb.com) (default) and [Qdrant](https://qdrant.tech), both fully local on disk and selected with `--backend`.
+They're worked examples of one pattern — embeddings out of Rerun, search in your vector database of choice, deep-links back into the viewer — that applies to any vector store.
+
+## What it shows off
+
+- Schema introspection and DataFusion content-filtered reads.
+- The experimental video dataloader (`DataSource` / `Field` / `VideoFrameDecoder`) for streaming and decoding H.264 frames on demand.
+- SigLIP-2 embeddings, with text and image features sharing one vector space.
+- `segment_url` deep-links that focus the viewer on a specific frame.
+
+## How it works
+
+Three scripts:
+
+1. `prepare_dataset.py` — registers a few DROID episodes to your local catalog as a dataset (using the episodes bundled in the repo, or downloading them from the Hugging Face Hub when those aren't present). This is the data the other two scripts read.
+
+2. `ingest.py` — builds the index. For each camera it **auto-detects** the embedding source:
+   - **Read path:** if the dataset already has a `/camera/{role}/embedding` column (DROID registered with `--create-embeddings`), it reads those vectors directly via a DataFusion query — no model, no video decoding.
+   - **Compute path:** otherwise it streams the `VideoStream` through the dataloader, decodes frames, and embeds them with SigLIP-2.
+
+   Either way it writes `(segment_id, camera, timestamp_ms, vector)` rows into the selected vector store: LanceDB by default, or Qdrant with `--backend qdrant`.
+
+3. `search.py` — embeds your example image or text prompt with the SigLIP-2 encoder, runs a vector search over that store (pass the same `--backend`), prints the ranked matches, and opens the best one in the viewer.
+
+Both scripts reach the store only through a small `VectorStore` interface in `vector_store.py`, so picking a backend is one `--backend` choice and adding one is a single subclass.
+Each backend keeps its index in its own directory (`./droid_lancedb` vs `./droid_qdrant`, override with `--db-path`), so the `--backend` you pass to `search.py` must match the one `ingest.py` wrote.
+
+## Run the code
+
+### 1. Install dependencies
+
+This example has its own `uv` project, separate from the workspace `.venv`, because it needs the experimental `rerun-sdk[dataloader]` extras plus heavy ML deps (`transformers`, `lancedb`).
+
+**Standalone** (sparse-checkout of just this directory, no local Rerun build):
+
+```bash
+uv sync --no-sources --no-dev
+```
+
+**Monorepo dev** (full repo checkout, editable local `rerun-sdk`):
+
+```bash
+cd examples/python/droid_semantic_search
+RERUN_ALLOW_MISSING_BIN=1 uv sync
+uv pip install ../../../rerun_py/rerun_dev_fixup
+```
+
+The second command installs the `.pth` shim that points `import rerun` (and the `rerun` CLI) at the in-repo editable source tree.
+It's a separate `uv pip install` rather than a dev-group dependency because uv resolves all dependency groups unconditionally, so a path-only package in `pyproject.toml` would break the standalone `--no-sources` resolution above.
+
+Then either `source .venv/bin/activate` or prefix subsequent commands with `uv run`.
+
+### 2. Start a local Rerun server
+
+This example reads data from a local open-source Rerun catalog server.
+In a separate terminal, start one:
+
+```bash
+rerun server
+```
+
+This serves a catalog at `rerun+http://127.0.0.1:51234` — the default the scripts use.
+Leave it running; the steps below connect to it.
+
+### 3. Register a dataset
+
+Registers a few DROID episodes to the catalog as `droid:sample`:
+
+```bash
+uv run python prepare_dataset.py
+```
+
+By default it auto-selects the source:
+
+- **Monorepo checkout** — it uses the episodes bundled in the repo at `tests/assets/rrd/sample_5` (via git-LFS), so there's nothing to download. If those are un-pulled LFS pointers, run `git lfs install && git lfs pull` first.
+- **Standalone checkout** — when the bundled episodes aren't present, it downloads a few from [`rerun/droid_sample`](https://huggingface.co/datasets/rerun/droid_sample) on the Hugging Face Hub into `./data`.
+
+Useful flags:
+
+- `--source bundled|huggingface` to force a source (default `auto`).
+- `--num-episodes N` to register more (or fewer) episodes; `0` for all (the full Hub dataset is ~3.3 GB).
+- `--dataset-name` to register under a different name (pass the same name to `ingest.py`/`search.py`).
+- `--no-optimize` to register episodes as-is (see the video decode yield note below).
+- `--catalog-url ""` to skip registration.
+
+### 4. Build the search index
+
+Index a handful of segments (the exterior camera works well for scene-level search):
+
+```bash
+uv run python ingest.py --num-segments 15 --cameras ext1
+```
+
+These sample episodes ship video only (no pre-computed embeddings), so `ingest.py` takes the **compute path**:
+the first run downloads the SigLIP-2 model (a few hundred MB) and then decodes and embeds frames — the slow step.
+Start with a small `--num-segments` to keep it quick; raise it once you've seen it work.
+
+### 5. Search
+
+Search by text and open the best hit in the viewer:
+
+```bash
+uv run python search.py "an open drawer full of tools" --top-k 5
+```
+
+Or search by example image instead of text (same vector space):
+
+```bash
+uv run python search.py --image ./query.jpg --top-k 5
+```
+
+Both scripts default to LanceDB; pass `--backend qdrant` to use [Qdrant](https://qdrant.tech) instead.
+Give the same `--backend` to `ingest.py` and `search.py`, since each writes to its own directory:
+
+```bash
+uv run python ingest.py --backend qdrant --num-segments 15 --cameras ext1
+uv run python search.py --backend qdrant "an open drawer full of tools" --top-k 5
+```
+
+Queries that discriminate well on DROID describe concrete, visible objects/scenes, e.g. `"a pink flower"`, `"a cardboard box"`, `"a white plastic bag"`, `"a robot arm over an empty table"`.
+
+Run `ingest.py --help` and `search.py --help` for the full flag list — index multiple cameras, change the sampling rate, widen the time selection around a hit, and more.
+
+## Scope and extension points
+
+- **Text or image queries.** Search by a text prompt or, with `--image <path>`, by an example frame. SigLIP-2 puts text and image features in one space, so image-to-image search reuses the exact same index and ranking — only the query encoder differs.
+- **Bring your own vector store.** Rerun supplies the embeddings (read from the dataset, or computed from platform-hosted video) and resolves matches back into viewer deep-links; search itself runs in an external store. This example ships two backends, LanceDB and Qdrant, behind the small `VectorStore` interface in `vector_store.py` — the same write-then-query flow drops onto any vector database, so adding a third is a single subclass.
+
+## Notes and gotchas
+
+- **SigLIP text tokenization.** SigLIP is trained with a fixed 64-token sequence and *must* be tokenized with `padding="max_length", max_length=64`. With dynamic padding the text embeddings are malformed and text→image retrieval collapses onto a single "hub" frame that wins every query.
+- **Video decode yield.** DROID doesn't log the `VideoStream:is_keyframe` markers the decoder needs to seek, so without them only ~25 % of sampled frames decode.
+  `prepare_dataset.py` derives the markers up front (via `optimize`), which brings yield to ~100 %.
+  Optimized copies land in `./optimized`; the originals are untouched.
+  Skip it with `--no-optimize` if you'd rather register the raw episodes.
+
+## Files
+
+- `prepare_dataset.py` — register DROID sample episodes (bundled or downloaded) to the catalog.
+- `ingest.py` — build the vector index from the dataset.
+- `search.py` — query the index and open results in the viewer.
+- `vector_store.py` — the `VectorStore` interface, with LanceDB and Qdrant backends.
+- `embeddings.py` — SigLIP-2 helpers (adapted from the DROID loader's `embedding_util.py`).

examples/python/droid_semantic_search/embeddings.py

@@ -0,0 +1,121 @@
+"""SigLIP-2 embedding helpers shared by `ingest.py` and `search.py`.
+
+These are trimmed copies of the helpers in the DROID loader
+(`dataplatform/examples/droid/droid-loader/src/droid_loader/embedding_util.py`),
+with the `Timer` instrumentation removed so this example stays standalone and
+doesn't pull in the `droid_loader` package. The model is the same one the
+loader uses to populate `/camera/{role}/embedding`, so query embeddings land in
+the same vector space as any pre-computed frame embeddings.
+"""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+import torch
+
+# Disable HF `tokenizers` (Rust) parallelism *before* importing transformers. Otherwise,
+# once the SigLIP tokenizer has been used and the process later forks (e.g. a DataLoader
+# worker), tokenizers prints "the current process just got forked, after parallelism has
+# already been used". We only tokenize tiny queries, so parallelism buys nothing here.
+# `setdefault` lets a caller still override via the real environment variable.
+os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
+
+from transformers import AutoModel, AutoProcessor  # imported after TOKENIZERS_PARALLELISM is set (above)
+
+if TYPE_CHECKING:
+    from PIL.Image import Image
+
+# The concrete SigLIP-2 model/processor that `from_pretrained` returns.
+EmbeddingModel = Any
+EmbeddingProcessor = Any
+
+# Dual image/text encoder; image and text features share one space, so text
+# queries retrieve image frames directly. 768-dim, L2-normalized output.
+EMBEDDING_MODEL = "google/siglip2-base-patch16-224"
+
+
+def _resolve_device(device: str | torch.device | None) -> torch.device:
+    if device is not None:
+        return torch.device(device)
+    if torch.cuda.is_available():
+        return torch.device("cuda")
+    if torch.backends.mps.is_available():
+        return torch.device("mps")
+    return torch.device("cpu")
+
+
+def load_embedding_model(
+    cache_dir: str | Path | None = None,
+    use_fast: bool = True,
+) -> tuple[EmbeddingModel, EmbeddingProcessor]:
+    """Load the SigLIP-2 model and its processor."""
+    print(f"Loading model '{EMBEDDING_MODEL}'")
+    model = AutoModel.from_pretrained(EMBEDDING_MODEL, cache_dir=cache_dir)
+    processor = AutoProcessor.from_pretrained(EMBEDDING_MODEL, cache_dir=cache_dir, use_fast=use_fast)
+    return model, processor
+
+
+def get_text_embeddings(
+    text: str | list[str],
+    model: EmbeddingModel,
+    processor: EmbeddingProcessor,
+    device: str | torch.device | None = None,
+) -> torch.Tensor:
+    """Embed one or more strings into the SigLIP-2 space.
+
+    Returns an L2-normalized `[N, 768]` CPU tensor (one row per input string).
+    """
+    if isinstance(text, str):
+        text = [text]
+    if not text:
+        raise ValueError("Input 'text' must be a non-empty string or list of strings.")
+
+    device = _resolve_device(device)
+    model = model.to(device)
+    model.eval()
+
+    # SigLIP is trained with a fixed 64-token sequence; it MUST be tokenized with
+    # padding="max_length" (max_length=64). With dynamic padding="True" the text
+    # embeddings are malformed and text->image retrieval collapses onto a hub image.
+    inputs = processor(text=text, return_tensors="pt", padding="max_length", max_length=64, truncation=True).to(device)
+    with torch.inference_mode():
+        # transformers 5.x: get_text_features returns the full encoder output, not a
+        # bare tensor — the embedding is its `pooler_output` (`[N, 768]`).
+        features = model.get_text_features(**inputs).pooler_output
+        normalized: torch.Tensor = features / features.norm(p=2, dim=-1, keepdim=True)
+        return normalized.cpu()
+
+
+def compute_image_embeddings(
+    images: list[Image],
+    model: EmbeddingModel,
+    processor: EmbeddingProcessor,
+    device: str | torch.device | None = None,
+    batch_size: int = 64,
+) -> torch.Tensor:
+    """Embed a list of PIL images into the SigLIP-2 space.
+
+    Returns an L2-normalized `[len(images), 768]` CPU tensor.
+    """
+    if not images:
+        raise ValueError("Input 'images' list cannot be empty.")
+
+    device = _resolve_device(device)
+    model = model.to(device)
+    model.eval()
+
+    all_embeddings: list[torch.Tensor] = []
+    with torch.inference_mode():
+        for start in range(0, len(images), batch_size):
+            batch = images[start : start + batch_size]
+            inputs = processor(images=batch, return_tensors="pt").to(device)
+            # transformers 5.x: get_image_features returns the full encoder output, not a
+            # bare tensor — the embedding is its `pooler_output` (`[N, 768]`).
+            features = model.get_image_features(**inputs).pooler_output
+            normalized = features / features.norm(p=2, dim=1, keepdim=True)
+            all_embeddings.append(normalized.cpu())
+
+    return torch.cat(all_embeddings, dim=0)

examples/python/droid_semantic_search/ingest.py

@@ -0,0 +1,330 @@
+"""Build a local vector index of DROID camera frames.
+
+For each requested camera the script auto-detects how to get embeddings:
+
+* **Read path** — if the dataset already has a `/camera/{role}/embedding` column
+  (DROID registered with `--create-embeddings`), read it straight out of the
+  catalog with a DataFusion query. No video decoding, no model needed.
+* **Compute path** — otherwise stream the H.264 `VideoStream` via the
+  experimental dataloader, decode frames, and embed them with SigLIP-2.
+
+Either way we end up with a columnar `(segment_id, camera, timestamp_ms, vector)`
+Arrow table, which we write to a local vector store (LanceDB or Qdrant, see
+`--backend`) and index for ANN search.
+
+Run inside the rerun SDK venv, e.g.:
+
+    pixi run uv run ../droid_semantic_search/ingest.py --num-segments 5 --cameras ext1
+"""
+
+from __future__ import annotations
+
+import argparse
+import itertools
+from collections.abc import Iterator
+from typing import Any
+
+import numpy as np
+import pyarrow as pa
+import pyarrow.compute as pc
+from vector_store import BACKENDS, DEFAULT_PATHS, open_store
+
+from rerun.catalog import CatalogClient
+
+# DROID camera roles, and the timeline everything is logged on.
+ALL_CAMERAS = ("wrist", "ext1", "ext2")
+TIMELINE = "real_time"
+
+# DROID is H.264, GOP size 64 at ~15 fps (see the droid-loader). These knobs are only a
+# fallback for episodes registered with `--no-optimize` (no keyframe markers): the decoder
+# then seeks by a fixed window, so we use 2x the GOP to make sure each window holds a keyframe.
+DROID_CODEC = "h264"
+DROID_KEYFRAME_INTERVAL = 128
+DROID_FPS_ESTIMATE = 15.0
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("--catalog-url", default="rerun+http://127.0.0.1:51234", help="Rerun catalog URL")
+    parser.add_argument("--dataset", default="droid:sample", help="Dataset name in the catalog")
+    parser.add_argument("--token", default=None, help="Auth token (if the catalog requires one)")
+    parser.add_argument(
+        "--cameras",
+        default="ext1",
+        help="Comma-separated camera roles, or 'all'. Exterior cams (ext1/ext2) give better scene-level matches.",
+    )
+    parser.add_argument("--num-segments", type=int, default=10, help="Number of segments to index (0 for all)")
+    parser.add_argument(
+        "--backend",
+        choices=BACKENDS,
+        default="lance",
+        help="Local vector store to write the index to.",
+    )
+    parser.add_argument(
+        "--db-path",
+        default=None,
+        help="Directory for the local vector DB (default: ./droid_lancedb or ./droid_qdrant per backend).",
+    )
+    parser.add_argument("--table", default="droid_frames", help="Table/collection name")
+    parser.add_argument(
+        "--rate-hz",
+        type=float,
+        default=2.0,
+        help="Compute-path only: frames per second to sample from each segment.",
+    )
+    parser.add_argument(
+        "--fetch-batch",
+        type=int,
+        default=32,
+        help="Compute-path only: samples fetched per server round-trip.",
+    )
+    parser.add_argument(
+        "--num-workers",
+        type=int,
+        default=0,
+        help="Compute-path only: DataLoader workers for fetching/decoding.",
+    )
+    return parser.parse_args()
+
+
+def resolve_cameras(arg: str) -> list[str]:
+    if arg.strip().lower() == "all":
+        return list(ALL_CAMERAS)
+    roles = [c.strip() for c in arg.split(",") if c.strip()]
+    unknown = [r for r in roles if r not in ALL_CAMERAS]
+    if unknown:
+        raise SystemExit(f"Unknown camera role(s) {unknown}; expected a subset of {ALL_CAMERAS} or 'all'.")
+    return roles
+
+
+def cameras_with_embeddings(schema: object, cameras: list[str]) -> set[str]:
+    """Return the subset of *cameras* that already have an embedding entity in the dataset."""
+    present = {p.strip("/") for p in schema.entity_paths()}  # type: ignore[attr-defined]
+    return {role for role in cameras if f"camera/{role}/embedding" in present}
+
+
+def _is_list_type(t: pa.DataType) -> bool:
+    return bool(pa.types.is_list(t) or pa.types.is_large_list(t) or pa.types.is_fixed_size_list(t))
+
+
+def _vector_dim(vectors: pa.Array) -> int:
+    """Embedding dimensionality of a (variable- or fixed-size) list array."""
+    if pa.types.is_fixed_size_list(vectors.type):
+        return int(vectors.type.list_size)
+    return int(pc.max(pc.list_value_length(vectors)).as_py())
+
+
+def _embedding_table(role: str, segment_ids: pa.Array, timestamps_ms: pa.Array, vectors: pa.Array) -> pa.Table:
+    """Assemble the four index columns into one Arrow table.
+
+    Both ingest paths funnel through here, so they share a single schema and
+    `pa.concat_tables` can stitch their results together with no per-row work.
+    """
+    return pa.table(
+        {
+            "segment_id": segment_ids,
+            "camera": pa.array([role] * len(segment_ids), pa.string()),
+            "timestamp_ms": timestamps_ms,
+            "vector": vectors,
+        },
+    )
+
+
+def _find_embedding_column(schema: pa.Schema, role: str) -> str:
+    """Locate the list-typed embedding column for *role* in a query result schema.
+
+    The DROID loader logs embeddings via `rr.AnyValues(embeddings=...)`, so the
+    exact column name (e.g. `/camera/ext1/embedding:embeddings`) is derived by
+    the platform — discover it rather than hardcoding.
+    """
+    candidates: list[str] = [f.name for f in schema if _is_list_type(f.type) and "embedding" in f.name.lower()]
+    if not candidates:
+        raise RuntimeError(f"No list-typed embedding column for camera '{role}'. Columns: {schema.names}")
+    for name in candidates:
+        if role in name:
+            return name
+    return candidates[0]
+
+
+def read_embedding_table(dataset: object, segments: list[str], role: str) -> pa.Table | None:
+    """Read pre-computed embeddings for *role* out of the catalog.
+
+    The query result is already columnar Arrow, so we stay in Arrow the whole
+    way — filter out missing rows and normalize the column types without ever
+    materializing Python row objects.
+    """
+    view = dataset.filter_segments(segments).filter_contents(  # type: ignore[attr-defined]
+        [f"/camera/{role}/embedding", f"/camera/{role}/embedding/**"],
+    )
+    table = view.reader(index=TIMELINE).to_arrow_table()
+
+    emb_col = _find_embedding_column(table.schema, role)
+    if "rerun_segment_id" not in table.schema.names or TIMELINE not in table.schema.names:
+        raise RuntimeError(f"Expected 'rerun_segment_id' and '{TIMELINE}' columns, got {table.schema.names}")
+
+    # Columnar equivalent of the per-row `if vec is None or seg is None: continue`.
+    keep = pc.and_(pc.is_valid(table.column(emb_col)), pc.is_valid(table.column("rerun_segment_id")))
+    table = table.filter(keep)
+    if table.num_rows == 0:
+        print(f"  [{role}] no pre-computed embeddings")
+        return None
+
+    vectors = table.column(emb_col).combine_chunks()
+    vectors = vectors.cast(pa.list_(pa.float32(), _vector_dim(vectors)))
+    segment_ids = table.column("rerun_segment_id").cast(pa.string())
+    # DROID's index timeline is nanosecond timestamps; LanceDB just wants an int.
+    timestamps_ms = table.column(TIMELINE).cast(pa.timestamp("ms")).cast(pa.int64())
+
+    out = _embedding_table(role, segment_ids, timestamps_ms, vectors)
+    print(f"  [{role}] read {out.num_rows} pre-computed embeddings")
+    return out
+
+
+def _identity_collate(batch: list[Any]) -> list[Any]:
+    """Collate that leaves the list of per-sample dicts untouched (picklable for workers)."""
+    return batch
+
+
+def compute_embedding_table(
+    dataset: object,
+    segments: list[str],
+    role: str,
+    *,
+    rate_hz: float,
+    fetch_batch: int,
+    num_workers: int,
+) -> pa.Table | None:
+    """Decode frames for *role* and embed them with SigLIP-2."""
+    # Heavy / optional deps are imported lazily so a pure read-path run stays light.
+    from embeddings import compute_image_embeddings, load_embedding_model
+    from PIL import Image
+    from torch.utils.data import DataLoader
+    from tqdm import tqdm
+
+    from rerun.experimental.dataloader import (
+        DataSource,
+        Field,
+        FixedRateSampling,
+        RerunMapDataset,
+        VideoFrameDecoder,
+    )
+
+    field_name = f"img_{role}"
+    source = DataSource(dataset, segments=segments)  # type: ignore[arg-type]
+    fields = {
+        field_name: Field(
+            f"/camera/{role}:VideoStream:sample",
+            decode=VideoFrameDecoder(
+                codec=DROID_CODEC,
+                keyframe_interval=DROID_KEYFRAME_INTERVAL,
+                fps_estimate=DROID_FPS_ESTIMATE,
+            ),
+        ),
+    }
+    ds = RerunMapDataset(
+        source=source,
+        index=TIMELINE,
+        fields=fields,
+        timeline_sampling=FixedRateSampling(rate_hz=rate_hz),
+    )
+    total = len(ds)
+    print(f"  [{role}] decoding ~{total} frames at {rate_hz} Hz …")
+
+    # The DataLoader earns its keep on the *fetch* side: `batch_size` batches the
+    # catalog round-trips, and `num_workers > 0` fans the CPU-bound video decode
+    # across worker processes. `shuffle=False` keeps indices in 0..N-1 order, which
+    # the running counter in `decoded_frames` relies on for the (segment, timestamp)
+    # pairing below.
+    loader = DataLoader(
+        ds,
+        batch_size=fetch_batch,
+        shuffle=False,
+        num_workers=num_workers,
+        collate_fn=_identity_collate,
+    )
+
+    def decoded_frames(pbar: tqdm[Any]) -> Iterator[tuple[Image.Image, str, int]]:
+        # The loader visits indices 0..N-1 in order, so a running counter pairs each
+        # decoded frame back to its (segment, timestamp) via `global_to_local`. The
+        # progress bar advances once per sample pulled from the loader (the slow,
+        # video-decoding step), including the ones we skip below.
+        global_idx = 0
+        for batch in loader:
+            for sample in batch:
+                tensor = sample[field_name]
+                seg_meta, idx_val = ds.sample_index.global_to_local(global_idx)
+                global_idx += 1
+                pbar.update(1)
+                if tensor is None:  # target preceded the first keyframe; skip
+                    continue
+                ts_ms = int(np.datetime64(idx_val).astype("datetime64[ms]").astype(np.int64))  # type: ignore[arg-type]
+                rgb = tensor.permute(1, 2, 0).cpu().numpy()  # [C,H,W] uint8 -> [H,W,C]
+                yield Image.fromarray(rgb), seg_meta.segment_id, ts_ms
+
+    # Embed the decoded stream chunk-by-chunk so peak memory stays at ~embed_batch
+    # frames rather than the whole role. Each chunk becomes one small Arrow table;
+    # `pa.concat_tables` stitches them at the end with no per-row work.
+    embed_batch = 64
+    model, processor = load_embedding_model()
+    chunks: list[pa.Table] = []
+    with tqdm(total=total, desc=f"[{role}] decode+embed", unit="frame") as pbar:
+        for chunk in itertools.batched(decoded_frames(pbar), embed_batch):  # type: ignore[attr-defined, unused-ignore]
+            frames = [frame for frame, _, _ in chunk]
+            segs = [seg for _, seg, _ in chunk]
+            timestamps = [ts_ms for _, _, ts_ms in chunk]
+            vectors = compute_image_embeddings(frames, model, processor, batch_size=embed_batch).numpy()
+            _, dim = vectors.shape
+            vector_col = pa.FixedSizeListArray.from_arrays(pa.array(vectors.reshape(-1), pa.float32()), dim)
+            chunks.append(
+                _embedding_table(role, pa.array(segs, pa.string()), pa.array(timestamps, pa.int64()), vector_col),
+            )
+
+    if not chunks:
+        print(f"  [{role}] no frames decoded")
+        return None
+
+    out = pa.concat_tables(chunks)
+    print(f"  [{role}] computed {out.num_rows} embeddings")
+    return out
+
+
+def main() -> None:
+    args = parse_args()
+    cameras = resolve_cameras(args.cameras)
+
+    client = CatalogClient(args.catalog_url, token=args.token)
+    dataset = client.get_dataset(args.dataset)
+
+    all_segments = dataset.segment_ids()
+    segments = all_segments if args.num_segments == 0 else all_segments[: args.num_segments]
+    if not segments:
+        raise SystemExit(f"Dataset '{args.dataset}' has no segments.")
+
+    have_emb = cameras_with_embeddings(dataset.schema(), cameras)
+    print(f"Indexing {len(segments)} segment(s); cameras={cameras}; pre-computed embeddings for {sorted(have_emb)}")
+
+    tables: list[pa.Table] = []
+    for role in cameras:
+        if role in have_emb:
+            table = read_embedding_table(dataset, segments, role)
+        else:
+            table = compute_embedding_table(
+                dataset,
+                segments,
+                role,
+                rate_hz=args.rate_hz,
+                fetch_batch=args.fetch_batch,
+                num_workers=args.num_workers,
+            )
+        if table is not None:
+            tables.append(table)
+
+    if not tables:
+        raise SystemExit("No embeddings produced; nothing to index.")
+
+    db_path = args.db_path or DEFAULT_PATHS[args.backend]
+    open_store(args.backend, db_path, args.table).write(pa.concat_tables(tables))
+
+
+if __name__ == "__main__":
+    main()

examples/python/droid_semantic_search/prepare_dataset.py

@@ -0,0 +1,222 @@
+"""Register a few DROID episodes to a local Rerun catalog so the rest of the example has data to index.
+
+Two sources, auto-selected (override with `--source`):
+
+* **Bundled** — the `tests/assets/rrd/sample_5` episodes shipped in the Rerun repo (via git-LFS).
+  Used automatically in a monorepo checkout: no download, works offline.
+* **Hugging Face** — a few episodes from the [`rerun/droid_sample`](https://huggingface.co/datasets/rerun/droid_sample)
+  dataset, downloaded into `./data`. Used when the bundled episodes aren't available
+  (e.g. a standalone sparse-checkout of just this example).
+
+Either way the episodes are registered to the catalog as a dataset (default name `droid:sample`).
+They carry H.264 `VideoStream`s but no pre-computed embeddings, so `ingest.py` will take its
+(slower) compute path: decode frames and embed them with SigLIP-2.
+
+Episodes are optimized first to derive keyframe markers, so the compute path can decode
+frames (DROID doesn't log the markers the decoder needs). Pass `--no-optimize` to skip.
+
+Run inside the rerun SDK venv, with a `rerun server` running in another terminal, e.g.:
+
+    uv run python prepare_dataset.py
+"""
+
+from __future__ import annotations
+
+import argparse
+from dataclasses import replace
+from pathlib import Path
+
+from tqdm import tqdm
+
+import rerun as rr
+from rerun.experimental import OptimizationProfile, RrdReader
+
+# `tests/assets/rrd/sample_5`, relative to this file at `examples/python/droid_semantic_search/`.
+BUNDLED_SAMPLE_DIR = Path(__file__).resolve().parents[3] / "tests" / "assets" / "rrd" / "sample_5"
+DEFAULT_REPO_ID = "rerun/droid_sample"
+DEFAULT_OUTPUT_DIR = Path(__file__).resolve().parent / "data"
+DEFAULT_OPTIMIZED_DIR = Path(__file__).resolve().parent / "optimized"
+DEFAULT_DATASET_NAME = "droid:sample"
+DEFAULT_CATALOG_URL = "rerun+http://127.0.0.1:51234"
+
+_LFS_POINTER_PREFIX = b"version https://git-lfs.github.com/spec/v1"
+
+
+def _is_lfs_pointer(path: Path) -> bool:
+    """True if *path* is an un-pulled git-LFS pointer file rather than the real RRD."""
+    with path.open("rb") as f:
+        return f.read(len(_LFS_POINTER_PREFIX)) == _LFS_POINTER_PREFIX
+
+
+def bundled_episode_paths() -> list[Path]:
+    """Return the bundled sample_5 RRDs (sorted), or an empty list if they're not available.
+
+    Raises if the directory exists but the files are un-pulled git-LFS pointers — that's a
+    recoverable monorepo setup issue worth surfacing rather than silently working around.
+    """
+    if not BUNDLED_SAMPLE_DIR.is_dir():
+        return []
+    rrds = sorted(BUNDLED_SAMPLE_DIR.glob("*.rrd"))
+    if not rrds:
+        return []
+    if any(_is_lfs_pointer(p) for p in rrds):
+        raise SystemExit(
+            f"The bundled DROID episodes in {BUNDLED_SAMPLE_DIR} are un-pulled git-LFS pointers.\n"
+            "Fetch them with `git lfs install && git lfs pull`, or pass `--source huggingface` to download instead.",
+        )
+    return rrds
+
+
+def download_episodes(repo_id: str, num_episodes: int, dest: Path) -> list[Path]:
+    """Download the first *num_episodes* (0 for all) `.rrd` files from *repo_id* into *dest*.
+
+    `snapshot_download` renders its own per-file progress bars, so the user sees the download advance.
+    """
+    from huggingface_hub import HfApi, snapshot_download
+
+    files = sorted(f for f in HfApi().list_repo_files(repo_id, repo_type="dataset") if f.endswith(".rrd"))
+    if not files:
+        raise SystemExit(f"No .rrd files found in '{repo_id}'.")
+    files = files if num_episodes == 0 else files[:num_episodes]
+
+    print(f"Downloading {len(files)} episode(s) from '{repo_id}' to {dest} …")
+    local_dir = snapshot_download(repo_id=repo_id, repo_type="dataset", allow_patterns=files, local_dir=dest)
+    return [Path(local_dir) / f for f in files]
+
+
+def resolve_episodes(source: str, *, num_episodes: int, repo_id: str, output_dir: Path) -> list[Path]:
+    """Pick the episode RRDs to register, per the requested *source* (`auto`/`bundled`/`huggingface`)."""
+    if source in ("auto", "bundled"):
+        bundled = bundled_episode_paths()
+        if bundled:
+            paths = bundled if num_episodes == 0 else bundled[:num_episodes]
+            print(f"Using {len(paths)} bundled episode(s) from {BUNDLED_SAMPLE_DIR}")
+            return paths
+        if source == "bundled":
+            raise SystemExit(f"No bundled episodes found at {BUNDLED_SAMPLE_DIR}; pass `--source huggingface`.")
+        print(f"Bundled episodes not found at {BUNDLED_SAMPLE_DIR}; downloading from the Hub instead.")
+
+    return download_episodes(repo_id, num_episodes, output_dir)
+
+
+def optimize_episodes(rrd_paths: list[Path], dest_dir: Path) -> list[Path]:
+    """Derive `VideoStream:is_keyframe` markers (DROID doesn't log them) so the decoder can seek.
+
+    Writes a fixed copy of each episode to *dest_dir* and returns the new paths. IDs are
+    preserved, so catalog segment IDs and `segment_url` links are unchanged.
+    """
+    dest_dir.mkdir(parents=True, exist_ok=True)
+    profile = replace(OptimizationProfile.OBJECT_STORE, fix_keyframe=True)
+
+    optimized: list[Path] = []
+    for src in tqdm(rrd_paths, desc="Optimizing", unit="episode"):
+        reader = RrdReader(src)
+        recordings = reader.recordings()
+        if len(recordings) != 1:
+            raise SystemExit(f"Expected one recording in {src}, found {len(recordings)}.")
+        entry = recordings[0]
+
+        store = reader.stream(store=entry).collect(optimize=profile)
+        dst = dest_dir / src.name
+        store.write_rrd(dst, application_id=entry.application_id, recording_id=entry.recording_id)
+        optimized.append(dst)
+
+    return optimized
+
+
+def register_to_catalog(rrd_paths: list[Path], *, catalog_url: str, dataset_name: str) -> None:
+    """Register per-episode RRDs to a catalog server instance.
+
+    Uses absolute `file://` URIs so the catalog can read the RRDs directly from the local filesystem.
+    Streams `iter_results()` so a progress bar advances as each segment finishes, rather than blocking
+    silently on `wait()`.
+    """
+    print(f"\nRegistering {len(rrd_paths)} episode(s) to {catalog_url} as dataset '{dataset_name}' …")
+    client = rr.catalog.CatalogClient(catalog_url)
+    dataset = client.create_dataset(dataset_name, exist_ok=True)
+
+    uris = [f"file://{p.resolve()}" for p in rrd_paths]
+    on_duplicate = rr.catalog.OnDuplicateSegmentLayer(rr.catalog.OnDuplicateSegmentLayer.REPLACE)
+    handle = dataset.register(uris, on_duplicate=on_duplicate)
+
+    failures: list[str] = []
+    for result in tqdm(handle.iter_results(), total=len(uris), desc="Registering", unit="segment"):
+        if result.is_error:
+            failures.append(f"{result.uri}: {result.error}")
+
+    if failures:
+        joined = "\n  ".join(failures)
+        raise SystemExit(f"Failed to register {len(failures)} of {len(uris)} episode(s):\n  {joined}")
+    print("  registration done")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument(
+        "--source",
+        choices=("auto", "bundled", "huggingface"),
+        default="auto",
+        help="Where to get episodes: 'bundled' (in-repo sample_5), 'huggingface' (download), "
+        "or 'auto' (bundled if available, else download). Default: auto.",
+    )
+    parser.add_argument(
+        "--repo-id",
+        default=DEFAULT_REPO_ID,
+        help=f"Hugging Face dataset repo id, for the download path (default: {DEFAULT_REPO_ID}).",
+    )
+    parser.add_argument(
+        "--num-episodes",
+        type=int,
+        default=5,
+        help="Number of episodes to register (0 for all). The full Hub dataset is ~3.3 GB.",
+    )
+    parser.add_argument(
+        "--output-dir",
+        type=Path,
+        default=DEFAULT_OUTPUT_DIR,
+        help=f"Directory to download episode RRDs into, for the download path (default: {DEFAULT_OUTPUT_DIR}).",
+    )
+    parser.add_argument(
+        "--optimize",
+        default=True,
+        action=argparse.BooleanOptionalAction,
+        help="Derive keyframe markers before registering, so ingest.py can decode frames "
+        "(~100%% yield vs ~25%%). Use --no-optimize to register episodes as-is.",
+    )
+    parser.add_argument(
+        "--optimized-dir",
+        type=Path,
+        default=DEFAULT_OPTIMIZED_DIR,
+        help=f"Directory to write optimized episode RRDs into (default: {DEFAULT_OPTIMIZED_DIR}).",
+    )
+    parser.add_argument(
+        "--catalog-url",
+        default=DEFAULT_CATALOG_URL,
+        help="Rerun catalog URL to register episodes with. Pass an empty string to skip registration.",
+    )
+    parser.add_argument(
+        "--dataset-name",
+        default=DEFAULT_DATASET_NAME,
+        help=f"Name of the dataset to create/use in the catalog (default: {DEFAULT_DATASET_NAME}).",
+    )
+    args = parser.parse_args()
+
+    rrd_paths = resolve_episodes(
+        args.source,
+        num_episodes=args.num_episodes,
+        repo_id=args.repo_id,
+        output_dir=args.output_dir,
+    )
+
+    if args.optimize:
+        print(f"\nOptimizing {len(rrd_paths)} episode(s) into {args.optimized_dir} …")
+        rrd_paths = optimize_episodes(rrd_paths, args.optimized_dir)
+
+    if args.catalog_url:
+        register_to_catalog(rrd_paths, catalog_url=args.catalog_url, dataset_name=args.dataset_name)
+    else:
+        print(f"Skipping registration (empty --catalog-url). Episodes: {[str(p) for p in rrd_paths]}")
+
+
+if __name__ == "__main__":
+    main()

examples/python/droid_semantic_search/pyproject.toml

@@ -0,0 +1,66 @@
+[project]
+name = "droid_semantic_search"
+version = "0.1.0"
+readme = "README.md"
+requires-python = ">=3.12,<3.13"
+dependencies = [
+  # `catalog` brings datafusion + pandas; `dataloader` brings torch, torchvision, av, pillow.
+  "rerun-sdk[catalog,dataloader]",
+  "lancedb", # default local vector store + ANN index (--backend lance)
+  "qdrant-client", # alternative local vector store (--backend qdrant)
+  "transformers>=5.13.0", # SigLIP-2 model + processor
+  "pyarrow<24", # building the vector-index table
+  "huggingface-hub", # prepare_dataset.py: download DROID sample episodes from the Hub
+  "tqdm", # progress bars for download/register/ingest
+]
+
+[dependency-groups]
+dev = ["mypy==1.19.1", "types-tqdm"]
+
+[tool.rerun-example]
+# Picked up by scripts/ci/isolated_examples.py and the `py-lint-isolated-examples` pixi task.
+isolated = true
+
+[tool.uv]
+# The example is flat scripts, not a wheel — skip project build, just sync deps.
+package = false
+
+# Default `uv sync` uses the in-repo editable rerun-sdk (monorepo dev mode).
+# After syncing, also run `uv pip install ../../../rerun_py/rerun_dev_fixup` to
+# install the .pth shim that makes `import rerun` resolve to the editable source tree.
+#
+# Standalone users (e.g. sparse-checkout of just this example) run instead:
+#   uv sync --no-sources --no-dev
+# That ignores the path source below and resolves `rerun-sdk` from PyPI.
+# rerun-dev-fixup is intentionally absent from this file: uv 0.7.x resolves all
+# dependency groups and extras unconditionally, so any path-only package here would
+# block standalone `--no-sources` resolution.
+[tool.uv.sources]
+rerun-sdk = { path = "../../../rerun_py", editable = true }
+
+# Merged onto the shared base at `../_isolated/mypy.ini` by
+# scripts/ci/isolated_examples.py — list the untyped third-party libs this
+# example actually imports.
+[[tool.mypy.overrides]]
+module = [
+  "lancedb.*",
+  "qdrant_client.*",
+  "torch.*",
+  "torchvision.*",
+  "av.*",
+  "PIL.*",
+  "huggingface_hub.*",
+  # pyarrow is pinned <24 (24.0.0 segfaults rerun on import); 23.x ships no py.typed,
+  # so all of pyarrow — including pyarrow.compute — is untyped to mypy.
+  "pyarrow.*",
+]
+ignore_missing_imports = true
+
+# transformers ships a `py.typed` marker but with incomplete stubs, so type-checking its
+# internals yields false positives we can't fix from here: model methods are wrapped in
+# `_Wrapped` descriptors that mistype `self` (`AutoModel has no attribute "to"`,
+# `AutoProcessor not callable`, etc.). Skip following it — we only need our own usage typed.
+[[tool.mypy.overrides]]
+module = ["transformers.*"]
+follow_imports = "skip"
+ignore_missing_imports = true

examples/python/droid_semantic_search/search.py

@@ -0,0 +1,128 @@
+"""Query the DROID frame index with a text prompt or example image and open the best hit in Rerun.
+
+Embeds the query with the SigLIP-2 text *or* image encoder (both share one
+vector space, the same one the indexed frame embeddings live in), runs a cosine
+nearest-neighbor search over the local vector store (LanceDB or Qdrant, see
+`--backend`), prints the ranked matches, and mints a `segment_url` deep-link
+that opens the top result focused on that frame in the Rerun viewer.
+
+Run inside the rerun SDK venv, e.g.:
+
+    pixi run uv run ../droid_semantic_search/search.py "a robot gripper reaching for a cup"
+    pixi run uv run ../droid_semantic_search/search.py --image ./query.jpg
+"""
+
+from __future__ import annotations
+
+import argparse
+import webbrowser
+from datetime import datetime, timedelta, timezone
+from typing import cast
+
+from embeddings import (
+    EmbeddingModel,
+    EmbeddingProcessor,
+    compute_image_embeddings,
+    get_text_embeddings,
+    load_embedding_model,
+)
+from vector_store import BACKENDS, DEFAULT_PATHS, open_store
+
+import rerun as rr
+from rerun.catalog import CatalogClient
+
+TIMELINE = "real_time"
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("query", nargs="?", help="Text prompt to search for")
+    parser.add_argument(
+        "--image", help="Path to an image to search by (image-to-image); mutually exclusive with the text query"
+    )
+    parser.add_argument("--catalog-url", default="rerun+http://127.0.0.1:51234", help="Rerun catalog URL")
+    parser.add_argument("--dataset", default="droid:sample", help="Dataset name (used to mint the viewer link)")
+    parser.add_argument(
+        "--login", action="store_true", help="Authenticate with the catalog via rr.login() before connecting"
+    )
+    parser.add_argument(
+        "--backend",
+        choices=BACKENDS,
+        default="lance",
+        help="Local vector store to query (must match what ingest.py wrote).",
+    )
+    parser.add_argument(
+        "--db-path",
+        default=None,
+        help="Directory of the local vector DB (default: ./droid_lancedb or ./droid_qdrant per backend).",
+    )
+    parser.add_argument("--table", default="droid_frames", help="Table/collection name")
+    parser.add_argument("--top-k", type=int, default=5, help="Number of matches to return")
+    parser.add_argument("--window-secs", type=float, default=2.0, help="Time window around the matched frame to show")
+    parser.add_argument(
+        "--open",
+        default=True,
+        action=argparse.BooleanOptionalAction,
+        help="Open the top hit in the viewer",
+    )
+    return parser.parse_args()
+
+
+def embed_image_query(path: str, model: EmbeddingModel, processor: EmbeddingProcessor) -> list[float]:
+    """Embed an image file for image-to-image search.
+
+    Wired to the `--image` flag: text and image features share one SigLIP-2
+    vector space, so a query image retrieves frames the same way a text prompt does.
+    """
+    from PIL import Image
+
+    image = Image.open(path).convert("RGB")
+    vector = compute_image_embeddings([image], model, processor).numpy()[0]
+    return cast("list[float]", vector.tolist())
+
+
+def viewer_url(dataset: object, segment_id: str, timestamp_ms: int, window_secs: float) -> str:
+    ts = datetime.fromtimestamp(timestamp_ms / 1000, tz=timezone.utc)
+    half = timedelta(seconds=window_secs / 2)
+    return dataset.segment_url(segment_id, timeline=TIMELINE, start=ts - half, end=ts + half)  # type: ignore[attr-defined, no-any-return]
+
+
+def main() -> None:
+    args = parse_args()
+
+    if bool(args.query) == bool(args.image):
+        raise SystemExit("Provide exactly one of: a text query or --image <path>.")
+
+    model, processor = load_embedding_model()
+    if args.image:
+        query_vec = embed_image_query(args.image, model, processor)
+        query_label = f"image {args.image}"
+    else:
+        query_vec = get_text_embeddings(args.query, model, processor).numpy()[0].tolist()
+        query_label = args.query
+
+    db_path = args.db_path or DEFAULT_PATHS[args.backend]
+    hits = open_store(args.backend, db_path, args.table).search(query_vec, args.top_k)
+    if not hits:
+        raise SystemExit("No matches found — is the index populated? Run ingest.py first.")
+
+    print(f'\nTop {len(hits)} matches for: "{query_label}"\n')
+    print(f"{'#':>2}  {'sim':>5}  {'camera':<6}  {'timestamp (UTC)':<24}  segment")
+    for rank, hit in enumerate(hits, start=1):
+        ts_iso = datetime.fromtimestamp(hit["timestamp_ms"] / 1000, tz=timezone.utc).isoformat(timespec="milliseconds")
+        print(f"{rank:>2}  {hit['similarity']:>5.3f}  {hit['camera']:<6}  {ts_iso:<24}  {hit['segment_id']}")
+
+    # Mint a deep-link into the viewer for the best match.
+    if args.login:
+        rr.login()
+    client = CatalogClient(args.catalog_url)
+    dataset = client.get_dataset(args.dataset)
+    best = hits[0]
+    url = viewer_url(dataset, best["segment_id"], best["timestamp_ms"], args.window_secs)
+    print(f"\nTop hit in viewer:\n{url}")
+    if args.open:
+        webbrowser.open(url)
+
+
+if __name__ == "__main__":
+    main()

examples/python/droid_semantic_search/vector_store.py

@@ -0,0 +1,161 @@
+"""Pluggable local vector-store backends for the DROID frame index.
+
+`ingest.py` and `search.py` talk to a vector store only through the small
+`VectorStore` interface here, so the same code path works whether you pick
+LanceDB (`--backend lance`) or Qdrant (`--backend qdrant`). Both run fully
+locally on disk — no server to stand up — so the example stays one-command.
+Adding another store (Pinecone, pgvector, Milvus, …) is a third subclass.
+
+The index is written from the columnar `(segment_id, camera, timestamp_ms,
+vector)` Arrow table that `ingest.py` assembles. Searches return hits carrying
+those three metadata fields plus a `similarity` in `[-1, 1]` (cosine),
+normalized here so callers never have to know which backend's distance/score
+convention is in play.
+"""
+
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+from typing import TYPE_CHECKING, Any
+
+if TYPE_CHECKING:
+    import pyarrow as pa
+
+BACKENDS = ("lance", "qdrant")
+
+# Per-backend default on-disk location, used when `--db-path` is omitted.
+DEFAULT_PATHS = {
+    "lance": "./droid_lancedb",
+    "qdrant": "./droid_qdrant",
+}
+
+
+class VectorStore(ABC):
+    """A local on-disk vector index over `(segment_id, camera, timestamp_ms, vector)` rows."""
+
+    @abstractmethod
+    def write(self, table: pa.Table) -> None:
+        """(Over)write the index from *table*, replacing any existing table/collection.
+
+        *table* has columns `segment_id` (string), `camera` (string),
+        `timestamp_ms` (int64), and `vector` (fixed-size list of float32).
+        """
+
+    @abstractmethod
+    def search(self, vector: list[float], top_k: int) -> list[dict[str, Any]]:
+        """Cosine nearest-neighbor search.
+
+        Returns up to *top_k* hit dicts, each with `segment_id`, `camera`,
+        `timestamp_ms`, and a cosine `similarity` (higher is closer).
+        """
+
+
+def open_store(backend: str, path: str, table: str) -> VectorStore:
+    if backend == "lance":
+        return LanceStore(path, table)
+    if backend == "qdrant":
+        return QdrantStore(path, table)
+    raise ValueError(f"Unknown backend '{backend}'; expected one of {BACKENDS}.")
+
+
+class LanceStore(VectorStore):
+    """LanceDB backend. Returns cosine *distance*, which we flip to similarity."""
+
+    def __init__(self, path: str, table: str) -> None:
+        self._path = path
+        self._table = table
+
+    def write(self, table: pa.Table) -> None:
+        import lancedb
+
+        dim = table.schema.field("vector").type.list_size
+
+        db = lancedb.connect(self._path)
+        tbl = db.create_table(self._table, data=table, mode="overwrite")
+        print(f"Wrote {table.num_rows} rows ({dim}-dim) to LanceDB table '{self._table}' in {self._path}")
+
+        # An ANN index needs enough rows to train; small demo tables fall back to
+        # brute-force search, which is exact and plenty fast at this scale.
+        try:
+            tbl.create_index(metric="cosine", vector_column_name="vector")
+            print("Built ANN index (cosine).")
+        except Exception as exc:
+            print(f"Skipped ANN index ({exc}); brute-force cosine search will be used.")
+
+    def search(self, vector: list[float], top_k: int) -> list[dict[str, Any]]:
+        import lancedb
+
+        tbl = lancedb.connect(self._path).open_table(self._table)
+        hits = tbl.search(vector).metric("cosine").limit(top_k).to_list()
+        return [
+            {
+                "segment_id": h["segment_id"],
+                "camera": h["camera"],
+                "timestamp_ms": h["timestamp_ms"],
+                "similarity": 1.0 - float(h["_distance"]),  # cosine distance -> similarity
+            }
+            for h in hits
+        ]
+
+
+class QdrantStore(VectorStore):
+    """Qdrant backend in local (embedded) mode. Returns cosine *score* directly."""
+
+    def __init__(self, path: str, collection: str) -> None:
+        self._path = path
+        self._collection = collection
+
+    def write(self, table: pa.Table) -> None:
+        from qdrant_client import QdrantClient, models
+
+        dim = table.schema.field("vector").type.list_size
+        segment_ids = table.column("segment_id").to_pylist()
+        cameras = table.column("camera").to_pylist()
+        timestamps_ms = table.column("timestamp_ms").to_pylist()
+        vectors = table.column("vector").to_pylist()
+
+        client = QdrantClient(path=self._path)
+
+        # Mirror Lance's overwrite semantics: drop any prior collection first.
+        if client.collection_exists(self._collection):
+            client.delete_collection(self._collection)
+        client.create_collection(
+            collection_name=self._collection,
+            vectors_config=models.VectorParams(size=dim, distance=models.Distance.COSINE),
+        )
+
+        points = [
+            models.PointStruct(
+                id=i,
+                vector=vectors[i],
+                payload={
+                    "segment_id": segment_ids[i],
+                    "camera": cameras[i],
+                    "timestamp_ms": timestamps_ms[i],
+                },
+            )
+            for i in range(table.num_rows)
+        ]
+        client.upsert(collection_name=self._collection, points=points)
+        print(f"Wrote {table.num_rows} rows ({dim}-dim) to Qdrant collection '{self._collection}' in {self._path}")
+
+    def search(self, vector: list[float], top_k: int) -> list[dict[str, Any]]:
+        from qdrant_client import QdrantClient
+
+        client = QdrantClient(path=self._path)
+        result = client.query_points(
+            collection_name=self._collection,
+            query=vector,
+            limit=top_k,
+            with_payload=True,
+        )
+        return [
+            {
+                "segment_id": payload["segment_id"],
+                "camera": payload["camera"],
+                "timestamp_ms": payload["timestamp_ms"],
+                "similarity": float(point.score),  # Qdrant cosine score is already a similarity
+            }
+            for point in result.points
+            if (payload := point.payload) is not None
+        ]