langbot-app--langbot/docs/VALKEY_SEARCH_INTEGRATION.md

Valkey Search Vector Database Integration

This document describes how to use Valkey Search (the search/vector module bundled in valkey/valkey-bundle) as the vector database backend for LangBot's knowledge base (RAG) feature.

What is Valkey Search?

Valkey Search is a module that adds vector similarity search and full-text search to Valkey, the open-source, BSD-licensed in-memory data store forked from Redis OSS. It is distributed in the valkey/valkey-bundle image alongside other modules (JSON, Bloom, LDAP).

LangBot talks to Valkey through the official valkey-glide client (Rust core + async Python wrapper), using its native ft (search) command namespace.

Key Features

• Vector search: ANN via HNSW or exact via FLAT, with COSINE / L2 / IP distance metrics
• Full-text search: term, prefix and phrase matching over indexed text fields
• Hybrid search: a metadata/text filter pre-selects candidates, then KNN ranks them
• In-memory speed: vectors and documents are stored as Valkey HASH keys
• Auth + TLS: optional username/password and TLS for production (toB / SaaS) deployments

Licensing

• Valkey core and the Search module are BSD-3-Clause.
• The valkey-glide client is Apache-2.0.

Both are compatible with LangBot.

Installation

Valkey Search support is included automatically on Linux and macOS. The official valkey-glide client does not currently publish a Windows package, so LangBot skips this optional dependency on Windows; LangBot remains usable there, but the Valkey Search backend is unavailable. To install the client manually on a supported platform:

pip install 'valkey-glide>=2.4.1,<3.0.0'

You also need a running Valkey server with the Search module loaded. The simplest way is the bundled image:

# Run valkey-bundle (includes the Search module) on host port 6380
podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
# (docker run ... works identically)

valkey-bundle ships multi-arch images (linux/amd64 + linux/arm64), so it runs on both CI (x86_64) and Apple-silicon dev machines.

Configuration

Valkey Search is opt-in and disabled by default — the default vdb.use stays chroma, so existing single-process deployments are unaffected. To enable it, edit your config.yaml:

vdb:
  use: valkey_search
  valkey_search:
    host: 'localhost'
    port: 6379            # use 6380 if you started the container as shown above
    db: 0
    password: ''          # optional (ACL / requirepass) — never logged
    username: ''          # optional (ACL user)
    tls: false            # optional (toB / SaaS)
    index_algorithm: 'HNSW'   # HNSW | FLAT
    distance_metric: 'COSINE' # COSINE | L2 | IP
    request_timeout: 5000     # per-request timeout in ms

Option	Default	Description
host	localhost	Valkey host
port	6379	Valkey port
db	0	Logical database id
password	''	Optional auth password (empty = no auth). Never logged.
username	''	Optional ACL username. Configuring a username without a password fails closed (raises) rather than connecting unauthenticated.
tls	false	Enable TLS for the connection
index_algorithm	HNSW	HNSW (approximate) or FLAT (exact)
distance_metric	COSINE	COSINE, L2, or IP
request_timeout	5000	Per-request timeout in milliseconds. The valkey-glide default (250ms) is too low for vector KNN under load; raise it further for remote/cross-AZ Valkey.

Connection behavior

The backend uses a lazy connection (lazy_connect=True): the client is created on first use and the connection is deferred to the first command. A misconfigured or unreachable Valkey server therefore does not block LangBot from booting — knowledge-base operations will error at call time instead, and you can recover by switching vdb.use back to another backend.

The connection sets a fixed client_name of langbot_vector_client so it is identifiable in CLIENT LIST and monitoring dashboards.

Supported search types

Type	Behavior
vector	Pure KNN over the embedding field
full_text	Term/phrase match over the indexed document text field
hybrid	Metadata/text filter pre-selects candidates, then KNN ranks them

⚠️ Important: vector_weight is NOT honored

Valkey Search hybrid queries follow a filter-then-KNN model: the filter (and/or full-text clause) narrows the candidate set, and the KNN stage ranks the survivors by vector distance. There is no native weighted score fusion (unlike, e.g., SeekDB's RRF boost).

For interface compatibility the backend still accepts a vector_weight argument, but it is ignored — passing different weights does not change result ordering. The first time a non-default weight is supplied, the backend logs a one-time warning.

If weighted hybrid ranking is needed in the future, it can be added application-side (run vector KNN and full-text search separately and blend the scores). That is intentionally out of scope for this integration.

Metadata & filtering

Documents are stored as Valkey HASH keys under the prefix kb:{collection}:{id} with fields:

• vector — the embedding, packed as little-endian FLOAT32
• document — the raw text (indexed as TEXT for full-text/hybrid search)
• file_id — promoted to an indexed TAG field so it is filterable
• metadata_json — the full metadata dict, preserved verbatim as JSON

Only indexed fields are filterable. Currently that is file_id. Filters referencing non-indexed metadata keys are dropped with a warning (the same pragmatism used by the Milvus and pgvector backends). All other metadata still round-trips intact via metadata_json.

Supported filter operators (canonical Chroma-style where syntax): $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin. Multiple top-level keys are AND-ed.

Testing

Unit tests (filter mapping, float32 packing, reply parsing, import guard) run in the fast lane with no server:

uv run pytest tests/unit_tests/vector/test_valkey_search_filter.py -q

Integration tests are slow-gated on TEST_VALKEY_URL and require a running server:

podman run -d --name valkey-test-langbot -p 6380:6379 valkey/valkey-bundle:9.1.0
TEST_VALKEY_URL=valkey://localhost:6380 \
    uv run pytest tests/integration/vector/test_valkey_search.py -m slow -q

The default upstream fast CI lane (-m "not slow") skips these, matching the existing PostgreSQL migration-test precedent.

Troubleshooting

Symptom	Cause / fix
Tests skip with "Valkey Search module not available"	The server is plain Valkey without the Search module. Use the valkey/valkey-bundle image.
ConnectionError at call time	Check host/port/auth; remember lazy_connect defers errors to first use.
Empty search results right after insert	The Search indexer is asynchronous; results become visible within a short delay. The integration tests poll/retry to account for this.
Hybrid ranking ignores vector_weight	Expected — see the caveat above.

Production considerations

• Cluster mode: Valkey Search in cluster mode uses an additional coordination port. This integration targets standalone mode; cluster support is a future consideration.
• Persistence: configure Valkey RDB/AOF persistence if the knowledge base must survive restarts; otherwise an in-memory store is ephemeral.
• Security: set password/username and tls: true for any non-local deployment. Credentials are never written to logs.