Dynamo + LMCache MP integration

Recipes for running NVIDIA Dynamo vLLM serving with KV cache offloaded to an LMCache multiprocess (mp) server, in both aggregated and disaggregated modes.

In mp mode LMCache runs as an out-of-process cache engine. vLLM workers attach to it through the LMCacheMPConnector and share KV tensors over CUDA IPC, so sharedMemory is disabled (LMCache owns /dev/shm) and hostIPC is enabled.

Layout

Path	What it is
`deploy/lmcache_engine.yaml`	`LMCacheEngine` CR for the shared MP server. Apply before the workers.
`deploy/agg_lmcache_mp.yaml`	Kubernetes `DynamoGraphDeployment`, aggregated (single worker).
`deploy/disagg_lmcache_mp.yaml`	Kubernetes `DynamoGraphDeployment`, disaggregated (prefill + decode workers).
`launch/agg_lmcache_mp.sh`	Local single-node launch script, aggregated (1 GPU).
`launch/disagg_lmcache_mp.sh`	Local single-node launch script, disaggregated (2 GPUs).

The LMCacheEngine (shared prerequisite)

deploy/lmcache_engine.yaml declares the MP server that both the aggregated and disaggregated workers attach to. The LMCache operator reconciles the CR into:

a per-node manager DaemonSet pod (the LMCache server that holds the CPU KV pool), and
a ConfigMap named <metadata.name>-connection (i.e. lmcache-mp-connection), which the vLLM workers mount at /etc/lmcache and read as --kv-transfer-config.

It must live in the same namespace as the workers (default) so the connection ConfigMap is created where the workers can mount it.

Version pinning: the server's bundled lmcache must be wire-compatible with the worker's. The worker image vllm-runtime:1.2.0-deepseek-v4-cuda13-dev.3 ships lmcache 0.4.4 (vLLM 0.20.1), and the recipe pairs it with the guide-validated server build nightly-2026-04-25 (lmcache 0.4.5.dev31, a pre-stable build wire-compatible with that worker). Replace my-tag in each manifest accordingly.

Usage

The deploy/ manifests are applied with kubectl against a cluster that already has the Dynamo platform and the LMCache operator installed. Apply the LMCacheEngine first, then one of the worker manifests:

kubectl apply -n default -f deploy/lmcache_engine.yaml
kubectl apply -n default -f deploy/disagg_lmcache_mp.yaml   # or agg_lmcache_mp.yaml

See the Kubernetes deployment guide for the full step-by-step (platform install, operator, namespace, HF token Secret, then kubectl apply).

The launch/ scripts are the single-node local equivalents. They run inside the Dynamo vllm-runtime container and source Dynamo's examples/common/{gpu_utils,launch_utils}.sh helpers, so run them from a Dynamo checkout (or copy them into examples/backends/vllm/launch/).

The deploy manifests pin the worker image to my-tag and the LMCacheEngine image separately; keep the two wire-compatible (see the validated pairings in the deployment guide).