vllm-project--vllm-omni/examples/online_serving/dreamzero

DreamZero

DreamZero is the robot-policy serving example for the OpenPI-compatible /v1/realtime/robot/openpi endpoint.

Supported checkpoints

Model	HuggingFace repo
DreamZero-DROID	GEAR-Dreams/DreamZero-DROID

Quick start

Start the server

bash examples/online_serving/dreamzero/run_server.sh

By default this launches:

vllm serve GEAR-Dreams/DreamZero-DROID --omni --port 8000 \
    --served-model-name dreamzero-droid \
    --deploy-config vllm_omni/deploy/dreamzero_tp1_cfg2.yaml \
    --enforce-eager --disable-log-stats

Override MODEL, PORT, HOST, DEPLOY_CONFIG, or SERVED_MODEL_NAME through the script environment if needed.

Install example dependencies

The core pip install -e . setup does not include the extra packages used by the DreamZero example scripts.

• openpi_client.py: openpi-client, websockets, opencv-python
• droid_sim_eval_client.py: mediapy, websockets, openpi-client

The DROID sim-eval script also needs an Isaac Lab environment that provides isaaclab, isaaclab_tasks, sim_evals, and gymnasium.

If you run the DROID client on Python < 3.12, also install typing-extensions.

Configure TP and CFG parallelism

The bundled DreamZero configs intentionally keep only:

Config	Purpose
vllm_omni/deploy/dreamzero.yaml	Default TP=1, CFG parallel disabled
vllm_omni/deploy/dreamzero_tp1_cfg2.yaml	TP=1, CFG parallel size=2

For other topologies, use CLI parallelism flags and update stage 0 devices with --stage-overrides. The number of listed devices must match tensor_parallel_size * cfg_parallel_size.

TP=2 with CFG parallel disabled:

vllm serve GEAR-Dreams/DreamZero-DROID --omni --port 8000 \
    --served-model-name dreamzero-droid \
    --deploy-config vllm_omni/deploy/dreamzero.yaml \
    --tensor-parallel-size 2 \
    --stage-overrides '{"0": {"devices": "0,1"}}' \
    --enforce-eager --disable-log-stats

TP=2 with CFG parallel size=2:

vllm serve GEAR-Dreams/DreamZero-DROID --omni --port 8000 \
    --served-model-name dreamzero-droid \
    --deploy-config vllm_omni/deploy/dreamzero.yaml \
    --tensor-parallel-size 2 \
    --cfg-parallel-size 2 \
    --stage-overrides '{"0": {"devices": "0,1,2,3"}}' \
    --enforce-eager --disable-log-stats

Download example assets

The OpenPI client and DROID sim-eval example expect the three camera MP4 files in outputs/dreamzero/assets.

hf download YangshenDeng/vllm-omni-dreamzero-assets --repo-type dataset --local-dir outputs/dreamzero/assets

Run the OpenPI client

python examples/online_serving/dreamzero/openpi_client.py \
    --host 127.0.0.1 \
    --port 8000 \
    --video-dir outputs/dreamzero/assets

This client uses downloaded example videos and talks to the OpenPI websocket server.

Run DROID sim eval

${ISAACLAB_LAUNCHER} -p examples/online_serving/dreamzero/droid_sim_eval_client.py \
    --host 127.0.0.1 \
    --port 8000 \
    --scene 1 \
    --episodes 1 \
    --headless \
    --device cuda:0

Set ISAACLAB_LAUNCHER=path/to/isaaclab.sh from the vLLM-Omni repository root before running the command. This launches Isaac Lab / sim-evals and runs the DROID benchmark loop against the same websocket endpoint.

Export comparison videos offline

python examples/offline_inference/dreamzero/export_prediction_video.py \
    --deploy-config vllm_omni/deploy/dreamzero_tp1_cfg2.yaml \
    --save-input-video \
    --save-gif

The export script uses local Omni inference, not the websocket server. It writes the input rollout video and the predicted output video artifacts for side-by-side inspection.

MolmoSpace demo

python examples/online_serving/dreamzero/molmospace_dreamzero_eval_demo.py \
    --host 127.0.0.1 \
    --port 8000 \
    --benchmark_dir /path/to/benchmark \
    --output_dir /path/to/output

This demo adapts DreamZero to the MolmoSpace-style remote policy eval loop.