chore: import upstream snapshot with attribution

docs/diffusion/api/cli.md

@@ -0,0 +1,274 @@
+# SGLang Diffusion CLI
+
+Use the CLI for one-off generation with `sglang generate` or to start a persistent HTTP server with `sglang serve`.
+
+### Overlay repos for non-diffusers models
+
+If `--model-path` points to a supported non-diffusers source repo, SGLang can resolve it
+through a self-hosted overlay repo.
+
+SGLang first checks a built-in overlay registry. Concrete built-in mappings can be added over time without changing the CLI surface.
+
+Override example:
+
+```bash
+export SGLANG_DIFFUSION_MODEL_OVERLAY_REGISTRY='{
+  "Wan-AI/Wan2.2-S2V-14B": {
+    "overlay_repo_id": "your-org/Wan2.2-S2V-14B-overlay",
+    "overlay_revision": "main"
+  }
+}'
+
+sglang generate \
+  --model-path Wan-AI/Wan2.2-S2V-14B \
+  --config configs/wan_s2v.yaml
+```
+
+The overlay repo should be a complete diffusers-style/componentized repo
+
+You can also pass the overlay repo itself as `--model-path` if it contains `_overlay/overlay_manifest.json`.
+
+Notes:
+1. `SGLANG_DIFFUSION_MODEL_OVERLAY_REGISTRY` is only an optional override for
+development and debugging. It accepts either a JSON object or a path to a JSON
+file, and can extend or replace built-in entries for the current process.
+2. On the first load, SGLang will:
+   - download overlay metadata from the overlay repo
+   - download the required files from the original source repo
+   - materialize a local standard component repo under `~/.cache/sgl_diffusion/materialized_models/`
+3. Later loads reuse the materialized local repo. The materialized repo is what the runtime loads as a normal componentized model directory.
+
+
+## Quick Start
+
+### Generate
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --prompt "A beautiful sunset over the mountains" \
+  --save-output
+```
+
+### Serve
+
+```bash
+sglang serve \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --num-gpus 4 \
+  --ulysses-degree 2 \
+  --ring-degree 2 \
+  --port 30010
+```
+
+For request and response examples, see [OpenAI-Compatible API](openai_api.md).
+
+```{tip}
+Use `sglang generate --help` and `sglang serve --help` for the full argument list. The CLI help output is the source of truth for exhaustive flags.
+```
+
+## Common Options
+
+### Model and runtime
+
+- `--model-path {MODEL}`: model path or Hugging Face model ID
+- `--lora-path {PATH}` and `--lora-nickname {NAME}`: load a LoRA adapter
+- `--lora-merge-mode {auto|merge|dynamic}`: choose how LoRA is applied. `auto` statically merges regular weights and uses dynamic LoRA for FSDP-sharded weights to avoid full-gather peaks.
+- `--num-gpus {N}`: number of GPUs to use
+- `--performance-mode {manual|auto|speed|memory}` / `--mode`: preset for latency/throughput and memory defaults. `auto` is the default and keeps safe offload defaults, using FSDP only for validated DiT-offload replacement paths; use `manual` to keep performance-related server args under explicit user control. Explicit offload, FSDP, and parallelism flags take precedence in all modes.
+- `--tp-size {N}`: tensor parallelism size, mainly for encoders
+- `--sp-degree {N}`: sequence parallelism size
+- `--ulysses-degree {N}` and `--ring-degree {N}`: USP parallelism controls
+- `--enable-cfg-parallel {true|false}`: enable or explicitly disable CFG parallelism
+- `--attention-backend {BACKEND}`: attention backend for native SGLang pipelines
+- `--component-attention-backends {MAP}`: per-component attention backend overrides, for example `text_encoder=torch_sdpa,transformer=fa`
+- `--attention-backend-config {CONFIG}`: attention backend configuration
+
+### Sampling and output
+
+- `--prompt {PROMPT}` and `--negative-prompt {PROMPT}`
+- `--image-path {PATH} [{PATH} ...]`: input image(s) for image-to-video or image-to-image generation
+- `--num-inference-steps {STEPS}` and `--seed {SEED}`
+- `--height {HEIGHT}`, `--width {WIDTH}`, `--num-frames {N}`, `--fps {FPS}`
+- `--output-path {PATH}`, `--output-file-name {NAME}`, `--save-output`, `--return-frames`
+
+For frame interpolation and upscaling, see [Post-Processing](post_processing.md).
+
+### Quantized transformers
+
+For quantized transformer checkpoints, prefer:
+
+- `--model-path` for the base pipeline
+- `--transformer-path` for a quantized `transformers` transformer component folder
+- `--transformer-weights-path` for a quantized safetensors file, directory, or repo
+- `--quantization` for online quantization (apply quantization to unquantized models at load time, activations are quantized dynamically)
+- `--quantization-ignored-layers` layer name patterns to keep unquantized (e.g. `attention.to_`)
+
+See [Quantization](../quantization.md) for supported quantization families and examples.
+
+## Configuration Files
+
+Use `--config` to load JSON or YAML configuration. Command-line flags override values from the config file.
+
+```bash
+sglang generate --config config.yaml
+```
+
+Example:
+
+```yaml
+model_path: FastVideo/FastHunyuan-diffusers
+prompt: A beautiful woman in a red dress walking down a street
+output_path: outputs/
+num_gpus: 2
+sp_size: 2
+tp_size: 1
+num_frames: 45
+height: 720
+width: 1280
+num_inference_steps: 6
+seed: 1024
+fps: 24
+precision: bf16
+vae_precision: fp16
+vae_tiling: true
+vae_sp: true
+enable_torch_compile: false
+```
+
+## Generate
+
+`sglang generate` runs a single generation job and exits when the job finishes.
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
+  --text-encoder-cpu-offload \
+  --pin-cpu-memory \
+  --num-gpus 4 \
+  --ulysses-degree 2 \
+  --ring-degree 2 \
+  --prompt "A curious raccoon" \
+  --save-output \
+  --output-path outputs \
+  --output-file-name "a-curious-raccoon.mp4"
+```
+
+```{note}
+HTTP server-only arguments are ignored by `sglang generate`.
+```
+
+For diffusers pipelines, Cache-DiT can be enabled with `SGLANG_CACHE_DIT_ENABLED=true` or `--cache-dit-config`. See [Cache-DiT](../performance/cache/cache_dit.md).
+
+### Layerwise Offload
+
+Use layerwise offload when a large component does not fit comfortably in GPU memory. By default, `--dit-layerwise-offload` only applies to legacy DiT components. Use `--layerwise-offload-components` to select pipeline component names explicitly (`--layerwise-offload-modules` is accepted as an alias):
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
+  --dit-layerwise-offload \
+  --layerwise-offload-components transformer text_encoder \
+  --dit-offload-prefetch-size 0 \
+  --prompt "A quiet city street after rain"
+```
+
+The values must match keys in the selected pipeline's `pipeline.modules`, such as `transformer`, `text_encoder`, `image_encoder`, `vae`, `condition_image_encoder`, `spatial_upsampler`, or `vocoder`. Use `all` to select every layerwise-offloadable component. Prefer the smallest component set that solves the memory issue because layerwise offload can increase latency.
+
+## Serve
+
+`sglang serve` starts the HTTP server and keeps the model loaded for repeated requests.
+
+```bash
+sglang serve \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --text-encoder-cpu-offload \
+  --pin-cpu-memory \
+  --num-gpus 4 \
+  --ulysses-degree 2 \
+  --ring-degree 2 \
+  --port 30010
+```
+
+### Cloud Storage
+
+SGLang Diffusion can upload generated images and videos to S3-compatible object storage after generation.
+
+```bash
+export SGLANG_CLOUD_STORAGE_TYPE=s3
+export SGLANG_S3_BUCKET_NAME=my-bucket
+export SGLANG_S3_ACCESS_KEY_ID=your-access-key
+export SGLANG_S3_SECRET_ACCESS_KEY=your-secret-key
+export SGLANG_S3_ENDPOINT_URL=https://minio.example.com
+```
+
+See [Environment Variables](../environment_variables.md) for the full set of storage options.
+
+## Component Path Overrides
+
+Override individual pipeline components such as `vae`, `transformer`, or `text_encoder` with `--<component>-path`.
+
+```bash
+sglang serve \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --vae-path fal/FLUX.2-Tiny-AutoEncoder
+```
+
+The component key must match the key in the model's `model_index.json`, and the path must be either a Hugging Face repo ID or a complete component directory.
+
+## Component Attention Backend Overrides
+
+Use `--component-attention-backends` when one pipeline component needs a different native attention backend from the global `--attention-backend`.
+
+```bash
+sglang generate \
+  --model-path Lightricks/LTX-2.3 \
+  --attention-backend fa \
+  --component-attention-backends text_encoder=torch_sdpa
+```
+
+The component key must match a pipeline module key such as `text_encoder`, `text_encoder_2`, `transformer`, `transformer_2`, or `connectors`. Component overrides take precedence over the global `--attention-backend` only while that component is being constructed.
+
+You can also pass dotted CLI entries:
+
+```bash
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --component-attention-backends.text_encoder torch_sdpa \
+  --component-attention-backends.transformer fa
+```
+
+## Diffusers Backend
+
+Use `--backend diffusers` to force vanilla diffusers pipelines when no native SGLang implementation exists or when a model requires a custom pipeline class.
+
+### Key Options
+
+| Argument | Values | Description |
+|----------|--------|-------------|
+| `--backend` | `auto`, `sglang`, `diffusers` | Choose native SGLang, force native, or force diffusers |
+| `--diffusers-attention-backend` | `flash`, `_flash_3_hub`, `sage`, `xformers`, `native` | Attention backend for diffusers pipelines |
+| `--trust-remote-code` | flag | Required for models with custom pipeline classes |
+| `--vae-tiling` and `--vae-slicing` | flag | Lower memory usage for VAE decode |
+| `--dit-precision` and `--vae-precision` | `fp16`, `bf16`, `fp32` | Precision controls |
+| `--enable-torch-compile` | flag | Enable `torch.compile` |
+| `--cache-dit-config` | `{PATH}` | Cache-DiT config for diffusers pipelines |
+
+### Example
+
+```bash
+sglang generate \
+  --model-path AIDC-AI/Ovis-Image-7B \
+  --backend diffusers \
+  --trust-remote-code \
+  --diffusers-attention-backend flash \
+  --prompt "A serene Japanese garden with cherry blossoms" \
+  --height 1024 \
+  --width 1024 \
+  --num-inference-steps 30 \
+  --save-output \
+  --output-path outputs \
+  --output-file-name ovis_garden.png
+```
+
+For pipeline-specific arguments not exposed in the CLI, pass `diffusers_kwargs` in a config file.

docs/diffusion/api/openai_api.md

@@ -0,0 +1,449 @@
+# SGLang Diffusion OpenAI API
+
+The SGLang diffusion HTTP server implements an OpenAI-compatible API for image and video generation, as well as LoRA adapter management.
+
+## Prerequisites
+
+- Python 3.11+ if you plan to use the OpenAI Python SDK.
+
+## Serve
+
+Launch the server using the `sglang serve` command.
+
+### Start the server
+
+```bash
+SERVER_ARGS=(
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers
+  --text-encoder-cpu-offload
+  --pin-cpu-memory
+  --num-gpus 4
+  --ulysses-degree=2
+  --ring-degree=2
+  --port 30010
+)
+
+sglang serve "${SERVER_ARGS[@]}"
+```
+
+- **--model-path**: Path to the model or model ID.
+- **--port**: HTTP port to listen on (default: `30000`).
+
+**Get Model Information**
+
+**Endpoint:** `GET /models`
+
+Returns information about the model served by this server, including model path, task type, pipeline configuration, and precision settings.
+
+**Curl Example:**
+
+```bash
+curl -sS -X GET "http://localhost:30010/models"
+```
+
+**Response Example:**
+
+```json
+{
+  "model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
+  "task_type": "T2V",
+  "pipeline_name": "wan_pipeline",
+  "pipeline_class": "WanPipeline",
+  "num_gpus": 4,
+  "dit_precision": "bf16",
+  "vae_precision": "fp16"
+}
+```
+
+---
+
+## Endpoints
+
+### Image Generation
+
+The server implements an OpenAI-compatible Images API under the `/v1/images` namespace.
+
+**Create an image**
+
+**Endpoint:** `POST /v1/images/generations`
+
+**Python Example (b64_json response):**
+
+```python
+import base64
+from openai import OpenAI
+
+client = OpenAI(api_key="sk-proj-1234567890", base_url="http://localhost:30010/v1")
+
+img = client.images.generate(
+    prompt="A calico cat playing a piano on stage",
+    size="1024x1024",
+    n=1,
+    response_format="b64_json",
+)
+
+image_bytes = base64.b64decode(img.data[0].b64_json)
+with open("output.png", "wb") as f:
+    f.write(image_bytes)
+```
+
+**Curl Example:**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/images/generations" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -d '{
+        "prompt": "A calico cat playing a piano on stage",
+        "size": "1024x1024",
+        "n": 1,
+        "response_format": "b64_json"
+      }'
+```
+
+> **Note**
+> If `response_format=url` is used and cloud storage is not configured, the API returns
+> a relative URL like `/v1/images/<IMAGE_ID>/content`.
+
+**Edit an image**
+
+**Endpoint:** `POST /v1/images/edits`
+
+This endpoint accepts a multipart form upload with input images and a text prompt. The server can return either a base64-encoded image or a URL to download the image.
+
+**Curl Example (b64_json response):**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/images/edits" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -F "image=@local_input_image.png" \
+  -F "url=image_url.jpg" \
+  -F "prompt=A calico cat playing a piano on stage" \
+  -F "size=1024x1024" \
+  -F "response_format=b64_json"
+```
+
+**Curl Example (URL response):**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/images/edits" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -F "image=@local_input_image.png" \
+  -F "url=image_url.jpg" \
+  -F "prompt=A calico cat playing a piano on stage" \
+  -F "size=1024x1024" \
+  -F "response_format=url"
+```
+
+**Download image content**
+
+When `response_format=url` is used with `POST /v1/images/generations` or `POST /v1/images/edits`,
+the API returns a relative URL like `/v1/images/<IMAGE_ID>/content`.
+
+**Endpoint:** `GET /v1/images/{image_id}/content`
+
+**Curl Example:**
+
+```bash
+curl -sS -L "http://localhost:30010/v1/images/<IMAGE_ID>/content" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -o output.png
+```
+
+### Video Generation
+
+The server implements a subset of the OpenAI Videos API under the `/v1/videos` namespace.
+
+**Create a video (text-to-video)**
+
+**Endpoint:** `POST /v1/videos`
+
+**Python Example:**
+
+```python
+from openai import OpenAI
+
+client = OpenAI(api_key="sk-proj-1234567890", base_url="http://localhost:30010/v1")
+
+video = client.videos.create(
+    prompt="A calico cat playing a piano on stage",
+    size="1280x720"
+)
+print(f"Video ID: {video.id}, Status: {video.status}")
+```
+
+**Curl Example:**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/videos" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -d '{
+        "prompt": "A calico cat playing a piano on stage",
+        "size": "1280x720"
+      }'
+```
+
+**Create a video (image-to-video)**
+
+For I2V or TI2V models (e.g., Wan2.1 I2V, LTX-2.3 two-stage), pass an input image via multipart form upload or a reference URL.
+
+**Curl Example (multipart form upload):**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/videos" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -F "prompt=A cat playing a piano" \
+  -F "input_reference=@input_image.png" \
+  -F "size=1280x720"
+```
+
+**Curl Example (reference URL):**
+
+```bash
+curl -sS -X POST "http://localhost:30010/v1/videos" \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -d '{
+        "prompt": "A cat playing a piano",
+        "reference_url": "https://example.com/input_image.png",
+        "size": "1280x720"
+      }'
+```
+
+**List videos**
+
+**Endpoint:** `GET /v1/videos`
+
+**Python Example:**
+
+```python
+videos = client.videos.list()
+for item in videos.data:
+    print(item.id, item.status)
+```
+
+**Curl Example:**
+
+```bash
+curl -sS -X GET "http://localhost:30010/v1/videos" \
+  -H "Authorization: Bearer sk-proj-1234567890"
+```
+
+**Download video content**
+
+**Endpoint:** `GET /v1/videos/{video_id}/content`
+
+**Python Example:**
+
+```python
+import time
+
+# Poll for completion
+while True:
+    page = client.videos.list()
+    item = next((v for v in page.data if v.id == video_id), None)
+    if item and item.status == "completed":
+        break
+    time.sleep(5)
+
+# Download content
+resp = client.videos.download_content(video_id=video_id)
+with open("output.mp4", "wb") as f:
+    f.write(resp.read())
+```
+
+**Curl Example:**
+
+```bash
+curl -sS -L "http://localhost:30010/v1/videos/<VIDEO_ID>/content" \
+  -H "Authorization: Bearer sk-proj-1234567890" \
+  -o output.mp4
+```
+
+---
+
+### LoRA Management
+
+The server supports dynamic loading, merging, and unmerging of LoRA adapters.
+
+**Important Notes:**
+- Mutual Exclusion: Only one LoRA configuration can be active per target at a time
+- Switching: To switch LoRAs, deactivate the current LoRA with `unmerge_lora_weights`, then `set` the new one
+- Caching: The server caches loaded LoRA weights in memory. Switching back to a previously loaded LoRA (same path) has little cost
+
+**Set LoRA Adapter**
+
+Loads one or more LoRA adapters and applies them to the model. By default, regular weights are statically merged, while FSDP-sharded weights use dynamic LoRA to avoid full-gather memory peaks.
+
+**Endpoint:** `POST /v1/set_lora`
+
+**Parameters:**
+- `lora_nickname` (string or list of strings, required): A unique identifier for the LoRA adapter(s). Can be a single string or a list of strings for multiple LoRAs
+- `lora_path` (string or list of strings/None, optional): Path to the `.safetensors` file(s) or Hugging Face repo ID(s). Required for the first load; optional if re-activating a cached nickname. If a list, must match the length of `lora_nickname`
+- `target` (string or list of strings, optional): Which transformer(s) to apply the LoRA to. If a list, must match the length of `lora_nickname`. Valid values:
+  - `"all"` (default): Apply to all transformers
+  - `"transformer"`: Apply only to the primary transformer (high noise for Wan2.2)
+  - `"transformer_2"`: Apply only to transformer_2 (low noise for Wan2.2)
+  - `"critic"`: Apply only to the critic model
+- `strength` (float or list of floats, optional): LoRA strength for merge, default 1.0. If a list, must match the length of `lora_nickname`. Values < 1.0 reduce the effect, values > 1.0 amplify the effect
+- `merge_mode` (string, optional): `"auto"` (default server policy), `"merge"` (force static merge), or `"dynamic"` (apply LoRA at forward time)
+
+**Single LoRA Example:**
+
+```bash
+curl -X POST http://localhost:30010/v1/set_lora \
+  -H "Content-Type: application/json" \
+  -d '{
+        "lora_nickname": "lora_name",
+        "lora_path": "/path/to/lora.safetensors",
+        "target": "all",
+        "strength": 0.8
+      }'
+```
+
+**Multiple LoRA Example:**
+
+```bash
+curl -X POST http://localhost:30010/v1/set_lora \
+  -H "Content-Type: application/json" \
+  -d '{
+        "lora_nickname": ["lora_1", "lora_2"],
+        "lora_path": ["/path/to/lora1.safetensors", "/path/to/lora2.safetensors"],
+        "target": ["transformer", "transformer_2"],
+        "strength": [0.8, 1.0]
+      }'
+```
+
+**Multiple LoRA with Same Target:**
+
+```bash
+curl -X POST http://localhost:30010/v1/set_lora \
+  -H "Content-Type: application/json" \
+  -d '{
+        "lora_nickname": ["style_lora", "character_lora"],
+        "lora_path": ["/path/to/style.safetensors", "/path/to/character.safetensors"],
+        "target": "all",
+        "strength": [0.7, 0.9]
+      }'
+```
+
+> [!NOTE]
+> When using multiple LoRAs:
+> - All list parameters (`lora_nickname`, `lora_path`, `target`, `strength`) must have the same length
+> - If `target` or `strength` is a single value, it will be applied to all LoRAs
+> - Multiple LoRAs applied to the same target are applied in order
+
+
+**Merge LoRA Weights**
+
+Manually merges the currently set LoRA weights into the base model.
+
+> [!NOTE]
+> With FSDP-sharded weights, manual merge may require a full-gather and can OOM. Use `set_lora` with `merge_mode="auto"` or `"dynamic"` for the lower-peak path.
+
+**Endpoint:** `POST /v1/merge_lora_weights`
+
+**Parameters:**
+- `target` (string, optional): Which transformer(s) to merge. One of "all" (default), "transformer", "transformer_2", "critic"
+- `strength` (float, optional): LoRA strength for merge, default 1.0. Values < 1.0 reduce the effect, values > 1.0 amplify the effect
+
+**Curl Example:**
+
+```bash
+curl -X POST http://localhost:30010/v1/merge_lora_weights \
+  -H "Content-Type: application/json" \
+  -d '{"strength": 0.8}'
+```
+
+
+**Unmerge LoRA Weights**
+
+Unmerges the currently active LoRA weights from the base model, restoring it to its original state. This **must** be called before setting a different LoRA.
+
+**Endpoint:** `POST /v1/unmerge_lora_weights`
+
+**Curl Example:**
+
+```bash
+curl -X POST http://localhost:30010/v1/unmerge_lora_weights \
+  -H "Content-Type: application/json"
+```
+
+**List LoRA Adapters**
+
+Returns loaded LoRA adapters and current application status per module.
+
+**Endpoint:** `GET /v1/list_loras`
+
+**Curl Example:**
+
+```bash
+curl -sS -X GET "http://localhost:30010/v1/list_loras"
+```
+
+**Response Example:**
+
+```json
+{
+  "loaded_adapters": [
+    { "nickname": "lora_a", "path": "/weights/lora_a.safetensors" },
+    { "nickname": "lora_b", "path": "/weights/lora_b.safetensors" }
+  ],
+  "active": {
+    "transformer": [
+      {
+        "nickname": "lora2",
+        "path": "tarn59/pixel_art_style_lora_z_image_turbo",
+        "merged": true,
+        "mode": "merged",
+        "strength": 1.0
+      }
+    ]
+  }
+}
+```
+
+Notes:
+- If LoRA is not enabled for the current pipeline, the server will return an error.
+- `num_lora_layers_with_weights` counts only layers that have LoRA weights applied for the active adapter.
+
+### Example: Switching LoRAs
+
+1.  Set LoRA A:
+    ```bash
+    curl -X POST http://localhost:30010/v1/set_lora -d '{"lora_nickname": "lora_a", "lora_path": "path/to/A"}'
+    ```
+2.  Generate with LoRA A...
+3.  Unmerge LoRA A:
+    ```bash
+    curl -X POST http://localhost:30010/v1/unmerge_lora_weights
+    ```
+4.  Set LoRA B:
+    ```bash
+    curl -X POST http://localhost:30010/v1/set_lora -d '{"lora_nickname": "lora_b", "lora_path": "path/to/B"}'
+    ```
+5.  Generate with LoRA B...
+
+### Adjust Output Quality
+
+The server supports adjusting output quality and compression levels for both image and video generation through the `output-quality` and `output-compression` parameters.
+
+#### Parameters
+
+- **`output-quality`** (string, optional): Preset quality level that automatically sets compression. **Default is `"default"`**. Valid values:
+  - `"maximum"`: Highest quality (100)
+  - `"high"`: High quality (90)
+  - `"medium"`: Medium quality (55)
+  - `"low"`: Lower quality (35)
+  - `"default"`: Auto-adjust based on media type (50 for video, 75 for image)
+
+- **`output-compression`** (integer, optional): Direct compression level override (0-100). **Default is `None`**. When provided (not `None`), takes precedence over `output-quality`.
+  - `0`: Lowest quality, smallest file size
+  - `100`: Highest quality, largest file size
+
+#### Notes
+
+- **Precedence**: When both `output-quality` and `output-compression` are provided, `output-compression` takes precedence
+- **Format Support**: Quality settings apply to JPEG, and video formats. PNG uses lossless compression and ignores these settings
+- **File Size vs Quality**: Lower compression values (or "low" quality preset) produce smaller files but may show visible artifacts

docs/diffusion/api/post_processing.md

@@ -0,0 +1,148 @@
+# Post-Processing
+
+SGLang diffusion supports optional post-processing steps that run after
+generation to improve temporal smoothness (frame interpolation) or spatial
+resolution (upscaling). These steps are independent of the diffusion model and
+can be combined in a single run.
+
+When both are enabled, **frame interpolation runs first** (increasing the frame
+count), then **upscaling runs on every frame** (increasing the spatial
+resolution).
+
+---
+
+## Frame Interpolation (video only)
+
+Frame interpolation synthesizes new frames between each pair of consecutive
+generated frames, producing smoother motion without re-running the diffusion
+model.
+
+The `--frame-interpolation-exp` flag controls how many rounds of interpolation
+to apply: each round inserts one new frame into every gap between adjacent
+frames, so the output frame count follows the formula:
+
+> **(N − 1) × 2^exp + 1**
+>
+> e.g. 5 original frames with `exp=1` → 4 gaps × 1 new frame + 5 originals = **9** frames;
+> with `exp=2` → **17** frames.
+
+### CLI Arguments
+
+| Argument | Description |
+|----------|-------------|
+| `--enable-frame-interpolation` | Enable frame interpolation. Model weights are downloaded automatically on first use. |
+| `--frame-interpolation-exp {EXP}` | Interpolation exponent — `1` = 2× temporal resolution, `2` = 4×, etc. (default: `1`) |
+| `--frame-interpolation-scale {SCALE}` | RIFE inference scale; use `0.5` for high-resolution inputs to save memory (default: `1.0`) |
+| `--frame-interpolation-model-path {PATH}` | Local directory or HuggingFace repo ID containing RIFE `flownet.pkl` weights (default: `elfgum/RIFE-4.22.lite`, downloaded automatically) |
+
+### Supported Models
+
+Frame interpolation uses the [RIFE](https://github.com/hzwer/Practical-RIFE)
+(Real-Time Intermediate Flow Estimation) architecture. Only **RIFE 4.22.lite**
+(`IFNet` with 4-scale `IFBlock` backbone) is supported. The network topology is
+hard-coded, so custom weights provided via `--frame-interpolation-model-path`
+must be a `flownet.pkl` checkpoint that is compatible with this architecture.
+
+Other RIFE versions (e.g., older `v4.x` variants with different block counts)
+or entirely different frame interpolation methods (FILM, AMT, etc.) are **not
+supported**.
+
+| Weight | HuggingFace Repo | Description |
+|--------|------------------|-------------|
+| RIFE 4.22.lite *(default)* | [`elfgum/RIFE-4.22.lite`](https://huggingface.co/elfgum/RIFE-4.22.lite) | Lightweight model, downloaded automatically on first use |
+
+### Example
+
+Generate a 5-frame video and interpolate to 9 frames ((5 − 1) × 2¹ + 1 = 9):
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
+  --prompt "A dog running through a park" \
+  --num-frames 5 \
+  --enable-frame-interpolation \
+  --frame-interpolation-exp 1 \
+  --save-output
+```
+
+---
+
+## Upscaling (image and video)
+
+Upscaling increases the spatial resolution of generated images or video frames
+using [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN). The model weights
+are downloaded automatically on first use and cached for subsequent runs.
+
+### CLI Arguments
+
+| Argument | Description |
+|----------|-------------|
+| `--enable-upscaling` | Enable post-generation upscaling using Real-ESRGAN. |
+| `--upscaling-scale {SCALE}` | Desired upscaling factor (default: `4`). The 4× model is used internally; if a different scale is requested, a bicubic resize is applied after the network output. |
+| `--upscaling-model-path {PATH}` | Local `.pth` file, HuggingFace repo ID, or `repo_id:filename` for Real-ESRGAN weights (default: `ai-forever/Real-ESRGAN` with `RealESRGAN_x4.pth`, downloaded automatically). Use the `repo_id:filename` format to specify a custom weight file from a HuggingFace repo (e.g. `my-org/my-esrgan:weights.pth`). |
+
+### Supported Models
+
+Upscaling supports two Real-ESRGAN network architectures. The correct
+architecture is **auto-detected** from the checkpoint keys, so you only need to
+point `--upscaling-model-path` at a valid `.pth` file:
+
+| Architecture | Example Weights | Description |
+|--------------|-----------------|-------------|
+| **RRDBNet** | `RealESRGAN_x4plus.pth` | Heavier model with higher quality; best for photos |
+| **SRVGGNetCompact** | `RealESRGAN_x4.pth` *(default)*, `realesr-animevideov3.pth`, `realesr-general-x4v3.pth` | Lightweight model; faster inference, good for video |
+
+The default weight file is
+[`ai-forever/Real-ESRGAN`](https://huggingface.co/ai-forever/Real-ESRGAN) with
+`RealESRGAN_x4.pth` (SRVGGNetCompact, 4× native scale).
+
+Other super-resolution models (e.g., SwinIR, HAT, BSRGAN) are **not supported**
+— only Real-ESRGAN checkpoints using the two architectures above are
+compatible.
+
+### Examples
+
+Generate a 1024×1024 image and upscale to 4096×4096:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --prompt "A cat sitting on a windowsill" \
+  --output-size 1024x1024 \
+  --enable-upscaling \
+  --save-output
+```
+
+Generate a video and upscale each frame by 4×:
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --prompt "A curious raccoon" \
+  --enable-upscaling \
+  --upscaling-scale 4 \
+  --save-output
+```
+
+---
+
+## Combining Frame Interpolation and Upscaling
+
+Frame interpolation and upscaling can be combined in a single run.
+Interpolation is applied first (increasing the frame count), then upscaling is
+applied to every frame (increasing the spatial resolution).
+
+Example — generate 5 frames, interpolate to 9 frames, and upscale each frame
+by 4×:
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --prompt "A curious raccoon" \
+  --num-frames 5 \
+  --enable-frame-interpolation \
+  --frame-interpolation-exp 1 \
+  --enable-upscaling \
+  --upscaling-scale 4 \
+  --save-output
+```

docs/diffusion/ci_perf.md

@@ -0,0 +1,31 @@
+# CI Performance
+
+## Perf Baseline Generation Script
+
+`python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py` starts a local diffusion server, issues requests for selected test cases, aggregates stage/denoise-step/E2E timings from the perf log, and writes the results back to the `scenarios` section of `perf_baselines.json`.
+
+### Usage
+
+Update a single case:
+
+```bash
+python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --case qwen_image_t2i
+```
+
+Select by regex:
+
+```bash
+python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --match 'qwen_image_.*'
+```
+
+Run all keys from the baseline file `scenarios`:
+
+```bash
+python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --all-from-baseline
+```
+
+Specify input/output paths and timeout:
+
+```bash
+python python/sglang/multimodal_gen/test/scripts/gen_perf_baselines.py --baseline python/sglang/multimodal_gen/test/server/perf_baselines.json --out /tmp/perf_baselines.json --timeout 600
+```

docs/diffusion/compatibility_matrix.md

@@ -0,0 +1,200 @@
+# Compatibility Matrix
+
+The table below shows every supported model and the optimizations supported for them.
+
+The symbols used have the following meanings:
+
+- ✅ = Full compatibility
+- ❌ = No compatibility
+- ⭕ = Does not apply to this model
+
+## Models x Optimization
+
+The `HuggingFace Model ID` can be passed directly to `from_pretrained()` methods, and sglang-diffusion will use the
+optimal
+default parameters when initializing and generating videos.
+
+### Video Generation Models
+
+| Model Name                   | Hugging Face Model ID                             | Resolutions          | TeaCache | Sliding Tile Attn | Sage Attn | Video Sparse Attention (VSA) | Sparse Linear Attention (SLA) | Sage Sparse Linear Attention (SageSLA) | Sparse Video Gen 2 (SVG2) | Laser Attn | Block Sparse Attn | Rain Fusion Attn |
+|:-----------------------------|:--------------------------------------------------|:---------------------|:--------:|:-----------------:|:---------:|:----------------------------:|:-----------------------------:|:--------------------------------------:|:-------------------------:|:-------------------------:|:-------------------------:|:-------------------------:|
+| FastWan2.1 T2V 1.3B          | `FastVideo/FastWan2.1-T2V-1.3B-Diffusers`         | 480p                 |    ⭕     |         ⭕         |     ⭕     |              ✅               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| FastWan2.2 TI2V 5B Full Attn | `FastVideo/FastWan2.2-TI2V-5B-FullAttn-Diffusers` | 720p                 |    ⭕     |         ⭕         |     ⭕     |              ✅               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| Wan2.2 TI2V 5B               | `Wan-AI/Wan2.2-TI2V-5B-Diffusers`                 | 720p                 |    ⭕     |         ⭕         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ❌             |               ✅               |                   ✅                    |             ✅             |
+| Wan2.2 T2V A14B              | `Wan-AI/Wan2.2-T2V-A14B-Diffusers`                | 480p<br>720p         |    ❌     |         ❌         |     ✅     |              ⭕               |               ✅               |                   ❌                    |             ✅             |
+| Wan2.2 I2V A14B              | `Wan-AI/Wan2.2-I2V-A14B-Diffusers`                | 480p<br>720p         |    ❌     |         ❌         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ❌             |               ✅               |                   ✅                    |             ✅             |
+| HunyuanVideo                 | `hunyuanvideo-community/HunyuanVideo`             | 720×1280<br>544×960  |    ❌     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ❌               |                   ❌                    |             ❌             |
+| FastHunyuan                  | `FastVideo/FastHunyuan-diffusers`                 | 720×1280<br>544×960  |    ❌     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ❌               |                   ❌                    |             ❌             |
+| Wan2.1 T2V 1.3B              | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`                | 480p                 |    ✅     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ✅               |                   ✅                    |             ✅             |
+| Wan2.1 T2V 14B               | `Wan-AI/Wan2.1-T2V-14B-Diffusers`                 | 480p, 720p           |    ✅     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ✅               |                   ✅                    |             ✅             |
+| Wan2.1 I2V 480P              | `Wan-AI/Wan2.1-I2V-14B-480P-Diffusers`            | 480p                 |    ✅     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ✅               |                   ✅                    |             ✅             |
+| Wan2.1 I2V 720P              | `Wan-AI/Wan2.1-I2V-14B-720P-Diffusers`            | 720p                 |    ✅     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ✅               |                   ✅                    |             ✅             |
+| TurboWan2.1 T2V 1.3B         | `IPostYellow/TurboWan2.1-T2V-1.3B-Diffusers`      | 480p                 |    ✅     |         ❌         |     ❌     |              ❌               |               ✅               |                   ✅                    |             ⭕             |
+| TurboWan2.1 T2V 14B          | `IPostYellow/TurboWan2.1-T2V-14B-Diffusers`       | 480p                 |    ✅     |         ❌         |     ❌     |              ❌               |               ✅               |                   ✅                    |             ⭕             |               ❌               |                   ❌                    |             ❌             |
+| TurboWan2.1 T2V 14B 720P     | `IPostYellow/TurboWan2.1-T2V-14B-720P-Diffusers`  | 720p                 |    ✅     |         ❌         |     ❌     |              ❌               |               ✅               |                   ✅                    |             ⭕             |
+| TurboWan2.2 I2V A14B         | `IPostYellow/TurboWan2.2-I2V-A14B-Diffusers`      | 720p                 |    ✅     |         ❌         |     ❌     |              ❌               |               ✅               |                   ✅                    |             ⭕             |
+| Wan2.1 Fun 1.3B InP          | `weizhou03/Wan2.1-Fun-1.3B-InP-Diffusers`          | 480p                 |    ✅     |         ✅         |     ✅     |              ⭕               |               ❌               |                   ❌                    |             ✅             |               ❌               |                   ❌                    |             ❌             |
+| Helios Base                  | `BestWishYsh/Helios-Base`                          | 720p                 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| Helios Mid                   | `BestWishYsh/Helios-Mid`                           | 720p                 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |
+| Helios Distilled             | `BestWishYsh/Helios-Distilled`                     | 720p                 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| LTX-2 (one/two-stage/TI2V)   | `Lightricks/LTX-2`                                | 768×512<br>1536×1024 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| LTX-2.3 (one/two-stage/TI2V/HQ) | `Lightricks/LTX-2.3`                           | 768×512<br>1536×1024<br>1920×1088 (HQ default) |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |               ❌               |                   ❌                    |             ❌             |             ❌             |
+| Cosmos3-Nano (T2V / I2V / T2I)  | `nvidia/Cosmos3-Nano`                          | 720p · 480p<br>1024×1024 (T2I)                 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+| Cosmos3-Super (T2V / I2V / T2I) | `nvidia/Cosmos3-Super`                         | 720p · 480p<br>1024×1024 (T2I)                 |    ❌     |         ❌         |     ❌     |              ❌               |               ❌               |                   ❌                    |             ❌             |               ❌               |                   ❌                    |             ❌             |
+
+**Note**:
+
+1. Wan2.2 TI2V 5B has some quality issues when performing I2V generation. We are working on fixing this issue.
+2. SageSLA is based on SpargeAttn. Install it first with `pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation`
+3. LTX pipeline selection:
+   - One-stage: `--pipeline-class-name LTX2Pipeline`
+   - Two-stage: `--pipeline-class-name LTX2TwoStagePipeline`
+   - Two-stage HQ: `--pipeline-class-name LTX2TwoStageHQPipeline` (HQ defaults to 1920×1088; you can still override `--width/--height`)
+   - LTX-2 and LTX-2.3 support both T2V and TI2V (`--image-path`) on one-stage and two-stage pipelines (including HQ).
+   - The spatial upsampler and distilled LoRA are auto-resolved from the model snapshot by default, and can still be overridden with `--spatial-upsampler-path` and `--distilled-lora-path`.
+   - For LTX models, the `Resolutions` column uses output video `width×height` semantics, matching `sglang generate --width ... --height ...`.
+4. LTX-2 / LTX-2.3 two-stage also supports `--ltx2-two-stage-device-mode {original,snapshot,resident}`:
+   - `snapshot` is the default and recommended mode.
+   - `resident` usually provides the best latency/throughput but uses much more VRAM.
+   - `original` keeps official two-stage semantics without the premerged stage-2 transformer path.
+   - Example (one prior run): `original` `154.67s`, `snapshot` `114.05s`, `resident` `75.71s`; peak VRAM trend is `original < snapshot < resident`.
+5. Cosmos3 ships in two sizes — `nvidia/Cosmos3-Nano` (8B) and
+   `nvidia/Cosmos3-Super` (32B). Both share the same pipeline; the only
+   difference is transformer depth and width, picked up from
+   `transformer/config.json` at load time. A single checkpoint serves T2V,
+   I2V (`--image-path`), and T2I (`--num-frames 1`).
+
+### Image Generation Models
+
+| Model Name                | HuggingFace Model ID                                     |
+|:--------------------------|:---------------------------------------------------------|
+| FLUX.1-dev                | `black-forest-labs/FLUX.1-dev`                           |
+| FLUX.2-dev                | `black-forest-labs/FLUX.2-dev`                           |
+| FLUX.2-dev-NVFP4          | `black-forest-labs/FLUX.2-dev-NVFP4`                     |
+| FLUX.2-Klein-4B           | `black-forest-labs/FLUX.2-klein-4B`                      |
+| FLUX.2-Klein-9B           | `black-forest-labs/FLUX.2-klein-9B`                      |
+| Z-Image                   | `Tongyi-MAI/Z-Image`                                    |
+| Z-Image-Turbo             | `Tongyi-MAI/Z-Image-Turbo`                              |
+| GLM-Image                 | `zai-org/GLM-Image`                                     |
+| Qwen Image                | `Qwen/Qwen-Image`                                       |
+| Qwen Image 2512           | `Qwen/Qwen-Image-2512`                                  |
+| Qwen Image Edit           | `Qwen/Qwen-Image-Edit`                                  |
+| Qwen Image Edit 2509      | `Qwen/Qwen-Image-Edit-2509`                             |
+| Qwen Image Edit 2511      | `Qwen/Qwen-Image-Edit-2511`                             |
+| Qwen Image Layered        | `Qwen/Qwen-Image-Layered`                               |
+| SD3 Medium                | `stabilityai/stable-diffusion-3-medium-diffusers`        |
+| SD3.5 Medium              | `stabilityai/stable-diffusion-3.5-medium-diffusers`      |
+| SD3.5 Large               | `stabilityai/stable-diffusion-3.5-large-diffusers`       |
+| Hunyuan3D-2               | `tencent/Hunyuan3D-2`                                    |
+| SANA 1.5 1.6B             | `Efficient-Large-Model/SANA1.5_1.6B_1024px_diffusers`   |
+| SANA 1.5 4.8B             | `Efficient-Large-Model/SANA1.5_4.8B_1024px_diffusers`   |
+| SANA 1600M 1024px         | `Efficient-Large-Model/Sana_1600M_1024px_diffusers`     |
+| SANA 600M 1024px          | `Efficient-Large-Model/Sana_600M_1024px_diffusers`      |
+| SANA 1600M 512px          | `Efficient-Large-Model/Sana_1600M_512px_diffusers`      |
+| SANA 600M 512px           | `Efficient-Large-Model/Sana_600M_512px_diffusers`       |
+| FireRed-Image-Edit 1.0    | `FireRedTeam/FireRed-Image-Edit-1.0`                     |
+| FireRed-Image-Edit 1.1    | `FireRedTeam/FireRed-Image-Edit-1.1`                     |
+| ERNIE-Image               | `baidu/ERNIE-Image`                                      |
+| ERNIE-Image-Turbo         | `baidu/ERNIE-Image-Turbo`                                |
+
+## Supported Components
+
+SGLang Diffusion supports overriding individual pipeline components with
+`--<component>-path`. The value can be either a Hugging Face repo ID or a local
+component directory.
+
+The same overrides can also be provided in config files through
+`component_paths.<component>`.
+
+### Common Syntax
+
+CLI:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --vae-path black-forest-labs/FLUX.2-small-decoder \
+  --transformer-path /models/flux2/transformer
+```
+
+Config file:
+
+```yaml
+model_path: black-forest-labs/FLUX.2-dev
+component_paths:
+  vae: black-forest-labs/FLUX.2-small-decoder
+  transformer: /models/flux2/transformer
+```
+
+Use the component name from the pipeline's `model_index.json` or the native pipeline's registered module name:
+
+| Component Type    | Supported Keys                                                                                                             | Notes                                                         |
+|:------------------|:---------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------|
+| VAE               | `vae`, `video_vae`, `audio_vae`                                                                                            | `vae` is the common image-generation override                 |
+| Transformer / DiT | `transformer`, `video_dit`, `audio_dit`                                                                                    | `transformer` is the standard override for the main denoiser  |
+| Text / Preprocess | `text_encoder`, `text_encoder_2`, `tokenizer`, `processor`, `image_processor`                                              | Replacement encoders often need matching preprocessing assets |
+| Auxiliary         | `scheduler`, `spatial_upsampler`, `vocoder`, `connectors`, `dual_tower_bridge`, `image_encoder`, `vision_language_encoder` | Only valid for pipelines that expose these components         |
+
+### Known Component Repos
+
+The table below lists concrete Hugging Face component repos that are already used in SGLang Diffusion docs or tests. It is not an exhaustive catalog of all compatible component repos.
+
+| Base Model                     | Override Key  | Example Repo                             | Notes                                     |
+|:-------------------------------|:--------------|:-----------------------------------------|:------------------------------------------|
+| `black-forest-labs/FLUX.2-dev` | `vae`         | `black-forest-labs/FLUX.2-small-decoder` | Decoder-only FLUX.2 VAE override          |
+| `black-forest-labs/FLUX.2-dev` | `vae`         | `fal/FLUX.2-Tiny-AutoEncoder`            | Existing tested custom VAE path           |
+
+### VAE
+
+- `--vae-path` is the common image-generation override.
+- `--video-vae-path` and `--audio-vae-path` are only relevant for pipelines with separate video or audio VAEs.
+
+### Transformer / DiT
+
+- `--transformer-path` is the standard override for the main denoising transformer.
+- For quantized transformers, prefer `--transformer-path` or `--transformer-weights-path`; see `quantization.md`.
+- `--video-dit-path` and `--audio-dit-path` are only for pipelines that split denoisers by modality.
+
+### Text Encoders and Preprocessors
+
+- `--text-encoder-path` and `--text-encoder-2-path` override primary and secondary text encoders.
+- `--tokenizer-path`, `--processor-path`, and `--image-processor-path` are useful when the replacement encoder requires matching preprocessing assets.
+
+### Auxiliary Components
+
+- `--scheduler-path` is only relevant when the pipeline exposes a scheduler component.
+- `--spatial-upsampler-path` is mainly for two-stage pipelines such as `LTX2TwoStagePipeline`.
+- `--vocoder-path`, `--connectors-path`, `--dual-tower-bridge-path`, `--image-encoder-path`, and `--vision-language-encoder-path` are only valid for pipelines that expose those components.
+
+### Notes
+
+1. Component overrides are only valid when the target pipeline actually uses
+   that component.
+2. The override key should match the component name in the pipeline's
+   `model_index.json` or the native pipeline's registered module name.
+
+## Verified LoRA Examples
+
+This section lists example LoRAs that have been explicitly tested and verified with each base model in the **SGLang Diffusion** pipeline.
+
+> Important:
+> LoRAs that are not listed here are not necessarily incompatible.
+> In practice, most standard LoRAs are expected to work, especially those following common Diffusers or SD-style conventions.
+> The entries below simply reflect configurations that have been manually validated by the SGLang team.
+
+### Verified LoRAs by Base Model
+
+| Base Model      | Supported LoRAs                                                                                                                                    |
+|:----------------|:---------------------------------------------------------------------------------------------------------------------------------------------------|
+| Wan2.2          | `lightx2v/Wan2.2-Distill-Loras`<br>`Cseti/wan2.2-14B-Arcane_Jinx-lora-v1`                                                                          |
+| Wan2.1          | `lightx2v/Wan2.1-Distill-Loras`                                                                                                                    |
+| Z-Image-Turbo   | `tarn59/pixel_art_style_lora_z_image_turbo`<br>`wcde/Z-Image-Turbo-DeJPEG-Lora`                                                                    |
+| Qwen-Image      | `lightx2v/Qwen-Image-Lightning`<br>`flymy-ai/qwen-image-realism-lora`<br>`prithivMLmods/Qwen-Image-HeadshotX`<br>`starsfriday/Qwen-Image-EVA-LoRA` |
+| Qwen-Image-Edit | `ostris/qwen_image_edit_inpainting`<br>`lightx2v/Qwen-Image-Edit-2511-Lightning`                                                                   |
+| Flux            | `dvyio/flux-lora-simple-illustration`<br>`XLabs-AI/flux-furry-lora`<br>`XLabs-AI/flux-RealismLora`                                                 |
+
+## Special requirements
+
+### Sliding Tile Attention
+
+- Currently, only Hopper GPUs (H100s) are supported.

docs/diffusion/contributing.md

@@ -0,0 +1,79 @@
+# Contributing to SGLang Diffusion
+
+This guide outlines the requirements for contributing to the SGLang Diffusion module (`sglang.multimodal_gen`).
+
+## Contributor Guides
+
+- [Support New Models](support_new_models.md): implementation guide for adding new diffusion pipelines
+- [CI Performance](ci_perf.md): update and regenerate perf baselines
+
+```{toctree}
+:maxdepth: 1
+
+support_new_models
+ci_perf
+```
+
+## On AI-Assisted ("Vibe Coding") PRs
+
+Vibe-coded PRs are welcome — we judge code quality, not how it was produced. The bar is the same for all PRs:
+
+- **No over-commenting.** If the name says it all, skip the docstring.
+- **No over-catching.** Don't guard against errors that virtually never happen in practice.
+- **Test before submitting.** AI-generated code can be subtly wrong — verify correctness end-to-end.
+
+## Commit Message Convention
+
+We follow a structured commit message format to maintain a clean history.
+
+**Format:**
+```text
+[diffusion] <scope>: <subject>
+```
+
+**Examples:**
+- `[diffusion] cli: add --perf-dump-path argument`
+- `[diffusion] scheduler: fix deadlock in batch processing`
+- `[diffusion] model: support Stable Diffusion 3.5`
+
+**Rules:**
+- **Prefix**: Always start with `[diffusion]`.
+- **Scope** (Optional): `cli`, `scheduler`, `model`, `pipeline`, `docs`, etc.
+- **Subject**: Imperative mood, short and clear (e.g., "add feature" not "added feature").
+
+## Performance Reporting
+
+For PRs that impact **latency**, **throughput**, or **memory usage**, you **should** provide a performance comparison report.
+
+### How to Generate a Report
+
+1.  **Baseline**: run the benchmark (for a single generation task)
+    ```bash
+    $ sglang generate --model-path <model> --prompt "A benchmark prompt" --perf-dump-path baseline.json
+    ```
+
+2.  **New**: run the same benchmark, without modifying any server_args or sampling_params
+    ```bash
+    $ sglang generate --model-path <model> --prompt "A benchmark prompt" --perf-dump-path new.json
+    ```
+
+3.  **Compare**: run the compare script, which will print a Markdown table to the console
+    ```bash
+    $ python python/sglang/multimodal_gen/benchmarks/compare_perf.py baseline.json new.json [new2.json ...]
+    ### Performance Comparison Report
+    ...
+    ```
+4. **Paste**: paste the table into the PR description
+
+## CI-Based Change Protection
+
+Consider adding tests to the `pr-test` or `nightly-test` suites to safeguard your changes, especially for PRs that:
+
+- support a new model
+    - add a testcase for this new model to `testcase_configs.py`
+- support or fix important features
+- significantly improve performance
+
+Please run the according testcase, then update/add the baseline to `perf_baselines.json` by following the instruction in console if applicable.
+
+See [test](https://github.com/sgl-project/sglang/tree/main/python/sglang/multimodal_gen/test) for examples

docs/diffusion/development.md

@@ -0,0 +1,5 @@
+# Development
+
+This page collects lower-level development material for SGLang Diffusion.
+
+- [Contributing](contributing.md): contribution workflow, adding new models, and CI perf baselines

docs/diffusion/disaggregation.md

@@ -0,0 +1,237 @@
+# Disaggregated Diffusion Pipeline
+
+Split a monolithic text-to-video/image pipeline into independent **Encoder**, **Denoiser**, and **Decoder** roles, each running on its own GPU(s). A central **DiffusionServer** routes requests through the pipeline.
+
+## Quick Start
+
+Disaggregation is controlled by a single flag: `--disagg-role`. Each component is launched independently, just like LLM PD disaggregation.
+
+| `--disagg-role` | What it runs |
+|----------------|--------------|
+| `monolithic` | (Default) Standard single-server mode |
+| `encoder` | All stages with the default `RoleType.ENCODER` affinity: `InputValidationStage`, `TextEncodingStage` (plus `ImageEncodingStage` / `ImageVAEEncodingStage` for image-conditioned pipelines), `LatentPreparationStage`, `TimestepPreparationStage`, and any model-specific "before denoising" stage (e.g. `QwenImageLayeredBeforeDenoisingStage`, `GlmImageBeforeDenoisingStage`). |
+| `denoiser` | `DenoisingStage` (and its subclasses: `CausalDMDDenoisingStage`, `DmdDenoisingStage`, `LTX2AVDenoisingStage`, `LTX2RefinementStage`, `Hunyuan3DShapeDenoisingStage`, ...) — the DiT forward loop plus the scheduler stepping it drives. |
+| `decoder` | `DecodingStage` (VAE decode) and its subclasses (`LTX2AVDecodingStage`, `HeliosDecodingStage`, ...). |
+| `server` | DiffusionServer head node + HTTP server (no GPU) |
+
+> Each stage declares its role via the `role_affinity` property on `PipelineStage` (default `ENCODER`). When `--disagg-role` is not `monolithic`, the pipeline only instantiates stages whose affinity matches, so the above table is the source of truth for what actually runs in each process.
+
+### Single-Machine Example (Verified)
+
+The following commands have been tested end-to-end on an 8×H200 machine with
+`Wan-AI/Wan2.1-T2V-1.3B-Diffusers`. Each role runs on a separate GPU via
+`--base-gpu-id`; the `server` head node requires no GPU.
+
+```bash
+# Terminal 1: Encoder (GPU 0)
+sglang serve --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+    --disagg-role encoder \
+    --disagg-server-addr tcp://127.0.0.1:19655 \
+    --scheduler-port 19000 \
+    --num-gpus 1 --base-gpu-id 0
+
+# Terminal 2: Denoiser (GPU 1)
+sglang serve --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+    --disagg-role denoiser \
+    --disagg-server-addr tcp://127.0.0.1:19655 \
+    --scheduler-port 19001 \
+    --num-gpus 1 --base-gpu-id 1
+
+# Terminal 3: Decoder (GPU 2)
+sglang serve --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+    --disagg-role decoder \
+    --disagg-server-addr tcp://127.0.0.1:19655 \
+    --scheduler-port 19002 \
+    --num-gpus 1 --base-gpu-id 2
+
+# Terminal 4: DiffusionServer head (no GPU, receives HTTP requests)
+sglang serve --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+    --disagg-role server \
+    --encoder-urls  "tcp://127.0.0.1:19000" \
+    --denoiser-urls "tcp://127.0.0.1:19001" \
+    --decoder-urls  "tcp://127.0.0.1:19002" \
+    --host 0.0.0.0 --port 22000 \
+    --scheduler-port 19655
+
+# Send request (video generation)
+curl http://127.0.0.1:22000/v1/videos \
+    -H "Content-Type: application/json" \
+    -d '{"model": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers", "prompt": "A curious raccoon exploring a garden, cinematic", "size": "832x480"}'
+```
+
+> **Tested result (8×H200):**
+> Encoder 2.3 s (TextEncoding) → Denoiser 312.8 s (50 steps, layerwise offload) → Decoder 7.1 s (VAE decode).
+> Total ~322 s for 81-frame 1024×1024 video.
+
+> **Tip:** `--base-gpu-id` controls which physical GPU the role uses.
+> Encoder and Decoder can share a GPU (e.g. both `--base-gpu-id 0`) to save resources,
+> but make sure the combined GPU memory is sufficient.
+
+### Multi-Machine Example
+
+The exact same CLI pattern — just replace `127.0.0.1` with actual IPs and add
+RDMA flags for direct transfer:
+
+```bash
+# Machine A (10.0.0.1): Encoder
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers \
+    --disagg-role encoder \
+    --disagg-server-addr tcp://10.0.0.4:19655 \
+    --scheduler-port 19000 \
+    --num-gpus 1 \
+    --disagg-p2p-hostname 10.0.0.1 --disagg-ib-device mlx5_0
+
+# Machine B (10.0.0.2): Denoiser (4 GPUs with SP)
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers \
+    --disagg-role denoiser \
+    --disagg-server-addr tcp://10.0.0.4:19655 \
+    --scheduler-port 19001 \
+    --num-gpus 4 --denoiser-sp 4 --denoiser-ulysses 2 --denoiser-ring 2 \
+    --disagg-p2p-hostname 10.0.0.2 --disagg-ib-device mlx5_0
+
+# Machine C (10.0.0.3): Decoder
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers \
+    --disagg-role decoder \
+    --disagg-server-addr tcp://10.0.0.4:19655 \
+    --scheduler-port 19002 \
+    --num-gpus 1 \
+    --disagg-p2p-hostname 10.0.0.3 --disagg-ib-device mlx5_0
+
+# Machine D (10.0.0.4): DiffusionServer head
+sglang serve --model-path Wan-AI/Wan2.1-T2V-14B-Diffusers \
+    --disagg-role server \
+    --encoder-urls  "tcp://10.0.0.1:19000" \
+    --denoiser-urls "tcp://10.0.0.2:19001" \
+    --decoder-urls  "tcp://10.0.0.3:19002" \
+    --host 0.0.0.0 --port 30000 \
+    --scheduler-port 19655 \
+    --disagg-dispatch-policy max_free_slots
+```
+
+> ZMQ handles startup order gracefully — instances and head can start in any order.
+
+## Multiple Instances per Role
+
+Use semicolons in `--*-urls` to register multiple instances:
+
+```bash
+# 2 encoders + 2 denoisers (4-GPU SP each) + 1 decoder
+sglang serve --model-path ... --disagg-role server \
+    --encoder-urls  "tcp://10.0.0.1:35000;tcp://10.0.0.2:35000" \
+    --denoiser-urls "tcp://10.0.0.3:35000;tcp://10.0.0.4:35000" \
+    --decoder-urls  "tcp://10.0.0.5:35000"
+```
+
+## Port Convention
+
+Result endpoints are derived deterministically from the head node's `--scheduler-port` (default: 5555):
+
+| Socket | Port |
+|--------|------|
+| DS frontend (ROUTER) | `scheduler_port` |
+| Encoder result (PULL) | `scheduler_port + 1` |
+| Denoiser result (PULL) | `scheduler_port + 2` |
+| Decoder result (PULL) | `scheduler_port + 3` |
+
+Role instances derive their result endpoint automatically from `--disagg-server-addr`. No manual endpoint configuration needed.
+
+## Transfer Mechanism
+
+Tensor data between roles (encoder→denoiser, denoiser→decoder) is transferred via a P2P transfer engine. The DiffusionServer only routes lightweight control messages (alloc/push/ready); actual tensor data flows directly between instances.
+
+**mooncake-transfer-engine** is required for disaggregated diffusion. It provides RDMA for direct GPU-to-GPU data movement.
+
+```bash
+pip install mooncake-transfer-engine
+```
+
+### Transfer Flow
+
+1. **Sender** (encoder/denoiser) stages tensors: async copy to transfer buffer (GPU or CPU pinned, depending on GPUDirect support), overlapped with metadata JSON serialization.
+2. **Sender** sends `transfer_staged` control message to DiffusionServer (metadata only, no tensor data).
+3. **DiffusionServer** sends `transfer_alloc` to receiver → receiver allocates buffer slot → replies `transfer_allocated`.
+4. **DiffusionServer** sends `transfer_push` to receiver with sender's address info.
+5. **Receiver** pulls data via transfer engine (Mooncake RDMA or mock), sends `transfer_ready`.
+6. **Receiver** loads tensors async on a dedicated transfer stream, overlapped with the previous request's compute.
+
+Decoder results (final output) flow back through DiffusionServer as raw ZMQ frames to the HTTP client.
+
+### RDMA Flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--disagg-p2p-hostname` | `127.0.0.1` | RDMA-reachable hostname/IP of this instance |
+| `--disagg-ib-device` | `None` | InfiniBand device (e.g., `mlx5_0`, `mlx5_roce0`) |
+| `--disagg-transfer-pool-size` | 256 MiB | Pinned memory pool per instance |
+
+Set `--disagg-p2p-hostname` to the actual IP on each machine. For multi-machine, `--disagg-ib-device` specifies the RDMA NIC.
+
+## Per-Role Parallelism
+
+| Flag | Description |
+|------|-------------|
+| `--encoder-tp` | Encoder tensor parallelism |
+| `--denoiser-tp` / `--denoiser-sp` / `--denoiser-ulysses` / `--denoiser-ring` | Denoiser parallelism |
+| `--decoder-tp` | Decoder tensor parallelism |
+
+If not specified, parallelism is auto-derived from `--num-gpus`.
+
+## Other Options
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--disagg-timeout` | `600` | Timeout (seconds) for pending requests |
+| `--disagg-dispatch-policy` | `round_robin` | `round_robin` or `max_free_slots` |
+
+## Python API
+
+For programmatic single-machine deployment, `launch_pool_disagg_server()` is available:
+
+```python
+from sglang.multimodal_gen.runtime.server_args import ServerArgs
+from sglang.multimodal_gen.runtime.launch_server import launch_pool_disagg_server
+
+server_args = ServerArgs.from_kwargs(
+    model_path="Wan-AI/Wan2.1-T2V-14B-Diffusers",
+    denoiser_sp=4, denoiser_ulysses=2, denoiser_ring=2,
+    disagg_ib_device="mlx5_0",
+)
+
+launch_pool_disagg_server(
+    server_args,
+    encoder_gpus=[[0]],
+    denoiser_gpus=[[1, 2, 3, 4], [5, 6, 7, 8]],
+    decoder_gpus=[[0]],
+)
+```
+
+## Architecture
+
+```
+Client ─── HTTP (port 30000) ──► FastAPI Server
+                                      │
+                                      ▼
+                              DiffusionServer (ROUTER, scheduler_port)
+                              ┌───────┼───────┐
+                   PUSH work  │       │       │  PUSH work
+                              ▼       │       ▼
+                    Encoder[0..N]     │    Decoder[0..K]
+                              │       │       ▲
+                   P2P tensor │       │       │ P2P tensor
+                   transfer   ▼       │       │ transfer
+                          Denoiser[0..M] ─────┘
+                                      │
+                    PULL results ◄────┘  (decoder → DS → client)
+```
+
+### Request State Machine
+
+```
+PENDING → ENCODER_WAITING → ENCODER_RUNNING → ENCODER_DONE
+                                                    │
+                        DENOISING_WAITING → DENOISING_RUNNING → DENOISING_DONE
+                                                                       │
+                                    DECODER_WAITING → DECODER_RUNNING → DONE
+```
+
+Any state can transition to `FAILED` or `TIMED_OUT`.

docs/diffusion/environment_variables.md

@@ -0,0 +1,101 @@
+# Environment Variables
+
+## Runtime
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `SGLANG_DIFFUSION_TARGET_DEVICE` | `cuda` | Target device for inference (`cuda`, `rocm`, `xpu`, `npu`, `musa`, `mps`, `cpu`) |
+| `SGLANG_DIFFUSION_ATTENTION_BACKEND` | not set | Override attention backend via env var (e.g. `fa`, `torch_sdpa`, `sage_attn`) |
+| `SGLANG_DIFFUSION_ATTENTION_CONFIG` | not set | Path to attention backend configuration file (JSON/YAML) |
+| `SGLANG_DIFFUSION_STAGE_LOGGING` | false | Enable per-stage timing logs |
+| `SGLANG_DIFFUSION_SERVER_DEV_MODE` | false | Enable dev-only HTTP endpoints for debugging |
+| `SGLANG_DIFFUSION_TORCH_PROFILER_DIR` | not set | Directory for torch profiler traces (absolute path). Enables profiling when set |
+| `SGLANG_DIFFUSION_CACHE_ROOT` | `~/.cache/sgl_diffusion` | Root directory for cache files |
+| `SGLANG_DIFFUSION_CONFIG_ROOT` | `~/.config/sgl_diffusion` | Root directory for configuration files |
+| `SGLANG_DIFFUSION_LOGGING_LEVEL` | `INFO` | Default logging level |
+| `SGLANG_DIFFUSION_WORKER_MULTIPROC_METHOD` | `fork` | Multiprocess context for workers (`fork` or `spawn`) |
+| `SGLANG_USE_RUNAI_MODEL_STREAMER` | true | Use Run:AI model streamer for model loading |
+
+## Platform-Specific
+
+### Apple MPS
+
+| Environment Variable | Default | Description                                                  |
+|----------------------|---------|--------------------------------------------------------------|
+| `SGLANG_USE_MLX`     | not set | Set to `1` to enable MLX fused Metal kernels for norm ops on MPS |
+
+### ROCm (AMD GPUs)
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `SGLANG_USE_ROCM_VAE` | false | Use AITer GroupNorm in VAE for improved performance on ROCm |
+| `SGLANG_USE_ROCM_CUDNN_BENCHMARK` | false | Enable MIOpen auto-tuning for VAE conv layers on ROCm |
+
+### Quantization
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND` | not set | Optional FlashInfer FP4 GEMM backend override for diffusion NVFP4. When unset, SGLang defaults to `flashinfer_trtllm`. |
+
+## Caching Acceleration
+
+These variables configure caching acceleration for Diffusion Transformer (DiT) models.
+SGLang supports multiple caching strategies - see [caching documentation](performance/cache/index.md) for an overview.
+
+### Cache-DiT Configuration
+
+See [cache-dit documentation](performance/cache/cache_dit.md) for detailed configuration.
+
+| Environment Variable                | Default | Description                              |
+|-------------------------------------|---------|------------------------------------------|
+| `SGLANG_CACHE_DIT_ENABLED`          | false   | Enable Cache-DiT acceleration            |
+| `SGLANG_CACHE_DIT_FN`               | 1       | First N blocks to always compute         |
+| `SGLANG_CACHE_DIT_BN`               | 0       | Last N blocks to always compute          |
+| `SGLANG_CACHE_DIT_WARMUP`           | 4       | Warmup steps before caching              |
+| `SGLANG_CACHE_DIT_RDT`              | 0.24    | Residual difference threshold            |
+| `SGLANG_CACHE_DIT_MC`               | 3       | Max continuous cached steps              |
+| `SGLANG_CACHE_DIT_TAYLORSEER`       | false   | Enable TaylorSeer calibrator             |
+| `SGLANG_CACHE_DIT_TS_ORDER`         | 1       | TaylorSeer order (1 or 2)                |
+| `SGLANG_CACHE_DIT_SCM_PRESET`       | none    | SCM preset (none/slow/medium/fast/ultra) |
+| `SGLANG_CACHE_DIT_SCM_POLICY`       | dynamic | SCM caching policy                       |
+| `SGLANG_CACHE_DIT_SCM_COMPUTE_BINS` | not set | Custom SCM compute bins                  |
+| `SGLANG_CACHE_DIT_SCM_CACHE_BINS`   | not set | Custom SCM cache bins                    |
+
+### Cache-DiT Secondary Transformer
+
+For dual-transformer models (e.g., Wan2.2 with high/low-noise experts), these variables configure caching for the secondary transformer. Each falls back to its primary counterpart if not set.
+
+| Environment Variable | Default | Description |
+|-------------------------------------|---------|------------------------------------------|
+| `SGLANG_CACHE_DIT_SECONDARY_FN` | (from primary) | First N blocks to always compute |
+| `SGLANG_CACHE_DIT_SECONDARY_BN` | (from primary) | Last N blocks to always compute |
+| `SGLANG_CACHE_DIT_SECONDARY_WARMUP` | (from primary) | Warmup steps before caching |
+| `SGLANG_CACHE_DIT_SECONDARY_RDT` | (from primary) | Residual difference threshold |
+| `SGLANG_CACHE_DIT_SECONDARY_MC` | (from primary) | Max continuous cached steps |
+| `SGLANG_CACHE_DIT_SECONDARY_TAYLORSEER` | (from primary) | Enable TaylorSeer calibrator |
+| `SGLANG_CACHE_DIT_SECONDARY_TS_ORDER` | (from primary) | TaylorSeer order (1 or 2) |
+
+## Cloud Storage
+
+These variables configure S3-compatible cloud storage for automatically uploading generated images and videos.
+
+| Environment Variable            | Default | Description                                            |
+|---------------------------------|---------|--------------------------------------------------------|
+| `SGLANG_CLOUD_STORAGE_TYPE`     | not set | Set to `s3` to enable cloud storage                    |
+| `SGLANG_S3_BUCKET_NAME`         | not set | The name of the S3 bucket                              |
+| `SGLANG_S3_ENDPOINT_URL`        | not set | Custom endpoint URL (for MinIO, OSS, etc.)             |
+| `SGLANG_S3_REGION_NAME`         | us-east-1 | AWS region name                                      |
+| `SGLANG_S3_ACCESS_KEY_ID`       | not set | AWS Access Key ID                                      |
+| `SGLANG_S3_SECRET_ACCESS_KEY`   | not set | AWS Secret Access Key                                  |
+
+## CUDA Crash Debugging
+
+These variables enable kernel API logging and optional input/output dumps around diffusion CUDA kernel call boundaries. They are useful when tracking down CUDA crashes such as illegal memory access, device-side assert, or shape mismatches in custom kernels.
+
+| Environment Variable | Default | Description |
+|----------------------|---------|-------------|
+| `SGLANG_KERNEL_API_LOGLEVEL` | `0` | Controls crash-debug kernel API logging. `1` logs API names, `3` logs tensor metadata, `5` adds tensor statistics, and `10` also writes dump snapshots. |
+| `SGLANG_KERNEL_API_LOGDEST` | `stdout` | Destination for crash-debug kernel API logs. Use `stdout`, `stderr`, or a file path. `%i` is replaced with the process PID. |
+| `SGLANG_KERNEL_API_DUMP_DIR` | `sglang_kernel_api_dumps` | Output directory for level-10 kernel API dumps. `%i` is replaced with the process PID. |
+| `SGLANG_KERNEL_API_DUMP_INCLUDE` | not set | Comma-separated wildcard patterns for kernel API names to include in level-10 dumps. |
+| `SGLANG_KERNEL_API_DUMP_EXCLUDE` | not set | Comma-separated wildcard patterns for kernel API names to exclude from level-10 dumps. |

docs/diffusion/index.md

@@ -0,0 +1,53 @@
+# SGLang Diffusion
+
+SGLang Diffusion is a high-performance inference framework for image and video generation. It provides native SGLang pipelines, diffusers backend support, an OpenAI-compatible server, and an optimized kernel stack built on both precompiled `sgl-kernel` operators and JIT kernels for key inference paths.
+
+## Key Features
+
+- Broad model support across Wan, Hunyuan, Cosmos3, Qwen-Image, FLUX, Z-Image, GLM-Image, and more
+- Fast inference with `sgl-kernel`, JIT kernels, scheduler improvements, and caching acceleration
+- Multiple interfaces: `sglang generate`, `sglang serve`, and an OpenAI-compatible API
+- Multi-platform support for NVIDIA, AMD, Intel XPU, Ascend, Apple Silicon, and Moore Threads
+
+## Quick Start
+
+```bash
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+```bash
+sglang generate --model-path Qwen/Qwen-Image \
+  --prompt "A beautiful sunset over the mountains" \
+  --save-output
+```
+
+```bash
+sglang serve --model-path Qwen/Qwen-Image --port 30010
+```
+
+## Start Here
+
+- [Installation](installation.md): install SGLang Diffusion and platform dependencies
+- [Compatibility Matrix](compatibility_matrix.md): check model, optimization, and component override support
+- [CLI](api/cli.md): run one-off generation jobs or launch a persistent server
+- [OpenAI-Compatible API](api/openai_api.md): send image and video requests to the HTTP server
+- [Attention Backends](performance/attention_backends.md): choose the best backend for your model and hardware
+- [Caching Acceleration](performance/cache/index.md): use Cache-DiT or TeaCache to reduce denoising cost
+- [Quantization](quantization.md): load quantized transformer checkpoints
+- [Contributing](contributing.md): contribution workflow, adding new models, and CI perf baselines
+
+## Additional Documentation
+
+- [Post-Processing](api/post_processing.md): frame interpolation and upscaling
+- [Performance Overview](performance/index.md): overview of attention, caching, and profiling
+- [Environment Variables](environment_variables.md): platform, caching, storage, and debugging configuration
+- [Support New Models](support_new_models.md): implementation guide for new diffusion pipelines
+- [CI Performance](ci_perf.md): performance baseline generation
+
+## References
+
+- [SGLang GitHub](https://github.com/sgl-project/sglang)
+- [Cache-DiT](https://github.com/vipshop/cache-dit)
+- [FastVideo](https://github.com/hao-ai-lab/FastVideo)
+- [xDiT](https://github.com/xdit-project/xDiT)
+- [Diffusers](https://github.com/huggingface/diffusers)

docs/diffusion/installation.md

@@ -0,0 +1,128 @@
+# Install SGLang-Diffusion
+
+You can install SGLang-Diffusion using one of the methods below. The standard installation already includes SGLang's optimized kernel stack, including both `sgl-kernel` and JIT kernels used by diffusion workloads.
+
+## Standard Installation (NVIDIA GPUs)
+
+### Method 1: With pip or uv
+
+It is recommended to use uv for a faster installation:
+
+```bash
+pip install --upgrade pip
+pip install uv
+uv pip install "sglang[diffusion]" --prerelease=allow
+```
+
+### Method 2: From source
+
+```bash
+# Use the latest release branch
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Install the Python packages
+pip install --upgrade pip
+pip install -e "python[diffusion]"
+
+# With uv
+uv pip install -e "python[diffusion]" --prerelease=allow
+```
+
+### Method 3: Using Docker
+
+The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang), built from the [Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/Dockerfile).
+Replace `<secret>` below with your HuggingFace Hub [token](https://huggingface.co/docs/hub/en/security-tokens).
+
+```bash
+docker run --gpus all \
+    --shm-size 32g \
+    -p 30000:30000 \
+    -v ~/.cache/huggingface:/root/.cache/huggingface \
+    --env "HF_TOKEN=<secret>" \
+    --ipc=host \
+    lmsysorg/sglang:dev \
+    zsh -c '\
+        echo "Installing diffusion dependencies..." && \
+        pip install -e "python[diffusion]" && \
+        echo "Starting SGLang-Diffusion..." && \
+        sglang generate \
+            --model-path black-forest-labs/FLUX.1-dev \
+            --prompt "A logo With Bold Large text: SGL Diffusion" \
+            --save-output \
+    '
+```
+
+## Platform-Specific: ROCm (AMD GPUs)
+
+For AMD Instinct GPUs (e.g., MI300X), you can use the ROCm-enabled Docker image:
+
+```bash
+docker run --device=/dev/kfd --device=/dev/dri --ipc=host \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --env HF_TOKEN=<secret> \
+  lmsysorg/sglang:v0.5.5.post2-rocm700-mi30x \
+  sglang generate --model-path black-forest-labs/FLUX.1-dev --prompt "A logo With Bold Large text: SGL Diffusion" --save-output
+```
+
+For detailed ROCm system configuration and installation from source, see [AMD GPUs](../platforms/amd_gpu.md).
+
+## Platform-Specific: MUSA (Moore Threads GPUs)
+
+For Moore Threads GPUs (MTGPU) with the MUSA software stack, please follow the instructions below to install from source:
+
+```bash
+# Clone the repository
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Install the Python packages
+pip install --upgrade pip
+rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
+pip install -e "python[all_musa]"
+```
+
+## Platform-Specific: Intel XPU
+
+For Intel Data Center GPU Max or Arc GPUs, follow the [XPU installation guide](../platforms/xpu.md) to set up the base environment, then install diffusion dependencies:
+
+```bash
+pip install -e "python[diffusion]"
+```
+
+## Platform-Specific: Ascend NPU
+
+For Ascend NPU, please follow the [NPU installation guide](../platforms/ascend/ascend_npu.md).
+
+Quick test:
+
+```bash
+sglang generate --model-path black-forest-labs/FLUX.1-dev \
+    --prompt "A logo With Bold Large text: SGL Diffusion" \
+    --save-output
+```
+
+## Platform-Specific: Apple MPS
+
+For Apple MPS, please follow the instructions below to install from source:
+
+```bash
+# Install ffmpeg
+brew install ffmpeg
+
+# Install uv
+brew install uv
+
+# Clone the repository
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Create and activate a virtual environment
+uv venv -p 3.11 sglang-diffusion
+source sglang-diffusion/bin/activate
+
+# Install the Python packages
+uv pip install --upgrade pip
+rm -f python/pyproject.toml && mv python/pyproject_other.toml python/pyproject.toml
+uv pip install -e "python[all_mps]"
+```

docs/diffusion/performance/attention_backends.md

@@ -0,0 +1,166 @@
+# Attention Backends
+
+This document describes the attention backends available in sglang diffusion (`sglang.multimodal_gen`) and how to select them.
+
+## Overview
+
+Attention backends are defined by `AttentionBackendEnum` (`sglang.multimodal_gen.runtime.platforms.interface.AttentionBackendEnum`) and selected via the CLI flag `--attention-backend`.
+
+Backend selection is performed by the shared attention layers (e.g. `LocalAttention` / `USPAttention` / `UlyssesAttention` in `sglang.multimodal_gen.runtime.layers.attention.layer`) and therefore applies to any model component using these layers (e.g. diffusion transformer / DiT and encoders).
+
+When using the diffusers backend, `--attention-backend` is passed through to diffusers'
+`set_attention_backend` (e.g., `flash`, `_flash_3_hub`, `sage`, `xformers`, `native`).
+
+- **CUDA**: prefers FlashAttention (FA3/FA4) when supported; otherwise falls back to PyTorch SDPA.
+- **ROCm**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA.
+- **Intel XPU**: uses XPU Flash Attention backend (fp16/bf16, head sizes 64/96/128/192/256); otherwise falls back to PyTorch SDPA.
+- **MUSA**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA.
+- **MPS**: always uses PyTorch SDPA.
+- **NPU**: for ring attention uses FA otherwise uses PyTorch SDPA.
+
+## Backend options
+
+For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBackendEnum`. The table below lists the backends implemented by the built-in platforms. `fa3`/`fa4` are accepted as aliases for `fa`.
+
+| CLI value | Enum value | Notes |
+|---|---|---|
+| `fa` / `fa3` / `fa4` | `FA` | FlashAttention. `fa3/fa4` are normalized to `fa` during argument parsing (`ServerArgs.__post_init__`). |
+| `torch_sdpa` | `TORCH_SDPA` | PyTorch `scaled_dot_product_attention`. |
+| `sliding_tile_attn` | `SLIDING_TILE_ATTN` | Sliding Tile Attention (STA). Requires `st_attn`. Configure via `--attention-backend-config`. |
+| `sage_attn` | `SAGE_ATTN` | Requires `sageattention`. Upstream SageAttention CUDA extensions target SM80/SM86/SM89/SM90/SM120 (compute capability 8.0/8.6/8.9/9.0/12.0); see upstream `setup.py`: https://github.com/thu-ml/SageAttention/blob/main/setup.py. |
+| `sage_attn_3` | `SAGE_ATTN_3` | Requires SageAttention3 installed per upstream instructions. |
+| `video_sparse_attn` | `VIDEO_SPARSE_ATTN` | Requires `vsa`. Configure `sparsity` via `--attention-backend-config`. |
+| `vmoba_attn` | `VMOBA_ATTN` | Requires `kernel.attn.vmoba_attn.vmoba`. Configure via `--attention-backend-config`. |
+| `aiter` | `AITER` | Requires `aiter`. |
+| `aiter_sage` | `AITER_SAGE` | Requires `aiter`. |
+| `sla_attn` | `SLA_ATTN` | Sparse Linear Attention. Requires `SpargeAttn`. Install with `pip install git+https://github.com/thu-ml/SpargeAttn.git --no-build-isolation`. |
+| `sage_sla_attn` | `SAGE_SLA_ATTN` | SageAttention + Sparse Linear Attention. Requires `SpargeAttn` (same install as SLA). |
+| `sparse_video_gen_2_attn` | `SPARSE_VIDEO_GEN_2_ATTN` | Requires `svg`. See installation instructions at https://github.com/svg-project/Sparse-VideoGen. |
+| `laser_attn` | `LASER_ATTN` | Requires `attentions` which can be installed with `sgl_kernel_npu`; available only for NPU. |
+| `block_sparse_attn` | `BLOCK_SPARSE_ATTN` | Requires `attentions` which can be installed with `sgl_kernel_npu`; available only for NPU. |
+| `rain_fusion_attn` | `RAIN_FUSION_ATTN` | Requires `attentions` which can be installed with `sgl_kernel_npu`; available only for NPU. |
+
+## Selection priority
+
+The selection order in `runtime/layers/attention/selector.py` is:
+
+1. `global_force_attn_backend(...)` / `global_force_attn_backend_context_manager(...)`
+2. Component override from `--component-attention-backends` while that component is being constructed
+3. CLI `--attention-backend` (`ServerArgs.attention_backend`)
+4. Auto selection (platform capability, dtype, and installed packages)
+
+## Configuration
+
+Some backends require additional configuration. You can pass these parameters via `--attention-backend-config`. This argument accepts:
+- A path to a JSON or YAML configuration file.
+- A JSON string (e.g., `'{"sparsity": 0.5}'`).
+- Key-value pairs (e.g., `"sparsity=0.5,enable_x=true"`).
+
+### Supported Configuration Parameters
+
+**Sliding Tile Attention (`sliding_tile_attn`)**
+
+| Parameter | Type | Description | Default |
+| :--- | :--- | :--- | :--- |
+| `mask_strategy_file_path` | `str` | **Required.** Path to the mask strategy JSON file. | - |
+| `sta_mode` | `str` | Mode of STA. | `STA_inference` |
+| `skip_time_steps` | `int` | Number of steps to use full attention before switching to sparse attention. | `15` |
+
+**Video Sparse Attention (`video_sparse_attn`)**
+
+| Parameter | Type | Description | Default |
+| :--- | :--- | :--- | :--- |
+| `sparsity` | `float` | Validation sparsity (0.0 - 1.0). | `0.0` |
+
+**V-MoBA (`vmoba_attn`)**
+
+| Parameter | Type | Description | Default |
+| :--- | :--- | :--- | :--- |
+| `temporal_chunk_size` | `int` | Chunk size for temporal dimension. | - |
+| `temporal_topk` | `int` | Top-K tokens to select in temporal dimension. | - |
+| `spatial_chunk_size` | `list[int]` | Chunk size for spatial dimension (H, W). | - |
+| `spatial_topk` | `int` | Top-K tokens to select in spatial dimension. | - |
+| `st_chunk_size` | `list[int]` | Chunk size for spatiotemporal dimension (T, H, W). | - |
+| `st_topk` | `int` | Top-K tokens to select in spatiotemporal dimension. | - |
+| `moba_select_mode` | `str` | Selection mode (e.g., `threshold`). | `threshold` |
+| `moba_threshold` | `float` | Threshold value for selection. | `0.25` |
+| `moba_threshold_type` | `str` | Type of thresholding (e.g., `query_head`). | `query_head` |
+| `first_full_step` | `int` | Number of initial steps to use full attention. | `12` |
+| `first_full_layer` | `int` | Number of initial layers to use full attention. | `0` |
+| `temporal_layer` | `int` | Number of temporal layers. | `1` |
+| `spatial_layer` | `int` | Number of spatial layers. | `1` |
+| `st_layer` | `int` | Number of spatiotemporal layers. | `1` |
+
+**Block Sparse attention ( `block_sparse_attn` ) and  Rain Fusion attention ( `rain_fusion_attn` )**
+| Parameter | Type | Description | Default |
+| :--- | :--- | :--- | :--- |
+| `skip_first_steps` | `int` | Number of steps to use laser attention before switching to sparse attention. | `10` |
+| `sparsity` | `float` | The sparsity coefficient must be in the range (0, 1). | `0.2` |
+
+## Platform support matrix
+
+| Backend | CUDA | ROCm | XPU | MUSA | MPS | NPU | Notes |
+|---|---:|---:|---:|---:|---:|---:|---|
+| `fa` | ✅ | ✅ | ✅ | ✅ | ❌ | ✅ | CUDA requires SM80+ and fp16/bf16. XPU uses its own flash attention backend. FlashAttention is only used when the required runtime is installed; otherwise it falls back to `torch_sdpa`. No extra installations are required for NPU |
+| `torch_sdpa` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Most compatible option across platforms. |
+| `sliding_tile_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `st_attn`. Configure via `--attention-backend-config`. |
+| `sage_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
+| `sage_attn_3` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only (optional dependency). |
+| `video_sparse_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `vsa`. Configure `sparsity` via `--attention-backend-config`. |
+| `sla_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `SpargeAttn`. |
+| `sage_sla_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `SpargeAttn`. |
+| `vmoba_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `kernel.attn.vmoba_attn.vmoba`. Configure via `--attention-backend-config`. |
+| `aiter` | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Requires `aiter`. |
+| `aiter_sage` | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Requires `aiter`. |
+| `sparse_video_gen_2_attn` | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | CUDA-only. Requires `svg`. |
+| `laser_attn` | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | NPU-only. Requires `attentions` from `sgl_kernel_npu`. Uses SDPA if seqlen is less than 2048. |
+| `block_sparse_attn` | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | NPU-only. Requires `attentions` from `sgl_kernel_npu`.  Configure via `--attention-backend-config`. |
+| `rain_fusion_attn` | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | NPU-only. Requires `attentions` from `sgl_kernel_npu`. Configure via `--attention-backend-config`. |
+
+## Usage
+
+### Select a backend via CLI
+
+```bash
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --prompt "..." \
+  --attention-backend fa
+```
+
+```bash
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --prompt "..." \
+  --attention-backend torch_sdpa
+```
+
+### Override one component
+
+Use component overrides when a specific module needs different attention semantics from the main transformer:
+
+```bash
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --prompt "..." \
+  --attention-backend fa \
+  --component-attention-backends text_encoder=torch_sdpa
+```
+
+Component keys match pipeline module names from `model_index.json`, such as `text_encoder`, `text_encoder_2`, `transformer`, `transformer_2`, or `connectors`.
+
+### Using Sliding Tile Attention (STA)
+
+```bash
+# Pass the mask strategy file path via config
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --prompt "..." \
+  --attention-backend sliding_tile_attn \
+  --attention-backend-config "mask_strategy_file_path=/abs/path/to/mask_strategy.json"
+```
+
+### Notes for ROCm / MPS
+
+- ROCm: use `--attention-backend torch_sdpa` or `fa` depending on what is available in your environment.
+- MPS: the platform implementation always uses `torch_sdpa`.

docs/diffusion/performance/cache/cache_dit.md

@@ -0,0 +1,418 @@
+# Cache-DiT
+
+SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to **1.69x inference speedup** with minimal quality loss.
+
+## Overview
+
+**Cache-DiT** uses intelligent caching strategies to skip redundant computation in the denoising loop:
+
+- **DBCache (Dual Block Cache)**: Dynamically decides when to cache transformer blocks based on residual differences
+- **TaylorSeer**: Uses Taylor expansion for calibration to optimize caching decisions
+- **SCM (Step Computation Masking)**: Step-level caching control for additional speedup
+
+## Basic Usage
+
+Enable Cache-DiT by exporting the environment variable and using `sglang generate` or `sglang serve` :
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A beautiful sunset over the mountains"
+```
+
+## Diffusers Backend
+
+Cache-DiT supports loading acceleration configs from a custom YAML file. For
+diffusers pipelines (`diffusers` backend), pass the YAML/JSON path via `--cache-dit-config`. This
+flow requires cache-dit >= 1.2.0 (`cache_dit.load_configs`).
+
+### Single GPU inference
+
+Define a `cache.yaml` file that contains:
+
+- DBCache + TaylorSeer
+
+```yaml
+cache_config:
+  max_warmup_steps: 8
+  warmup_interval: 2
+  max_cached_steps: -1
+  max_continuous_cached_steps: 2
+  Fn_compute_blocks: 1
+  Bn_compute_blocks: 0
+  residual_diff_threshold: 0.12
+  enable_taylorseer: true
+  taylorseer_order: 1
+```
+
+Then apply the config with:
+
+```bash
+sglang generate \
+  --backend diffusers \
+  --model-path Qwen/Qwen-Image \
+  --cache-dit-config cache.yaml \
+  --prompt "A beautiful sunset over the mountains"
+```
+
+- DBCache + TaylorSeer + SCM (Step Computation Mask)
+
+```yaml
+cache_config:
+  max_warmup_steps: 8
+  warmup_interval: 2
+  max_cached_steps: -1
+  max_continuous_cached_steps: 2
+  Fn_compute_blocks: 1
+  Bn_compute_blocks: 0
+  residual_diff_threshold: 0.12
+  enable_taylorseer: true
+  taylorseer_order: 1
+  # Must set the num_inference_steps for SCM. The SCM will automatically
+  # generate the steps computation mask based on the num_inference_steps.
+  # Reference: https://cache-dit.readthedocs.io/en/latest/user_guide/CACHE_API/#scm-steps-computation-masking
+  num_inference_steps: 28
+  steps_computation_mask: fast
+```
+
+- DBCache + TaylorSeer + SCM (Step Computation Mask) + Cache CFG
+
+```yaml
+cache_config:
+  max_warmup_steps: 8
+  warmup_interval: 2
+  max_cached_steps: -1
+  max_continuous_cached_steps: 2
+  Fn_compute_blocks: 1
+  Bn_compute_blocks: 0
+  residual_diff_threshold: 0.12
+  enable_taylorseer: true
+  taylorseer_order: 1
+  num_inference_steps: 28
+  steps_computation_mask: fast
+  enable_sperate_cfg: true # e.g, Qwen-Image, Wan, Chroma, Ovis-Image, etc.
+```
+
+### Distributed inference
+
+- 1D Parallelism
+
+Define a parallelism only config yaml `parallel.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+```
+
+Then, apply the distributed inference acceleration config from yaml. `ulysses_size: auto` means that cache-dit will auto detect the `world_size` as the ulysses_size. Otherwise, you should manually set it as specific int number, e.g, 4.
+
+Then apply the distributed config with: (Note: please add `--num-gpus N` to specify the number of gpus for distributed inference)
+
+```bash
+sglang generate \
+  --backend diffusers \
+  --num-gpus 4 \
+  --model-path Qwen/Qwen-Image \
+  --cache-dit-config parallel.yaml \
+  --prompt "A futuristic cityscape at sunset"
+```
+
+- 2D Parallelism
+
+You can also define a 2D parallelism config yaml `parallel_2d.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  tp_size: 2
+  attention_backend: native
+```
+Then, apply the 2D parallelism config from yaml. Here `tp_size: 2` means using tensor parallelism with size 2. The `ulysses_size: auto` means that cache-dit will auto detect the `world_size // tp_size` as the ulysses_size.
+
+- 3D Parallelism
+
+You can also define a 3D parallelism config yaml `parallel_3d.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: 2
+  ring_size: 2
+  tp_size: 2
+  attention_backend: native
+```
+Then, apply the 3D parallelism config from yaml. Here `ulysses_size: 2`, `ring_size: 2`, `tp_size: 2` means using ulysses parallelism with size 2, ring parallelism with size 2 and tensor parallelism with size 2.
+
+- Ulysses Anything Attention
+
+To enable Ulysses Anything Attention, you can define a parallelism config yaml `parallel_uaa.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  ulysses_anything: true
+```
+
+- Ulysses FP8 Communication
+
+For device that don't have NVLink support, you can enable Ulysses FP8 Communication to further reduce the communication overhead. You can define a parallelism config yaml `parallel_fp8.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  ulysses_float8: true
+```
+
+- Async Ulysses CP
+
+You can also enable async ulysses CP to overlap the communication and computation. Define a parallelism config yaml `parallel_async.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  ulysses_async: true # Now, only support for FLUX.1, Qwen-Image, Ovis-Image and Z-Image.
+```
+Then, apply the config from yaml. Here `ulysses_async: true` means enabling async ulysses CP.
+
+- TE-P and VAE-P
+
+You can also specify the extra parallel modules in the yaml config. For example, define a parallelism config yaml `parallel_extra.yaml` file that contains:
+
+```yaml
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  extra_parallel_modules: ["text_encoder", "vae"]
+```
+
+
+### Hybrid Cache and Parallelism
+
+Define a hybrid cache and parallel acceleration config yaml `hybrid.yaml` file that contains:
+
+```yaml
+cache_config:
+  max_warmup_steps: 8
+  warmup_interval: 2
+  max_cached_steps: -1
+  max_continuous_cached_steps: 2
+  Fn_compute_blocks: 1
+  Bn_compute_blocks: 0
+  residual_diff_threshold: 0.12
+  enable_taylorseer: true
+  taylorseer_order: 1
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  extra_parallel_modules: ["text_encoder", "vae"]
+```
+
+Then, apply the hybrid cache and parallel acceleration config from yaml.
+
+```bash
+sglang generate \
+  --backend diffusers \
+  --num-gpus 4 \
+  --model-path Qwen/Qwen-Image \
+  --cache-dit-config hybrid.yaml \
+  --prompt "A beautiful sunset over the mountains"
+```
+
+### Attention Backend
+
+In some cases, users may want to only specify the attention backend without any other optimization configs. In this case, you can define a yaml file `attention.yaml` that only contains:
+
+```yaml
+attention_backend: "flash" # '_flash_3' for Hopper
+```
+
+### Quantization
+
+You can also specify the quantization config in the yaml file, required `torchao>=0.16.0`. For example, define a yaml file `quantize.yaml` that contains:
+
+```yaml
+quantize_config: # quantization configuration for transformer modules
+  # float8 (DQ), float8_weight_only, float8_blockwise, int8 (DQ), int8_weight_only, etc.
+  quant_type: "float8"
+  # layers to exclude from quantization (transformer). layers that contains any of the
+  # keywords in the exclude_layers list will be excluded from quantization. This is useful
+  # for some sensitive layers that are not robust to quantization, e.g., embedding layers.
+  exclude_layers:
+    - "embedder"
+    - "embed"
+  verbose: false # whether to print verbose logs during quantization
+```
+Then, apply the quantization config from yaml. Please also enable torch.compile for better performance if you are using quantization. For example:
+
+```bash
+sglang generate \
+  --backend diffusers \
+  --model-path Qwen/Qwen-Image \
+  --warmup \
+  --cache-dit-config quantize.yaml \
+  --enable-torch-compile \
+  --dit-cpu-offload false \
+  --text-encoder-cpu-offload false \
+  --prompt "A beautiful sunset over the mountains"
+```
+
+### Combined Configs: Cache + Parallelism + Quantization
+
+You can also combine all the above configs together in a single yaml file `combined.yaml` that contains:
+
+```yaml
+cache_config:
+  max_warmup_steps: 8
+  warmup_interval: 2
+  max_cached_steps: -1
+  max_continuous_cached_steps: 2
+  Fn_compute_blocks: 1
+  Bn_compute_blocks: 0
+  residual_diff_threshold: 0.12
+  enable_taylorseer: true
+  taylorseer_order: 1
+parallelism_config:
+  ulysses_size: auto
+  attention_backend: native
+  extra_parallel_modules: ["text_encoder", "vae"]
+quantize_config:
+  quant_type: "float8"
+  exclude_layers:
+    - "embedder"
+    - "embed"
+  verbose: false
+```
+Then, apply the combined cache, parallelism and quantization config from yaml. Please also enable torch.compile for better performance if you are using quantization.
+
+## Advanced Configuration
+
+### DBCache Parameters
+
+DBCache controls block-level caching behavior:
+
+| Parameter | Env Variable              | Default | Description                              |
+|-----------|---------------------------|---------|------------------------------------------|
+| Fn        | `SGLANG_CACHE_DIT_FN`     | 1       | Number of first blocks to always compute |
+| Bn        | `SGLANG_CACHE_DIT_BN`     | 0       | Number of last blocks to always compute  |
+| W         | `SGLANG_CACHE_DIT_WARMUP` | 4       | Warmup steps before caching starts       |
+| R         | `SGLANG_CACHE_DIT_RDT`    | 0.24    | Residual difference threshold            |
+| MC        | `SGLANG_CACHE_DIT_MC`     | 3       | Maximum continuous cached steps          |
+
+### TaylorSeer Configuration
+
+TaylorSeer improves caching accuracy using Taylor expansion:
+
+| Parameter | Env Variable                  | Default | Description                     |
+|-----------|-------------------------------|---------|---------------------------------|
+| Enable    | `SGLANG_CACHE_DIT_TAYLORSEER` | false   | Enable TaylorSeer calibrator    |
+| Order     | `SGLANG_CACHE_DIT_TS_ORDER`   | 1       | Taylor expansion order (1 or 2) |
+
+### Combined Configuration Example
+
+DBCache and TaylorSeer are complementary strategies that work together, you can configure both sets of parameters
+simultaneously:
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_FN=2 \
+SGLANG_CACHE_DIT_BN=1 \
+SGLANG_CACHE_DIT_WARMUP=4 \
+SGLANG_CACHE_DIT_RDT=0.4 \
+SGLANG_CACHE_DIT_MC=4 \
+SGLANG_CACHE_DIT_TAYLORSEER=true \
+SGLANG_CACHE_DIT_TS_ORDER=2 \
+sglang generate --model-path black-forest-labs/FLUX.1-dev \
+    --prompt "A curious raccoon in a forest"
+```
+
+### SCM (Step Computation Masking)
+
+SCM provides step-level caching control for additional speedup. It decides which denoising steps to compute fully and
+which to use cached results.
+
+**SCM Presets**
+
+SCM is configured with presets:
+
+| Preset   | Compute Ratio | Speed    | Quality    |
+|----------|---------------|----------|------------|
+| `none`   | 100%          | Baseline | Best       |
+| `slow`   | ~75%          | ~1.3x    | High       |
+| `medium` | ~50%          | ~2x      | Good       |
+| `fast`   | ~35%          | ~3x      | Acceptable |
+| `ultra`  | ~25%          | ~4x      | Lower      |
+
+**Usage**
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_SCM_PRESET=medium \
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A futuristic cityscape at sunset"
+```
+
+**Custom SCM Bins**
+
+For fine-grained control over which steps to compute vs cache:
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+SGLANG_CACHE_DIT_SCM_COMPUTE_BINS="8,3,3,2,2" \
+SGLANG_CACHE_DIT_SCM_CACHE_BINS="1,2,2,2,3" \
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A futuristic cityscape at sunset"
+```
+
+**SCM Policy**
+
+| Policy    | Env Variable                          | Description                                 |
+|-----------|---------------------------------------|---------------------------------------------|
+| `dynamic` | `SGLANG_CACHE_DIT_SCM_POLICY=dynamic` | Adaptive caching based on content (default) |
+| `static`  | `SGLANG_CACHE_DIT_SCM_POLICY=static`  | Fixed caching pattern                       |
+
+## Environment Variables
+
+All Cache-DiT parameters can be configured via environment variables.
+See [Environment Variables](../../environment_variables.md) for the complete list.
+
+## Supported Models
+
+SGLang Diffusion x Cache-DiT supports almost all models originally supported in SGLang Diffusion:
+
+| Model Family | Example Models              |
+|--------------|-----------------------------|
+| Wan          | Wan2.1, Wan2.2              |
+| Flux         | FLUX.1-dev, FLUX.2-dev      |
+| Z-Image      | Z-Image-Turbo               |
+| Qwen         | Qwen-Image, Qwen-Image-Edit |
+| Hunyuan      | HunyuanVideo                |
+
+## Performance Tips
+
+1. **Start with defaults**: The default parameters work well for most models
+2. **Use TaylorSeer**: It typically improves both speed and quality
+3. **Tune R threshold**: Lower values = better quality, higher values = faster
+4. **SCM for extra speed**: Use `medium` preset for good speed/quality balance
+5. **Warmup matters**: Higher warmup = more stable caching decisions
+
+## Limitations
+
+- **SGLang-native pipelines**: Distributed support (TP/SP) is not yet validated; Cache-DiT will be automatically
+  disabled when `world_size > 1`.
+- **SCM minimum steps**: SCM requires >= 8 inference steps to be effective
+- **Model support**: Only models registered in Cache-DiT's BlockAdapterRegister are supported
+
+## Troubleshooting
+
+### SCM disabled for low step count
+
+For models with < 8 inference steps (e.g., DMD distilled models), SCM will be automatically disabled. DBCache
+acceleration still works.
+
+## References
+
+- [Cache-DiT](https://github.com/vipshop/cache-dit)
+- [SGLang Diffusion](../index.md)

docs/diffusion/performance/cache/index.md

@@ -0,0 +1,65 @@
+# Caching Acceleration
+
+SGLang provides two complementary caching strategies for Diffusion Transformer (DiT) models. Both reduce denoising cost by skipping redundant computation, but they operate at different levels.
+
+## Overview
+
+SGLang supports two complementary caching approaches:
+
+| Strategy | Scope | Mechanism | Best For |
+|----------|-------|-----------|----------|
+| **Cache-DiT** | Block-level | Skip individual transformer blocks dynamically | Advanced, higher speedup |
+| **TeaCache** | Timestep-level | Skip entire denoising steps based on L1 similarity | Simple, built-in |
+
+## Cache-DiT
+
+[Cache-DiT](https://github.com/vipshop/cache-dit) provides block-level caching with
+advanced strategies like DBCache and TaylorSeer. It can achieve up to **1.69x speedup**.
+
+See [cache_dit.md](cache_dit.md) for detailed configuration.
+
+### Quick Start
+
+```bash
+SGLANG_CACHE_DIT_ENABLED=true \
+sglang generate --model-path Qwen/Qwen-Image \
+    --prompt "A beautiful sunset over the mountains"
+```
+
+### Key Features
+
+- **DBCache**: Dynamic block-level caching based on residual differences
+- **TaylorSeer**: Taylor expansion-based calibration for optimized caching
+- **SCM**: Step-level computation masking for additional speedup
+
+## TeaCache
+
+TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
+
+See [teacache.md](teacache.md) for detailed documentation.
+
+### Quick Overview
+
+- Tracks L1 distance between modulated inputs across timesteps
+- When accumulated distance is below threshold, reuses cached residual
+- Supports CFG with separate positive/negative caches
+
+### Supported Models
+
+- Wan (wan2.1, wan2.2)
+- Hunyuan (HunyuanVideo)
+- Z-Image
+
+For Flux and Qwen models, TeaCache is automatically disabled when CFG is enabled.
+
+```{toctree}
+:maxdepth: 1
+
+cache_dit
+teacache
+```
+
+## References
+
+- [Cache-DiT Repository](https://github.com/vipshop/cache-dit)
+- [TeaCache Paper](https://arxiv.org/abs/2411.14324)

docs/diffusion/performance/cache/teacache.md

@@ -0,0 +1,84 @@
+# TeaCache
+
+> **Note**: This is one of two caching strategies available in SGLang.
+> For an overview of all caching options, see [caching](../index.md).
+
+TeaCache (Temporal similarity-based caching) accelerates diffusion inference by detecting when consecutive denoising steps are similar enough to skip computation entirely.
+
+## Overview
+
+TeaCache works by:
+1. Tracking the L1 distance between modulated inputs across consecutive timesteps
+2. Accumulating the rescaled L1 distance over steps
+3. When accumulated distance is below a threshold, reusing the cached residual
+4. Supporting CFG (Classifier-Free Guidance) with separate positive/negative caches
+
+## How It Works
+
+### L1 Distance Tracking
+
+At each denoising step, TeaCache computes the relative L1 distance between the current and previous modulated inputs:
+
+```
+rel_l1 = |current - previous|.mean() / |previous|.mean()
+```
+
+This distance is then rescaled using polynomial coefficients and accumulated:
+
+```
+accumulated += poly(coefficients)(rel_l1)
+```
+
+### Cache Decision
+
+- If `accumulated >= threshold`: Force computation, reset accumulator
+- If `accumulated < threshold`: Skip computation, use cached residual
+
+### CFG Support
+
+For models that support CFG cache separation (Wan, Hunyuan, Z-Image), TeaCache maintains separate caches for positive and negative branches:
+- `previous_modulated_input` / `previous_residual` for positive branch
+- `previous_modulated_input_negative` / `previous_residual_negative` for negative branch
+
+For models that don't support CFG separation (Flux, Qwen), TeaCache is automatically disabled when CFG is enabled.
+
+## Configuration
+
+TeaCache is configured via `TeaCacheParams` in the sampling parameters:
+
+```python
+from sglang.multimodal_gen.configs.sample.teacache import TeaCacheParams
+
+params = TeaCacheParams(
+    teacache_thresh=0.1,           # Threshold for accumulated L1 distance
+    coefficients=[1.0, 0.0, 0.0],  # Polynomial coefficients for L1 rescaling
+)
+```
+
+### Parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `teacache_thresh` | float | Threshold for accumulated L1 distance. Lower = more caching, faster but potentially lower quality |
+| `coefficients` | list[float] | Polynomial coefficients for L1 rescaling. Model-specific tuning |
+
+### Model-Specific Configurations
+
+Different models may have different optimal configurations. The coefficients are typically tuned per-model to balance speed and quality.
+
+## Supported Models
+
+TeaCache is built into the following model families:
+
+| Model Family | CFG Cache Separation | Notes |
+|--------------|---------------------|-------|
+| Wan (wan2.1, wan2.2) | Yes | Full support |
+| Hunyuan (HunyuanVideo) | Yes | To be supported |
+| Z-Image | Yes | To be supported |
+| Flux | No | To be supported |
+| Qwen | No | To be supported |
+
+
+## References
+
+- [TeaCache: Accelerating Diffusion Models with Temporal Similarity](https://arxiv.org/abs/2411.14324)

docs/diffusion/performance/deployment_cookbook.md

@@ -0,0 +1,94 @@
+# Deployment Cookbook
+
+This page gives practical defaults for choosing CPU offload, FSDP, CFG parallelism, SP, and TP.
+
+## Quick Rule
+
+Use the simplest setting that fits your memory target:
+
+| Goal                                       | Recommended setting                                                                |
+|--------------------------------------------|------------------------------------------------------------------------------------|
+| Fastest single-GPU run when the model fits | Disable CPU offload and do not use FSDP.                                           |
+| Lower single-GPU memory usage              | Use component CPU offload, or layerwise DiT offload for supported Wan/MOVA models. |
+| Faster multi-GPU Qwen/Wan CFG generation   | Use FSDP with CFG parallelism and disable CPU offload.                             |
+| Sequence length or video-shape scaling     | Use SP/Ulysses/Ring when the model benefits from sequence parallelism.             |
+| TP compatibility or encoder-heavy paths    | Set TP explicitly; do not treat TP as the default latency optimization.            |
+
+Base the decision on available memory on the selected GPU(s).
+
+- For multi-GPU deployment: the least-free selected GPU is the bottleneck. A busy 80GiB GPU can behave like a much smaller GPU.
+- For single-GPU deployment: FSDP shards DiT weights across multiple GPUs. It is not useful for keeping a single-GPU deployment on one GPU; for that case use CPU offload.
+
+## Performance Modes
+
+`--performance-mode` applies safe presets without overriding explicit offload, FSDP, or parallelism flags. `auto` is the default. Use `manual` when you need to keep performance-related server args under explicit user control. `--mode` is a short alias.
+
+| Mode       | Meaning                                                                                                                   |
+|------------|---------------------------------------------------------------------------------------------------------------------------|
+| `manual`   | Keeps performance-related server args under explicit user control.                                                       |
+| `auto`     | Default. Keeps legacy safe offload defaults and uses FSDP/CFG only on validated multi-GPU deployments where FSDP can replace DiT offload. |
+| `speed`    | Favors GPU-resident execution for lower latency and higher throughput. Disables CPU offload when unset; may OOM.           |
+| `memory`   | Favors lower GPU memory. Uses component offload, or Wan/MOVA layerwise DiT offload when supported.                         |
+
+`auto` checks selected GPU memory before applying FSDP. In multi-GPU runs it uses the least available memory across selected GPUs, and only turns on FSDP automatically when doing so can replace DiT offload. Text encoder, image encoder, and other component residency still follow the offload policy unless the model marks a high-memory resident path as safe. When the model default uses CFG and the user did not set a parallelism policy, `auto` may also enable CFG parallelism. `speed` intentionally does not check memory; it is the mode for users who prefer latency/throughput and accept OOM risk.
+
+The modes tune residency for native pipeline components declared to the component residency manager. Today this covers the major DiT, text/image encoder, VAE, vocoder, and upsampler components; DiT can use layerwise offload when supported, while text encoders use either resident execution or component CPU offload. Do not assume text-encoder layerwise offload unless a model implements and validates it.
+
+NOTE:
+The preset is intentionally coarse. A future continuous value such as `0.0` to `1.0` could express the speed-memory tradeoff more precisely, but it would need model-specific memory models and clearer user expectations. Until then, use the preset plus explicit flags for overrides.
+
+Examples:
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --num-gpus 2 \
+  --performance-mode auto
+```
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
+  --performance-mode memory
+```
+
+Explicit flags win over the mode:
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --num-gpus 2 \
+  --performance-mode auto \
+  --use-fsdp-inference false
+```
+
+In this example, `auto` will not re-enable FSDP. The same applies to parallelism; for example, `--enable-cfg-parallel false` keeps CFG parallelism disabled.
+
+## Interpreting The Levers
+
+**No offload** keeps model components resident on GPU. It is usually fastest when memory is sufficient.
+
+**Component CPU offload** lowers GPU memory by moving large components to CPU. It is simple and robust, but it usually trades latency for memory.
+
+**Layerwise DiT offload** lowers DiT memory further for supported Wan/MOVA models by moving DiT layers between CPU and GPU. It can be the best single-GPU memory mode, but may increase latency and lower throughput.
+
+**FSDP** shards DiT weights across multiple GPUs and all-gathers weights during forward. It can reduce DiT CPU offload cost on multi-GPU deployments, especially for validated Wan I2V workloads.
+
+FSDP sharding granularity matters. SGLang Diffusion prefers sharding direct repeated transformer block entries such as `transformer_blocks.0` or `blocks.0`. Coarser sharding lowers wrapper count but can increase all-gather peak memory; finer sharding can reduce transient memory but adds communication and scheduling overhead. If a model does not define an explicit sharding rule, the loader falls back to repeated block class names and common direct numbered block paths.
+
+**CFG parallelism** splits positive and negative CFG branches across GPUs. For Qwen/Wan workloads with normal step counts, this is the most reliable multi-GPU speedup observed so far.
+
+**SP/Ulysses/Ring** splits sequence work. It can help video workloads, but validated Qwen/Wan runs showed CFG parallelism outperforming SP for latency.
+
+**TP** is supported for compatibility and some model structures, but current measurements do not make it the default latency path for Qwen/Wan.
+
+## Current Benchmark Takeaways
+
+Observed regular-scale trends:
+
+- Z-Image: single-GPU no-offload was faster than FSDP/SP in the tested setting; keep FSDP off unless memory or parallelism requires it.
+- Qwen-Image: keep the default non-FSDP path unless a specific FSDP/SP/Ring setting has been benchmarked on the target hardware.
+- Wan: FSDP can replace DiT offload on validated multi-GPU workloads, while text/image encoders may still need component offload. Keep model-specific precision checks before making FSDP automatic for a path.
+- Component offload mainly reduced memory; it did not improve latency in the tested no-offload-vs-offload runs.
+
+Always benchmark with your actual resolution, frame count, step count, and GPU type before locking production defaults.

docs/diffusion/performance/index.md

@@ -0,0 +1,44 @@
+# Performance
+
+This section covers the main performance levers for SGLang Diffusion: attention backends, caching acceleration, and profiling.
+
+## Overview
+
+| Optimization | Type | Description |
+|--------------|------|-------------|
+| **Cache-DiT** | Caching | Block-level caching with DBCache, TaylorSeer, and SCM |
+| **TeaCache** | Caching | Timestep-level caching based on temporal similarity |
+| **Attention Backends** | Kernel | Optimized attention implementations (FlashAttention, SageAttention, etc.) |
+| **Profiling** | Diagnostics | PyTorch Profiler and Nsight Systems guidance |
+
+## Start Here
+
+- Use [Attention Backends](attention_backends.md) to choose the best backend for your model and hardware.
+- Use [Deployment Cookbook](deployment_cookbook.md) to choose CPU offload, FSDP, CFG parallelism, SP, and TP.
+- Use [Caching Acceleration](cache/index.md) to reduce denoising cost with Cache-DiT or TeaCache.
+- Use [Profiling](profiling.md) when you need to diagnose a bottleneck rather than guess.
+
+## Caching at a Glance
+
+- [Cache-DiT](cache/cache_dit.md) is block-level caching for diffusers pipelines and higher speedup-oriented tuning.
+- [TeaCache](cache/teacache.md) is timestep-level caching built into SGLang model families.
+
+```{toctree}
+:maxdepth: 1
+
+attention_backends
+deployment_cookbook
+cache/index
+profiling
+```
+
+## Current Baseline Snapshot
+
+For Ring SP benchmark details, see:
+
+- [Ring SP Performance](ring_sp_performance.md)
+
+## References
+
+- [Cache-DiT Repository](https://github.com/vipshop/cache-dit)
+- [TeaCache Paper](https://arxiv.org/abs/2411.14324)

docs/diffusion/performance/profiling.md

@@ -0,0 +1,136 @@
+# Profiling Multimodal Generation
+
+This guide covers profiling techniques for multimodal generation pipelines in SGLang.
+
+## PyTorch Profiler
+
+PyTorch Profiler provides detailed kernel execution time, call stack, and GPU utilization metrics.
+
+### Denoising Stage Profiling
+
+Profile the denoising stage with sampled timesteps (default: 5 steps after 1 warmup step):
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --seed 0 \
+  --profile
+```
+
+**Parameters:**
+- `--profile`: Enable profiling for the denoising stage
+- `--num-profiled-timesteps N`: Number of timesteps to profile after warmup (default: 5)
+  - Smaller values reduce trace file size
+  - Example: `--num-profiled-timesteps 10` profiles 10 steps after 1 warmup step
+
+### Full Pipeline Profiling
+
+Profile all pipeline stages (text encoding, denoising, VAE decoding, etc.):
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --seed 0 \
+  --profile \
+  --profile-all-stages
+```
+
+**Parameters:**
+- `--profile-all-stages`: Used with `--profile`, profile all pipeline stages instead of just denoising
+
+### Output Location
+
+By default, trace files are saved in the ./logs/ directory.
+
+The exact output file path will be shown in the console output, for example:
+
+```bash
+[mm-dd hh:mm:ss] Saved profiler traces to: /sgl-workspace/sglang/logs/mocked_fake_id_for_offline_generate-5_steps-global-rank0.trace.json.gz
+```
+
+### View Traces
+
+Load and visualize trace files at:
+- https://ui.perfetto.dev/ (recommended)
+- chrome://tracing (Chrome only)
+
+For large trace files, reduce `--num-profiled-timesteps` or avoid using `--profile-all-stages`.
+
+
+### `--perf-dump-path` (Stage/Step Timing Dump)
+
+Besides profiler traces, you can also dump a lightweight JSON report that contains:
+- stage-level timing breakdown for the full pipeline
+- step-level timing breakdown for the denoising stage (per diffusion step)
+
+This is useful to quickly identify which stage dominates end-to-end latency, and whether denoising steps have uniform runtimes (and if not, which step has an abnormal spike).
+
+The dumped JSON contains a `denoise_steps_ms` field formatted as an array of objects, each with a `step` key (the step index) and a `duration_ms` key.
+
+Example:
+
+```bash
+sglang generate \
+  --model-path <MODEL_PATH_OR_ID> \
+  --prompt "<PROMPT>" \
+  --perf-dump-path perf.json
+```
+
+## Nsight Systems
+
+Nsight Systems provides low-level CUDA profiling with kernel details, register usage, and memory access patterns.
+
+### Installation
+
+See the [SGLang profiling guide](https://github.com/sgl-project/sglang/blob/main/docs/developer_guide/benchmark_and_profiling.md#profile-with-nsight) for installation instructions.
+
+### Basic Profiling
+
+Profile the entire pipeline execution:
+
+```bash
+nsys profile \
+  --trace-fork-before-exec=true \
+  --cuda-graph-trace=node \
+  --force-overwrite=true \
+  -o QwenImage \
+  sglang generate \
+    --model-path Qwen/Qwen-Image \
+    --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+    --seed 0
+```
+
+### Targeted Stage Profiling
+
+Use `--delay` and `--duration` to capture specific stages and reduce file size:
+
+```bash
+nsys profile \
+  --trace-fork-before-exec=true \
+  --cuda-graph-trace=node \
+  --force-overwrite=true \
+  --delay 10 \
+  --duration 30 \
+  -o QwenImage_denoising \
+  sglang generate \
+    --model-path Qwen/Qwen-Image \
+    --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+    --seed 0
+```
+
+**Parameters:**
+- `--delay N`: Wait N seconds before starting capture (skip initialization overhead)
+- `--duration N`: Capture for N seconds (focus on specific stages)
+- `--force-overwrite`: Overwrite existing output files
+
+## Notes
+
+- **Reduce trace size**: Use `--num-profiled-timesteps` with smaller values or `--delay`/`--duration` with Nsight Systems
+- **Stage-specific analysis**: Use `--profile` alone for denoising stage, add `--profile-all-stages` for full pipeline
+- **Multiple runs**: Profile with different prompts and resolutions to identify bottlenecks across workloads
+
+## FAQ
+
+- If you are profiling `sglang generate` with Nsight Systems and find that the generated profiler file did not capture any CUDA kernels, you can resolve this issue by increasing the model's inference steps to extend the execution time.

docs/diffusion/performance/ring_sp_performance.md

@@ -0,0 +1,67 @@
+# Ring SP Benchmark: Wan2.2-TI2V-5B (u1r2 vs Baseline)
+
+This page reports Ring-SP performance for `Wan2.2-TI2V-5B-Diffusers` using:
+
+- Parallel config: `sp=2, ulysses=1, ring=2` (short: `u1r2`)
+- Baseline config: `sp=1, ulysses=1, ring=1` (short: `u1r1`)
+
+## Benchmark Setup
+
+- Model: `Wan2.2-TI2V-5B-Diffusers`
+- GPU: `48G RTX40 series * 2`
+
+## Online Serving
+
+### Ring SP (`u1r2`)
+
+```bash
+sglang serve \
+  --model-type diffusion \
+  --model-path /model/HuggingFace/Wan-AI/Wan2.2-TI2V-5B-Diffusers \
+  --num-gpus 2 --sp-degree 2 --ulysses-degree 1 --ring-degree 2 \
+  --port 8898
+```
+
+### Baseline (`u1r1`)
+
+```bash
+sglang serve \
+  --model-type diffusion \
+  --model-path /model/HuggingFace/Wan-AI/Wan2.2-TI2V-5B-Diffusers \
+  --num-gpus 1 --sp-degree 1 --ulysses-degree 1 --ring-degree 1 \
+  --port 8898
+```
+
+## Benchmarks
+
+### Benchmark Disclaimer
+
+These benchmarks are provided for reference under one specific setup and command configuration. Actual performance may vary with model settings, runtime environment, and request patterns.
+
+### Stage Time Breakdown
+
+| Stage / Metric | `u1r2` (s) | `u1r1` baseline (s) | Speedup |
+|---|---:|---:|---:|
+| InputValidation | 0.1060 | 0.1029 | 0.97x |
+| TextEncoding | 1.3965 | 2.2261 | 1.59x |
+| LatentPreparation | 0.0002 | 0.0002 | 1.00x |
+| TimestepPreparation | 0.0003 | 0.0004 | 1.33x |
+| Denoising | 52.6358 | 71.6785 | 1.36x |
+| Decoding | 7.6708 | 13.4314 | 1.75x |
+| **Total** | **63.74** | **90.63** | **1.42x** |
+
+### Memory Usage
+
+| Memory Metric | `u1r2` (GB) | `u1r1` baseline (GB) | Delta |
+|---|---:|---:|---:|
+| Peak GPU Memory | 20.07 | 27.40 | -7.33 |
+| Peak Allocated | 13.35 | 20.40 | -7.05 |
+| Memory Overhead | 6.72 | 7.00 | -0.28 |
+| Overhead Ratio | 33.5% | 25.6% | +7.9pp |
+
+## Summary
+
+- End-to-end latency improves from `90.63s` to `63.74s` (`1.42x`).
+- Main gains come from `Denoising` (`1.36x`) and `Decoding` (`1.75x`).
+- Absolute memory usage drops noticeably on Ring-SP (`Peak GPU Memory -7.33GB`, `Peak Allocated -7.05GB`).
+- Overhead ratio rises (`+7.9pp`), so future tuning can focus on reducing communication/runtime overhead while preserving the latency gain.

docs/diffusion/quantization.md

@@ -0,0 +1,447 @@
+# Quantization
+
+SGLang-Diffusion supports quantized transformer checkpoints. In most cases, keep
+the base model and the quantized transformer override separate.
+
+## Quick Reference
+
+Use these paths:
+
+- `--model-path`: the base or original model
+- `--transformer-path`: a quantized transformers-style transformer component directory that already contains its own `config.json`
+- `--transformer-weights-path`: quantized transformer weights provided as a single safetensors file, a sharded safetensors directory, a local path, or a Hugging Face repo ID
+- `--quantization`: apply online quantization to unquantized models at load time (activations are quantized dynamically)
+- `--quantization-ignored-layers` layer name patterns to keep unquantized (e.g. `attention.to_`)
+
+Recommended example for pre-quantized checkpoints:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --transformer-weights-path black-forest-labs/FLUX.2-dev-NVFP4 \
+  --prompt "a curious pikachu"
+```
+
+For quantized transformers-style transformer component folders:
+
+```bash
+sglang generate \
+  --model-path /path/to/base-model \
+  --transformer-path /path/to/quantized-transformer \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion"
+```
+
+NOTE: Some model-specific integrations also accept a quantized repo or local
+directory directly as `--model-path`, but that is a compatibility path. If a
+repo contains multiple candidate checkpoints, pass
+`--transformer-weights-path` explicitly.
+
+## Quant Families
+
+Here, `quant_family` means a checkpoint and loading family with shared CLI
+usage and loader behavior. It is not just the numeric precision or a kernel
+backend.
+
+| quant_family     | checkpoint form                                                                            | canonical CLI                                        | supported models                                             | extra dependency                      | platform / notes                                                                                                      |
+|------------------|--------------------------------------------------------------------------------------------|------------------------------------------------------|--------------------------------------------------------------|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------|
+| `fp8` / `mxfp4` (online quantization)  | Unquantized checkpoint (offline via AMD Quark coming soon) | `--quantization {fp8,mxfp4}` | Z-Image-Turbo (validated), others likely work. More support coming soon. | MXFP4: `aiter` on ROCm | MXFP4 requires ROCm and MI350+ (gfx95x). Weights quantized at load time, activations quantized to `fp8` / `mxfp4` dynamically. |
+| `fp8` (offline quantization) | Quantized transformer component folder, or safetensors with `quantization_config` metadata | `--transformer-path` or `--transformer-weights-path`                   | ALL                                     | None                                  | Component-folder and single-file flows are both supported                                                                              |
+| `modelopt-fp8`    | Converted ModelOpt FP8 transformer directory or repo with `config.json`                    | `--transformer-path`                                                    | FLUX.1, FLUX.2, Wan2.2, HunyuanVideo, Qwen Image, Qwen Image Edit | None                                  | Serialized config stays `quant_method=modelopt` with `quant_algo=FP8`; `dit_layerwise_offload` is supported and `dit_cpu_offload` stays disabled |
+| `modelopt-nvfp4`  | Mixed transformer directory/repo with `config.json`, or raw NVFP4 safetensors export/repo | `--transformer-path` for mixed overrides; `--transformer-weights-path` for raw exports | FLUX.1, FLUX.2, Wan2.2                  | None                                  | Mixed override repos keep the base model separate; raw exports such as `black-forest-labs/FLUX.2-dev-NVFP4` still use the weights-path flow |
+| `nunchaku-svdq`   | Pre-quantized Nunchaku transformer weights, usually named `svdq-{int4\|fp4}_r{rank}-...`   | `--transformer-weights-path`                                           | Model-specific support such as Qwen-Image, FLUX, and Z-Image | `nunchaku`                            | SGLang can infer precision and rank from the filename and supports both `int4` and `nvfp4`                                             |
+| `msmodelslim`     | Pre-quantized msmodelslim transformer weights                                              | `--model-path`                                                         | Wan2.2 family                           | None                                  | Currently only compatible with the Ascend NPU family and supports both `w8a8` and `w4a4`                                               |
+
+## Online Quantization
+
+Online quantization applies quantization to unquantized models at load time. This is useful for when pre-quantized checkpoints are not available.
+
+### FP8 Online Quantization
+
+Apply FP8 quantization to any unquantized model:
+
+```bash
+sglang generate \
+  --model-path Tongyi-MAI/Z-Image-Turbo \
+  --quantization fp8 \
+  --prompt "a beautiful sunset" \
+  --save-output
+```
+
+### MXFP4 Online Quantization
+
+MXFP4 provides aggressive 4-bit compression with online quantization. **Note: Requires ROCm and MI350+ (gfx95x) GPU.**
+
+```bash
+sglang generate \
+  --model-path Tongyi-MAI/Z-Image-Turbo \
+  --quantization mxfp4 \
+  --prompt "a beautiful sunset" \
+  --save-output
+```
+**Note:** Requires `aiter` package with MXFP4 kernel support
+
+### Skipping Layers
+
+By default, online quantization quantizes every linear layer in
+the transformer. However, `--quantization-ignored-layers` can be used to keep specific layers in their original precision:
+
+```bash
+sglang generate \
+  --model-path Tongyi-MAI/Z-Image-Turbo \
+  --quantization fp8 \
+  --quantization-ignored-layers attention.to_ \
+  --prompt "a beautiful sunset" \
+  --save-output
+
+sglang generate \
+  --model-path Tongyi-MAI/Z-Image-Turbo \
+  --quantization mxfp4 \
+  --quantization-ignored-layers attention.to_ \
+  --prompt "a beautiful sunset" \
+  --save-output
+```
+
+Each pattern is matched against the full layer prefix (e.g. `layers.0.attention.to_q`). A layer is skipped and left unquantizd if its prefix contains any of the given patterns.
+
+## Validated ModelOpt Checkpoints
+
+This section is the canonical support matrix for the nine diffusion ModelOpt
+checkpoints currently wired up in SGLang docs and validation coverage.
+
+Published checkpoints keep the serialized quantization config as
+`quant_method=modelopt`; the FP8 vs NVFP4 split below is a documentation label
+derived from `quant_algo`.
+
+Eight of the nine repos live under `lmsys/*`. The FLUX.2 NVFP4 entry keeps the
+official `black-forest-labs/FLUX.2-dev-NVFP4` repo.
+
+| Quant Algo | Base Model | Preferred CLI | HF Repo | Current Scope | Notes |
+| --- | --- | --- | --- | --- | --- |
+| `FP8` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-fp8-sglang-transformer` | single-transformer override, deterministic latent/image comparison, H100 benchmark, torch-profiler trace | SGLang converter keeps a validated BF16 fallback set for modulation and FF projection layers; use `--model-id FLUX.1-dev` for local mirrors |
+| `FP8` | `black-forest-labs/FLUX.2-dev` | `--transformer-path` | `lmsys/flux2-dev-modelopt-fp8-sglang-transformer` | single-transformer override load and generation path | published SGLang-ready transformer override |
+| `FP8` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer` | primary `transformer` quantized, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint, and do not describe this as dual-transformer full-model FP8 unless that path is validated separately |
+| `FP8` | `hunyuanvideo-community/HunyuanVideo` | `--transformer-path` | `lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer` | single-transformer override, BF16-vs-FP8 video comparison, H100 benchmark, torch-profiler trace | HunyuanVideo uses different ModelOpt/diffusers and SGLang runtime module names; the converter maps those names before writing FP8 scale tensors and BF16 fallback ignores |
+| `FP8` | `Qwen/Qwen-Image` | `--transformer-path` | `lmsys/qwen-image-modelopt-fp8-sglang-transformer` | single-transformer override, BF16-vs-FP8 image comparison, H100 benchmark, torch-profiler trace | shares the Qwen Image FP8 fallback preset; keep `img_in`, `txt_in`, timestep embedder, `norm_out.linear`, `proj_out`, `img_mod`/`txt_mod`, and `img_mlp.net.2` in BF16 |
+| `FP8` | `Qwen/Qwen-Image-Edit-2511` | `--transformer-path` | `lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer` | TI2I edit path, BF16-vs-FP8 image comparison, H100 benchmark | shares `QwenImageTransformer2DModel` with Qwen Image and uses the same Qwen Image FP8 fallback preset |
+| `NVFP4` | `black-forest-labs/FLUX.1-dev` | `--transformer-path` | `lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer` | mixed BF16+NVFP4 transformer override, correctness validation, 4x RTX 5090 benchmark, torch-profiler trace | use `build_modelopt_nvfp4_transformer.py`; validated builder keeps selected FLUX.1 modules in BF16 and sets `swap_weight_nibbles=false` |
+| `NVFP4` | `black-forest-labs/FLUX.2-dev` | `--transformer-weights-path` | `black-forest-labs/FLUX.2-dev-NVFP4` | packed-QKV load path | official raw export repo; validated packed export detection and runtime layout handling |
+| `NVFP4` | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | `--transformer-path` | `lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer` | primary `transformer` quantized with ModelOpt NVFP4, `transformer_2` kept BF16 | primary-transformer-only path; keep `transformer_2` on the base checkpoint; the default FP4 GEMM backend is `flashinfer_trtllm` |
+
+These nine checkpoints are also the intended case set for the B200 diffusion
+CI job (`multimodal-gen-test-1-b200`).
+
+## ModelOpt FP8
+
+### Usage Examples
+
+Converted ModelOpt FP8 checkpoints should be loaded as transformer component
+overrides. If the repo or local directory already contains `config.json`, use
+`--transformer-path`.
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --transformer-path lmsys/flux2-dev-modelopt-fp8-sglang-transformer \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --save-output
+```
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
+  --transformer-path lmsys/wan22-t2v-a14b-modelopt-fp8-sglang-transformer \
+  --prompt "a fox walking through neon rain" \
+  --save-output
+```
+
+```bash
+sglang generate \
+  --model-path hunyuanvideo-community/HunyuanVideo \
+  --transformer-path lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer \
+  --height 544 --width 960 --num-frames 17 \
+  --prompt "A cinematic shot of a red sports car driving through rain at night" \
+  --save-output
+```
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --transformer-path lmsys/qwen-image-modelopt-fp8-sglang-transformer \
+  --prompt "A tiny astronaut reading a book under a glass greenhouse" \
+  --save-output
+```
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image-Edit-2511 \
+  --transformer-path lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer \
+  --image-path /path/to/input.png \
+  --prompt "Turn the scene into a warm watercolor illustration" \
+  --save-output
+```
+
+### Notes
+
+- `--transformer-path` is the canonical flag for converted ModelOpt FP8
+  transformer component repos or directories that already carry `config.json`.
+- If the override repo or local directory contains its own `config.json`,
+  SGLang reads the quantization config from that override instead of relying on
+  the base model config.
+- `--transformer-weights-path` still works when you intentionally point at raw
+  weight files or a directory that should be metadata-probed as weights first.
+- `dit_layerwise_offload` is supported for ModelOpt FP8 checkpoints.
+- `dit_cpu_offload` still stays disabled for ModelOpt FP8 checkpoints.
+- The layerwise offload path now preserves the non-contiguous FP8 weight stride
+  expected by the runtime FP8 GEMM path.
+- On disk, the quantization config stays `quant_method=modelopt` with
+  `quant_algo=FP8`; the `modelopt-fp8` label in this document is a support
+  family name, not a serialized config key.
+- `hunyuanvideo-community/HunyuanVideo` uses the `hunyuan-video` converter
+  preset. Use `--model-type hunyuan-video` to force it, or rely on
+  auto-detection from `_class_name=HunyuanVideoTransformer3DModel`.
+- The validated HunyuanVideo FP8 fallback preset keeps `context_embedder`,
+  `x_embedder.proj`, timestep/guidance/text embedder linear layers,
+  `norm_out.linear`, `proj_out`, double-block modulation linear layers, and
+  single-block modulation linear layers in BF16.
+- HunyuanVideo ModelOpt exports use diffusers module names that do not match
+  SGLang runtime module names for fused QKV and fused QKV+MLP layers. The
+  converter maps the names before selecting scale tensors and before writing
+  the runtime ignore list.
+- `Qwen/Qwen-Image` and `Qwen/Qwen-Image-Edit-2511` share the `qwen-image`
+  converter preset. Use `--model-type qwen-image` to force it, or rely on
+  auto-detection from `_class_name=QwenImageTransformer2DModel`.
+- The validated Qwen Image FP8 fallback preset keeps `img_in`, `txt_in`,
+  timestep embedder linear layers, `norm_out.linear`, `proj_out`,
+  `transformer_blocks.*.(img_mod|txt_mod)`, and
+  `transformer_blocks.*.img_mlp.net.2` in BF16.
+- For Qwen Image FP8 conversion, write explicit BF16 fallback tensors before
+  honoring ModelOpt ignored weights. Otherwise converter stats can report a
+  fallback while the output checkpoint still retains the source FP8 tensor.
+- To build the converted checkpoint yourself from a ModelOpt diffusers export,
+  use `python -m sglang.multimodal_gen.tools.build_modelopt_fp8_transformer`.
+
+## ModelOpt NVFP4
+
+### Usage Examples
+
+For mixed ModelOpt NVFP4 transformer overrides that already contain
+`config.json`, keep the base model and quantized transformer separate and use
+`--transformer-path`:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.1-dev \
+  --transformer-path lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --save-output
+```
+
+For raw NVFP4 exports such as the official FLUX.2 release, use
+`--transformer-weights-path`:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev \
+  --transformer-weights-path black-forest-labs/FLUX.2-dev-NVFP4 \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --save-output
+```
+
+SGLang also supports passing the NVFP4 repo or local directory directly as
+`--model-path`:
+
+```bash
+sglang generate \
+  --model-path black-forest-labs/FLUX.2-dev-NVFP4 \
+  --prompt "A Logo With Bold Large Text: SGL Diffusion" \
+  --save-output
+```
+
+For a dual-transformer Wan2.2 export where only the primary `transformer`
+was quantized:
+
+```bash
+sglang generate \
+  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
+  --transformer-path lmsys/wan22-t2v-a14b-modelopt-nvfp4-sglang-transformer \
+  --prompt "a fox walking through neon rain" \
+  --save-output
+```
+
+### Notes
+
+- Use `--transformer-path` for mixed ModelOpt NVFP4 transformer repos or local
+  directories that already include `config.json`.
+- Use `--transformer-weights-path` for raw NVFP4 exports, individual
+  safetensors files, or repo layouts that should be treated as weights first.
+- For dual-transformer pipelines such as `Wan2.2-T2V-A14B-Diffusers`, the
+  primary `--transformer-path` override targets only `transformer`. Use a
+  per-component override such as `--transformer-2-path` only when you
+  intentionally want a non-default `transformer_2`.
+- On Blackwell, the diffusion ModelOpt NVFP4 path defaults to FlashInfer
+  TensorRT-LLM FP4 GEMM (`flashinfer_trtllm`).
+- Direct `--model-path` loading is a compatibility path for FLUX.2 NVFP4-style
+  repos or local directories.
+- If `--transformer-weights-path` is provided explicitly, it takes precedence
+  over the compatibility `--model-path` flow.
+- For local directories, SGLang first looks for `*-mixed.safetensors`, then
+  falls back to loading from the directory.
+- To force the diffusion ModelOpt FP4 path onto a different FlashInfer
+  backend, set `SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND`. Supported values
+  include `flashinfer_cudnn`, `flashinfer_cutlass`, and `flashinfer_trtllm`.
+- On disk, the quantization config stays `quant_method=modelopt` with
+  `quant_algo=NVFP4`; the `modelopt-nvfp4` label here is again a documentation
+  family name rather than a serialized config key.
+
+## Nunchaku (SVDQuant)
+
+### Install
+
+Install the runtime dependency first:
+
+```bash
+pip install nunchaku
+```
+
+For platform-specific installation methods and troubleshooting, see the
+[Nunchaku installation guide](https://nunchaku.tech/docs/nunchaku/installation/installation.html).
+
+### File Naming and Auto-Detection
+
+For Nunchaku checkpoints, `--model-path` should still point to the original
+base model, while `--transformer-weights-path` points to the quantized
+transformer weights.
+
+If the basename of `--transformer-weights-path` contains the pattern
+`svdq-(int4|fp4)_r{rank}`, SGLang will automatically:
+- enable SVDQuant
+- infer `--quantization-precision`
+- infer `--quantization-rank`
+
+Examples:
+
+| checkpoint name fragment | inferred precision | inferred rank | notes |
+|--------------------------|--------------------|---------------|-------|
+| `svdq-int4_r32`          | `int4`             | `32`          | Standard INT4 checkpoint |
+| `svdq-int4_r128`         | `int4`             | `128`         | Higher-quality INT4 checkpoint |
+| `svdq-fp4_r32`           | `nvfp4`            | `32`          | `fp4` in the filename maps to CLI value `nvfp4` |
+| `svdq-fp4_r128`          | `nvfp4`            | `128`         | Higher-quality NVFP4 checkpoint |
+
+Common filenames:
+
+| filename | precision | rank | typical use |
+|----------|-----------|------|-------------|
+| `svdq-int4_r32-qwen-image.safetensors` | `int4` | `32` | Balanced default |
+| `svdq-int4_r128-qwen-image.safetensors` | `int4` | `128` | Quality-focused |
+| `svdq-fp4_r32-qwen-image.safetensors` | `nvfp4` | `32` | RTX 50-series / NVFP4 path |
+| `svdq-fp4_r128-qwen-image.safetensors` | `nvfp4` | `128` | Quality-focused NVFP4 |
+| `svdq-int4_r32-qwen-image-lightningv1.0-4steps.safetensors` | `int4` | `32` | Lightning 4-step |
+| `svdq-int4_r128-qwen-image-lightningv1.1-8steps.safetensors` | `int4` | `128` | Lightning 8-step |
+
+If your checkpoint name does not follow this convention, pass
+`--enable-svdquant`, `--quantization-precision`, and `--quantization-rank`
+explicitly.
+
+### Usage Examples
+
+Recommended auto-detected flow:
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --transformer-weights-path /path/to/svdq-int4_r32-qwen-image.safetensors \
+  --prompt "a beautiful sunset" \
+  --save-output
+```
+
+Manual override when the filename does not encode the quant settings:
+
+```bash
+sglang generate \
+  --model-path Qwen/Qwen-Image \
+  --transformer-weights-path /path/to/custom_nunchaku_checkpoint.safetensors \
+  --enable-svdquant \
+  --quantization-precision int4 \
+  --quantization-rank 128 \
+  --prompt "a beautiful sunset" \
+  --save-output
+```
+
+### Notes
+
+- `--transformer-weights-path` is the canonical flag for Nunchaku checkpoints.
+  Older config names such as `quantized_model_path` are treated as
+  compatibility aliases.
+- Auto-detection only happens when the checkpoint basename matches
+  `svdq-(int4|fp4)_r{rank}`.
+- The CLI values are `int4` and `nvfp4`. In filenames, the NVFP4 variant is
+  written as `fp4`.
+- Lightning checkpoints usually expect matching `--num-inference-steps`, such
+  as `4` or `8`.
+- Current runtime validation only allows Nunchaku on NVIDIA CUDA Ampere (SM8x)
+  or SM12x GPUs. Hopper (SM90) is currently rejected.
+
+## [ModelSlim](https://gitcode.com/Ascend/msmodelslim)
+MindStudio-ModelSlim (msModelSlim) is a model offline quantization compression tool launched by MindStudio and optimized for Ascend hardware.
+
+- **Installation**
+
+    ```bash
+    # Clone repo and install msmodelslim:
+    git clone https://gitcode.com/Ascend/msmodelslim.git
+    cd msmodelslim
+    bash install.sh
+    ```
+
+- **Multimodal_sd quantization**
+
+    Download the original floating-point weights of the large model. Taking Wan2.2-T2V-A14B as an example, you can go to [Wan2.2-T2V-A14B](https://modelscope.cn/models/Wan-AI/Wan2.2-T2V-A14B) to obtain the original model weights. Then install other dependencies (related to the model, refer to the modelscope model card).
+    > Note: You can find pre-quantized validated models on [modelscope/Eco-Tech](https://modelscope.cn/models/Eco-Tech).
+
+  Run quantization using one-click quantization (recommended):
+
+  ```bash
+  msmodelslim quant \
+    --model_path /path/to/wan2_2_float_weights \
+    --save_path /path/to/wan2_2_quantized_weights \
+    --device npu \
+    --model_type Wan2_2 \
+    --quant_type w8a8 \
+    --trust_remote_code True
+  ```
+
+  For more detailed examples of quantization of models, as well as information about their support, see the [examples](https://gitcode.com/Ascend/msmodelslim/blob/master/example/multimodal_sd/README.md) section in ModelSLim repo.
+
+  > Note: SGLang does not support quantized embeddings, please disable this option when quantizing using msmodelslim.
+
+- **Auto-Detection and different formats**
+
+    For msmodelslim checkpoints, it's enough to specify only ```--model-path```, the detection of quantization occurs automatically for each layer using parsing of      `quant_model_description.json` config.
+
+    In the case of `Wan2.2` only `Diffusers` weights storage format are supported, whereas modelslim saves the quantized model in the original `Wan2.2` format,
+    for conversion in use `python/sglang/multimodal_gen/tools/wan_repack.py` script:
+
+    ```bash
+    python wan_repack.py \
+      --input-path {path_to_quantized_model} \
+      --output-path {path_to_converted_model}
+    ```
+
+    After that, please copy all files from original `Diffusers` checkpoint (instead of `transformer`/`tranfsormer_2` folders)
+
+- **Usage Example**
+
+    With auto-detected flow:
+
+    ```bash
+    sglang generate \
+      --model-path Eco-Tech/Wan2.2-T2V-A14B-Diffusers-w8a8 \
+      --prompt "a beautiful sunset" \
+      --save-output
+    ```
+
+- **Available Quantization Methods**:
+    - [x]  ```W4A4_DYNAMIC``` linear with online quantization of activations
+    - [x]  ```W8A8``` linear with offline quantization of activations
+    - [x]  ```W8A8_DYNAMIC``` linear with online quantization of activations
+    - [x]  ```mxfp8``` linear with online/offline MXFP8 quantization (Ascend A5, CANN ≥ 8.0.RC3; see [Ascend NPU quantization](../platforms/ascend/ascend_npu_quantization.md#diffusion-model-quantization-on-ascend-npu))

docs/diffusion/reference.md

@@ -0,0 +1,11 @@
+# Reference
+
+Reference material for environment-based configuration and runtime behavior.
+
+- [Environment Variables](environment_variables.md): platform, caching, cloud storage, and debugging variables
+
+```{toctree}
+:maxdepth: 1
+
+environment_variables
+```

docs/diffusion/support_new_models.md

@@ -0,0 +1,388 @@
+# How to Support New Diffusion Models
+
+This document explains how to add support for new diffusion models in SGLang Diffusion.
+
+## Architecture Overview
+
+SGLang Diffusion is engineered for both performance and flexibility, built upon a pipeline architecture. This
+design allows developers to construct pipelines for various diffusion models while keeping the core generation
+loop standardized for optimization.
+
+At its core, the architecture revolves around two key concepts, as highlighted in our [blog post](https://lmsys.org/blog/2025-11-07-sglang-diffusion/#architecture):
+
+-   **`ComposedPipeline`**: This class orchestrates a series of `PipelineStage`s to define the complete generation process for a specific model. It acts as the main entry point for a model and manages the data flow between the different stages of the diffusion process.
+-   **`PipelineStage`**: Each stage is a modular component that encapsulates a function within the diffusion process. Examples include prompt encoding, the denoising loop, or VAE decoding.
+
+### Two Pipeline Styles
+
+SGLang Diffusion supports two pipeline composition styles. Both are valid; choose the one that best fits your model.
+
+#### Style A: Hybrid Monolithic Pipeline (Recommended Default)
+
+The recommended default for most new models. Uses a three-stage structure:
+
+```
+BeforeDenoisingStage (model-specific)  →  DenoisingStage (standard)  →  DecodingStage (standard)
+```
+
+| Stage | Ownership | Responsibility |
+|-------|-----------|----------------|
+| `{Model}BeforeDenoisingStage` | Model-specific | All pre-processing: input validation, text/image encoding, latent preparation, timestep computation |
+| `DenoisingStage` | Framework-standard | The denoising loop (DiT/UNet forward passes), shared across all models |
+| `DecodingStage` | Framework-standard | VAE decoding from latent space to pixel space, shared across all models |
+
+**Why recommended?** Modern diffusion models often have highly heterogeneous pre-processing requirements — different text encoders, different latent formats, different conditioning mechanisms. The Hybrid approach keeps pre-processing isolated per model, avoids fragile shared stages with excessive conditional logic, and lets developers port Diffusers reference code quickly.
+
+#### Style B: Modular Composition Style
+
+Uses the framework's fine-grained standard stages (`TextEncodingStage`, `LatentPreparationStage`, `TimestepPreparationStage`, etc.) to build the pipeline by composition. Convenience methods like `add_standard_t2i_stages()` and `add_standard_ti2i_stages()` make this very concise.
+
+This style is appropriate when:
+- **The new model's pre-processing can largely reuse existing stages** — e.g., a model that uses standard CLIP/T5 text encoding + standard latent preparation with minimal customization.
+- **A model-specific optimization needs to be extracted as a standalone stage** — e.g., a specialized encoding or conditioning step that benefits from being a separate stage for profiling, parallelism control, or reuse across multiple pipeline variants.
+
+#### How to Choose
+
+| Situation | Recommended Style |
+|-----------|-------------------|
+| Model has unique/complex pre-processing (VLM captioning, AR token generation, custom latent packing, etc.) | **Hybrid** — consolidate into a BeforeDenoisingStage |
+| Model fits neatly into standard text-to-image or text+image-to-image pattern | **Modular** — use `add_standard_t2i_stages()` / `add_standard_ti2i_stages()` |
+| Porting a Diffusers pipeline with many custom steps | **Hybrid** — copy the `__call__` logic into a single stage |
+| Adding a variant of an existing model that shares most logic | **Modular** — reuse existing stages, customize via PipelineConfig callbacks |
+| A specific pre-processing step needs special parallelism or profiling isolation | **Modular** — extract that step as a dedicated stage |
+
+## Key Components for Implementation
+
+To add support for a new diffusion model, you will need to define or configure the following components:
+
+1.  **`PipelineConfig`**: A dataclass holding static configurations for your model pipeline — precision settings, model architecture parameters, and callback methods used by the standard `DenoisingStage` and `DecodingStage`. Each model has its own subclass.
+
+2.  **`SamplingParams`**: A dataclass defining runtime generation parameters — `prompt`, `negative_prompt`, `guidance_scale`, `num_inference_steps`, `seed`, `height`, `width`, etc.
+
+3.  **Pre-processing stage(s)**: Either a single model-specific `{Model}BeforeDenoisingStage` (Hybrid style) or a combination of standard stages (Modular style). See [Two Pipeline Styles](#two-pipeline-styles) above.
+
+4.  **`ComposedPipeline`**: A class that wires together your pre-processing stage(s) with the standard `DenoisingStage` and `DecodingStage`. See base definitions:
+    - [`ComposedPipelineBase`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/pipelines_core/composed_pipeline_base.py)
+    - [`PipelineStage`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/pipelines_core/stages/base.py)
+    - [Central registry](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/registry.py)
+
+5.  **Modules (model components)**: Each pipeline references modules loaded from the model repository (e.g., Diffusers `model_index.json`):
+    - `text_encoder`: Encodes text prompts into embeddings.
+    - `tokenizer`: Tokenizes raw text input for the text encoder(s).
+    - `processor`: Preprocesses images and extracts features; often used in image-to-image tasks.
+    - `image_encoder`: Specialized image feature extractor.
+    - `dit/transformer`: The core denoising network (DiT/UNet architecture) operating in latent space.
+    - `scheduler`: Controls the timestep schedule and denoising dynamics.
+    - `vae`: Variational Autoencoder for encoding/decoding between pixel space and latent space.
+
+## Pipeline Stages Reference
+
+### Core Stages (used by all pipelines)
+
+| Stage Class                      | Description                                                                                             |
+| -------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| `DenoisingStage`                 | Executes the main denoising loop, iteratively applying the model (DiT/UNet) to refine the latents.      |
+| `DecodingStage`                  | Decodes the final latent tensor back into pixel space using the VAE.                                    |
+| `DmdDenoisingStage`              | A specialized denoising stage for DMD model architectures.                                              |
+| `CausalDMDDenoisingStage`        | A specialized causal denoising stage for specific video models.                                         |
+
+### Pre-processing Stages (for Modular Composition Style)
+
+The following fine-grained stages can be composed to build the pre-processing portion of a pipeline. They are best suited for models whose pre-processing largely fits the standard patterns. If your model requires significant customization, consider the Hybrid style with a single `BeforeDenoisingStage` instead.
+
+| Stage Class                      | Description                                                                                             |
+| -------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| `InputValidationStage`           | Validates user-provided `SamplingParams`.                                                               |
+| `TextEncodingStage`              | Encodes text prompts into embeddings using one or more text encoders.                                   |
+| `ImageEncodingStage`             | Encodes input images into embeddings, often used in image-to-image tasks.                               |
+| `ImageVAEEncodingStage`          | Encodes an input image into latent space using the VAE.                                                 |
+| `TimestepPreparationStage`       | Prepares the scheduler's timesteps for the diffusion process.                                           |
+| `LatentPreparationStage`         | Creates the initial noisy latent tensor that will be denoised.                                          |
+
+## Implementation Guide
+
+### Step 1: Obtain and Study the Reference Implementation
+
+Before writing any code, obtain the model's original implementation or Diffusers pipeline code:
+- The model's Diffusers pipeline source (e.g., the `pipeline_*.py` file from the `diffusers` library or HuggingFace repo)
+- Or the model's official reference implementation (e.g., from the model author's GitHub repo)
+- Or the HuggingFace model ID to look up `model_index.json` and the associated pipeline class
+
+Once you have the reference code, study it thoroughly:
+
+1. Find the model's `model_index.json` to identify required modules.
+2. Read the Diffusers pipeline's `__call__` method to understand:
+   - How text prompts are encoded
+   - How latents are prepared (shape, dtype, scaling)
+   - How timesteps/sigmas are computed
+   - What conditioning kwargs the DiT expects
+   - How the denoising loop works
+   - How VAE decoding is done
+
+### Step 2: Evaluate Reuse of Existing Pipelines and Stages
+
+Before creating any new files, check whether an existing pipeline or stage can be reused or extended. Only create new pipelines/stages when the existing ones would need substantial structural changes or when no architecturally similar implementation exists.
+
+- **Compare against existing pipelines** (Flux, Wan, Qwen-Image, GLM-Image, HunyuanVideo, LTX, etc.). If the new model shares most of its structure with an existing one, prefer adding a new config variant or reusing existing stages.
+- **Check existing stages** in `runtime/pipelines_core/stages/` and `stages/model_specific_stages/`.
+- **Check existing model components** — many models share VAEs (e.g., `AutoencoderKL`), text encoders (CLIP, T5), and schedulers. Reuse these directly.
+
+### Step 3: Implement Model Components
+
+Adapt the model's core components:
+
+- **DiT/Transformer**: Implement in [`runtime/models/dits/`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/models/dits/)
+- **Encoders**: Implement in [`runtime/models/encoders/`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/models/encoders/)
+- **VAEs**: Implement in [`runtime/models/vaes/`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/models/vaes/)
+- **Schedulers**: Implement in [`runtime/models/schedulers/`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/runtime/models/schedulers/) if needed
+
+Use SGLang's fused kernels where possible (see `LayerNormScaleShift`, `RMSNormScaleShift`, `apply_qk_norm`, etc.).
+
+**Tensor Parallel (TP) and Sequence Parallel (SP)**: For multi-GPU deployment, it is recommended to add TP/SP support to the DiT model. This can be done incrementally after the single-GPU implementation is verified. Reference implementations:
+- **Wan model** (`runtime/models/dits/wanvideo.py`) — Full TP + SP: `ColumnParallelLinear`/`RowParallelLinear` for attention, sequence dimension sharding via `get_sp_world_size()`
+- **Qwen-Image model** (`runtime/models/dits/qwen_image.py`) — SP via `USPAttention` (Ulysses + Ring Attention)
+
+### Step 4: Create Configs
+
+- **DiT Config**: `configs/models/dits/{model_name}.py`
+- **VAE Config**: `configs/models/vaes/{model_name}.py`
+- **SamplingParams**: `configs/sample/{model_name}.py`
+
+### Step 5: Create PipelineConfig
+
+The `PipelineConfig` provides callbacks that the standard `DenoisingStage` and `DecodingStage` use:
+
+```python
+# python/sglang/multimodal_gen/configs/pipeline_configs/my_model.py
+
+@dataclass
+class MyModelPipelineConfig(ImagePipelineConfig):
+    task_type: ModelTaskType = ModelTaskType.T2I
+    vae_precision: str = "bf16"
+    should_use_guidance: bool = True
+    dit_config: DiTConfig = field(default_factory=MyModelDitConfig)
+    vae_config: VAEConfig = field(default_factory=MyModelVAEConfig)
+
+    def get_freqs_cis(self, batch, device, rotary_emb, dtype):
+        """Prepare rotary position embeddings for the DiT."""
+        ...
+
+    def prepare_pos_cond_kwargs(self, batch, latent_model_input, t, **kwargs):
+        """Build positive conditioning kwargs for each denoising step."""
+        return {
+            "hidden_states": latent_model_input,
+            "encoder_hidden_states": batch.prompt_embeds[0],
+            "timestep": t,
+        }
+
+    def prepare_neg_cond_kwargs(self, batch, latent_model_input, t, **kwargs):
+        """Build negative conditioning kwargs for CFG."""
+        return {
+            "hidden_states": latent_model_input,
+            "encoder_hidden_states": batch.negative_prompt_embeds[0],
+            "timestep": t,
+        }
+
+    def get_decode_scale_and_shift(self):
+        """Return (scale, shift) for latent denormalization before VAE decode."""
+        ...
+```
+
+### Step 6: Implement Pre-processing
+
+Choose based on your model's needs (see [How to Choose](#how-to-choose)):
+
+#### Option A: BeforeDenoisingStage (Hybrid Style)
+
+Create a single stage that handles all pre-processing. Best when the model has custom/complex pre-processing logic.
+
+```python
+# python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/my_model.py
+
+class MyModelBeforeDenoisingStage(PipelineStage):
+    """Monolithic pre-processing stage for MyModel.
+
+    Consolidates: input validation, text/image encoding, latent
+    preparation, and timestep computation.
+    """
+
+    def __init__(self, vae, text_encoder, tokenizer, transformer, scheduler):
+        super().__init__()
+        self.vae = vae
+        self.text_encoder = text_encoder
+        self.tokenizer = tokenizer
+        self.transformer = transformer
+        self.scheduler = scheduler
+
+    @torch.no_grad()
+    def forward(self, batch: Req, server_args: ServerArgs) -> Req:
+        device = get_local_torch_device()
+
+        # 1. Encode prompt (model-specific logic)
+        prompt_embeds, negative_prompt_embeds = self._encode_prompt(...)
+
+        # 2. Prepare latents
+        latents = self._prepare_latents(...)
+
+        # 3. Prepare timesteps
+        timesteps, sigmas = self._prepare_timesteps(...)
+
+        # 4. Populate batch for DenoisingStage
+        batch.prompt_embeds = [prompt_embeds]
+        batch.negative_prompt_embeds = [negative_prompt_embeds]
+        batch.latents = latents
+        batch.timesteps = timesteps
+        batch.num_inference_steps = len(timesteps)
+        batch.sigmas = sigmas.tolist()
+        batch.generator = generator
+        batch.raw_latent_shape = latents.shape
+        return batch
+```
+
+#### Option B: Standard Stages (Modular Style)
+
+Skip creating a custom stage entirely — configure via `PipelineConfig` callbacks and use framework helpers. Best when the model fits standard patterns.
+
+(This option has no separate stage file; the pipeline class in Step 7 calls `add_standard_t2i_stages()` directly.)
+
+**Key batch fields that `DenoisingStage` expects** (regardless of which option you choose):
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `batch.latents` | `torch.Tensor` | Initial noisy latent tensor |
+| `batch.timesteps` | `torch.Tensor` | Timestep schedule |
+| `batch.num_inference_steps` | `int` | Number of denoising steps |
+| `batch.sigmas` | `list[float]` | Sigma schedule (must be a Python list, not numpy) |
+| `batch.prompt_embeds` | `list[torch.Tensor]` | Positive prompt embeddings (wrapped in a list) |
+| `batch.negative_prompt_embeds` | `list[torch.Tensor]` | Negative prompt embeddings (wrapped in a list) |
+| `batch.generator` | `torch.Generator` | RNG generator for reproducibility |
+| `batch.raw_latent_shape` | `tuple` | Original latent shape before any packing |
+
+### Step 7: Define the Pipeline Class
+
+#### Hybrid Style
+
+```python
+# python/sglang/multimodal_gen/runtime/pipelines/my_model.py
+
+class MyModelPipeline(LoRAPipeline, ComposedPipelineBase):
+    pipeline_name = "MyModelPipeline"  # Must match model_index.json _class_name
+
+    _required_config_modules = [
+        "text_encoder", "tokenizer", "vae", "transformer", "scheduler",
+    ]
+
+    def create_pipeline_stages(self, server_args: ServerArgs):
+        # 1. Monolithic pre-processing (model-specific)
+        self.add_stage(
+            MyModelBeforeDenoisingStage(
+                vae=self.get_module("vae"),
+                text_encoder=self.get_module("text_encoder"),
+                tokenizer=self.get_module("tokenizer"),
+                transformer=self.get_module("transformer"),
+                scheduler=self.get_module("scheduler"),
+            ),
+        )
+
+        # 2. Standard denoising loop (framework-provided)
+        self.add_stage(
+            DenoisingStage(
+                transformer=self.get_module("transformer"),
+                scheduler=self.get_module("scheduler"),
+            ),
+        )
+
+        # 3. Standard VAE decoding (framework-provided)
+        self.add_standard_decoding_stage()
+
+
+EntryClass = [MyModelPipeline]
+```
+
+#### Modular Style
+
+```python
+# python/sglang/multimodal_gen/runtime/pipelines/my_model.py
+
+class MyModelPipeline(LoRAPipeline, ComposedPipelineBase):
+    pipeline_name = "MyModelPipeline"
+
+    _required_config_modules = [
+        "text_encoder", "tokenizer", "vae", "transformer", "scheduler",
+    ]
+
+    def create_pipeline_stages(self, server_args: ServerArgs):
+        # All pre-processing + denoising + decoding in one call
+        self.add_standard_t2i_stages(
+            prepare_extra_timestep_kwargs=[prepare_mu],  # model-specific hooks
+        )
+
+
+EntryClass = [MyModelPipeline]
+```
+
+### Step 8: Register the Model
+
+Register your configs in [`registry.py`](https://github.com/sgl-project/sglang/blob/main/python/sglang/multimodal_gen/registry.py):
+
+```python
+register_configs(
+    model_family="my_model",
+    sampling_param_cls=MyModelSamplingParams,
+    pipeline_config_cls=MyModelPipelineConfig,
+    hf_model_paths=["org/my-model-name"],
+)
+```
+
+The `EntryClass` in your pipeline file is automatically discovered by the registry — no additional registration needed for the pipeline class itself.
+
+### Step 9: Verify Output Quality
+
+After implementation, verify that the generated output is not noise. A noisy or garbled output is the most common sign of an incorrect implementation. Common causes include:
+
+- Incorrect latent scale/shift factors
+- Wrong timestep/sigma schedule (order, dtype, or value range)
+- Mismatched conditioning kwargs
+- Rotary embedding style mismatch (`is_neox_style`)
+
+Debug by comparing intermediate tensor values against the Diffusers reference pipeline with the same seed.
+
+## Reference Implementations
+
+### Hybrid Style
+
+| Model | Pipeline | BeforeDenoisingStage | PipelineConfig |
+|-------|----------|---------------------|----------------|
+| GLM-Image | `runtime/pipelines/glm_image.py` | `stages/model_specific_stages/glm_image.py` | `configs/pipeline_configs/glm_image.py` |
+| Qwen-Image-Layered | `runtime/pipelines/qwen_image.py` | `stages/model_specific_stages/qwen_image_layered.py` | `configs/pipeline_configs/qwen_image.py` |
+
+### Modular Style
+
+| Model | Pipeline | Notes |
+|-------|----------|-------|
+| Qwen-Image (T2I) | `runtime/pipelines/qwen_image.py` | Uses `add_standard_t2i_stages()` |
+| Qwen-Image-Edit | `runtime/pipelines/qwen_image.py` | Uses `add_standard_ti2i_stages()` |
+| Flux | `runtime/pipelines/flux.py` | Uses `add_standard_t2i_stages()` with custom `prepare_mu` |
+| Wan | `runtime/pipelines/wan_pipeline.py` | Uses `add_standard_ti2v_stages()` |
+
+## Checklist
+
+Before submitting your implementation, verify:
+
+**Common (both styles):**
+- [ ] **Pipeline file** at `runtime/pipelines/{model_name}.py` with `EntryClass`
+- [ ] **PipelineConfig** at `configs/pipeline_configs/{model_name}.py`
+- [ ] **SamplingParams** at `configs/sample/{model_name}.py`
+- [ ] **DiT model** at `runtime/models/dits/{model_name}.py`
+- [ ] **Model configs** (DiT, VAE) at `configs/models/dits/` and `configs/models/vaes/`
+- [ ] **Registry entry** in `registry.py` via `register_configs()`
+- [ ] `pipeline_name` matches Diffusers `model_index.json` `_class_name`
+- [ ] `_required_config_modules` lists all modules from `model_index.json`
+- [ ] `PipelineConfig` callbacks (`prepare_pos_cond_kwargs`, etc.) match the DiT's `forward()` signature
+- [ ] Uses framework-standard `DenoisingStage` and `DecodingStage` (not custom denoising loops)
+- [ ] **TP/SP support** considered for DiT model (recommended; reference `wanvideo.py` for TP+SP, `qwen_image.py` for USPAttention)
+- [ ] **Output quality verified** — generated images/videos are not noise; compared against Diffusers reference output
+
+**Hybrid style only:**
+- [ ] **BeforeDenoisingStage** at `stages/model_specific_stages/{model_name}.py`
+- [ ] `BeforeDenoisingStage.forward()` populates all batch fields required by `DenoisingStage`

docs/diffusion/usage.md

@@ -0,0 +1,17 @@
+# Usage
+
+Use this section for day-to-day inference workflows with SGLang Diffusion.
+
+- [CLI](api/cli.md): run one-off jobs with `sglang generate` or start a server with `sglang serve`
+- [OpenAI-Compatible API](api/openai_api.md): request format, endpoints, and SDK examples
+- [Post-Processing](api/post_processing.md): frame interpolation and upscaling
+- [Quantization](quantization.md): quantized transformer checkpoints and supported quantization families
+
+```{toctree}
+:maxdepth: 1
+
+api/cli
+api/openai_api
+api/post_processing
+quantization
+```