chore: import upstream snapshot with attribution
PR Test (NPU) / check-changes (push) Has been cancelled
PR Test (NPU) / pr-gate (push) Has been cancelled
PR Test (NPU) / set-image-config (push) Has been cancelled
PR Test (NPU) / stage-b-test-1-npu-a2 (0) (push) Has been cancelled
PR Test (NPU) / stage-b-test-1-npu-a2 (1) (push) Has been cancelled
PR Test (NPU) / stage-b-test-2-npu-a2 (0) (push) Has been cancelled
PR Test (NPU) / stage-b-test-2-npu-a2 (1) (push) Has been cancelled
PR Test (NPU) / stage-b-test-4-npu-a3 (push) Has been cancelled
PR Test (NPU) / stage-b-test-16-npu-a3 (push) Has been cancelled
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (push) Has been cancelled
PR Test (NPU) / multimodal-gen-test-2-npu-a3 (push) Has been cancelled
PR Test (Arm64) / pr-gate (push) Has been cancelled
PR Test (Arm64) / check-changes (push) Has been cancelled
PR Test (Arm64) / build-test (push) Has been cancelled
PR Test (sgl-router) / gate (push) Has been cancelled
PR Test (sgl-router) / tier-1 — lint (push) Has been cancelled
PR Test (sgl-router) / tier-2 — build + test (push) Has been cancelled
PR Test (sgl-router) / tier-3 — docker (placeholder) (push) Has been cancelled
PR Test (sgl-router) / tier-3 — k8s integration (push) Has been cancelled
PR Test (sgl-router) / tier-3 — e2e (push) Has been cancelled
PR Test (sgl-router) / finish (push) Has been cancelled
PR Test (NPU) / single-node-poc (map[name:qwen3_6_27b_w8a8_1p_in64k_out1k_50ms runner:linux-aarch64-a3-2 test_case:test/registered/ascend/performance/qwen3_6_27b/test_npu_qwen3_6_27b_w8a8_1p_in64k_out1k_50ms.py test_type:perf]) (push) Has been cancelled
PR Test (NPU) / pr-test-npu-finish (push) Has been cancelled
PR Test (Xeon) / pr-gate (push) Has been cancelled
PR Test (Xeon) / check-changes (push) Has been cancelled
PR Test (Xeon) / build-test (, xeon-gnr, base-b-test-cpu) (push) Has been cancelled
PR Test (XPU) / check-changes (push) Has been cancelled
PR Test (XPU) / pr-gate (push) Has been cancelled
PR Test (XPU) / stage-a-test-1-gpu-xpu (push) Has been cancelled
PR Test (XPU) / wait-for-stage-a (push) Has been cancelled
PR Test (XPU) / stage-b-test-1-gpu-xpu (push) Has been cancelled
PR Test (XPU) / finish (push) Has been cancelled
CI Model Inventory / build-inventory (push) Has been cancelled
Lint / lint (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark Compilation Check (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark - Manual Policy (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark - Request Processing (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark Summary (push) Has been cancelled
PR Test (SMG) / build-wheel (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on windows (x86_64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on macos (x86_64 - auto) (push) Has been cancelled
PR Test (SMG) / python-unit-tests (push) Has been cancelled
PR Test (SMG) / unit-tests (push) Has been cancelled
PR Test (SMG) / benchmarks (push) Has been cancelled
PR Test (SMG) / chat-completions (push) Has been cancelled
PR Test (SMG) / chat-completions-4gpu (push) Has been cancelled
PR Test (SMG) / e2e (push) Has been cancelled
PR Test (SMG) / docker-build-test (push) Has been cancelled
PR Test (SMG) / k8s-integration (push) Has been cancelled
PR Test (SMG) / finish (push) Has been cancelled
PR Test (SMG) / summarize-benchmarks (push) Has been cancelled
Release SGLang Model Gateway Docker Image / publish (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on macos (aarch64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - musllinux_1_1) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - musllinux_1_1) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / Build SDist (push) Has been cancelled
Release SGLang Model Gateway to PyPI / Upload to PyPI (push) Has been cancelled
Release SGLang Kernels / build-cu129-matrix (aarch64, 12.9, 3.10, arm-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / build-cu129-matrix (x86_64, 12.9, 3.10, x64-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / release-cu129 (push) Has been cancelled
Release SGLang Kernels / build-cu130-matrix (aarch64, 13.0, 3.10, arm-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / build-cu130-matrix (x86_64, 13.0, 3.10, x64-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / release-cu130 (push) Has been cancelled
Release SGLang Kernels / build-rocm-matrix (3.10, 700) (push) Has been cancelled
Release SGLang Kernels / build-rocm-matrix (3.10, 720) (push) Has been cancelled
Release SGLang Kernels / release-rocm700 (push) Has been cancelled
Release SGLang Kernels / release-rocm720 (push) Has been cancelled
Release SGLang Kernels / build-musa43 (43, 3.10) (push) Has been cancelled
Release SGLang Kernels / release-musa43 (push) Has been cancelled
PR Test (NPU) / check-changes (push) Has been cancelled
PR Test (NPU) / pr-gate (push) Has been cancelled
PR Test (NPU) / set-image-config (push) Has been cancelled
PR Test (NPU) / stage-b-test-1-npu-a2 (0) (push) Has been cancelled
PR Test (NPU) / stage-b-test-1-npu-a2 (1) (push) Has been cancelled
PR Test (NPU) / stage-b-test-2-npu-a2 (0) (push) Has been cancelled
PR Test (NPU) / stage-b-test-2-npu-a2 (1) (push) Has been cancelled
PR Test (NPU) / stage-b-test-4-npu-a3 (push) Has been cancelled
PR Test (NPU) / stage-b-test-16-npu-a3 (push) Has been cancelled
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (push) Has been cancelled
PR Test (NPU) / multimodal-gen-test-2-npu-a3 (push) Has been cancelled
PR Test (Arm64) / pr-gate (push) Has been cancelled
PR Test (Arm64) / check-changes (push) Has been cancelled
PR Test (Arm64) / build-test (push) Has been cancelled
PR Test (sgl-router) / gate (push) Has been cancelled
PR Test (sgl-router) / tier-1 — lint (push) Has been cancelled
PR Test (sgl-router) / tier-2 — build + test (push) Has been cancelled
PR Test (sgl-router) / tier-3 — docker (placeholder) (push) Has been cancelled
PR Test (sgl-router) / tier-3 — k8s integration (push) Has been cancelled
PR Test (sgl-router) / tier-3 — e2e (push) Has been cancelled
PR Test (sgl-router) / finish (push) Has been cancelled
PR Test (NPU) / single-node-poc (map[name:qwen3_6_27b_w8a8_1p_in64k_out1k_50ms runner:linux-aarch64-a3-2 test_case:test/registered/ascend/performance/qwen3_6_27b/test_npu_qwen3_6_27b_w8a8_1p_in64k_out1k_50ms.py test_type:perf]) (push) Has been cancelled
PR Test (NPU) / pr-test-npu-finish (push) Has been cancelled
PR Test (Xeon) / pr-gate (push) Has been cancelled
PR Test (Xeon) / check-changes (push) Has been cancelled
PR Test (Xeon) / build-test (, xeon-gnr, base-b-test-cpu) (push) Has been cancelled
PR Test (XPU) / check-changes (push) Has been cancelled
PR Test (XPU) / pr-gate (push) Has been cancelled
PR Test (XPU) / stage-a-test-1-gpu-xpu (push) Has been cancelled
PR Test (XPU) / wait-for-stage-a (push) Has been cancelled
PR Test (XPU) / stage-b-test-1-gpu-xpu (push) Has been cancelled
PR Test (XPU) / finish (push) Has been cancelled
CI Model Inventory / build-inventory (push) Has been cancelled
Lint / lint (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark Compilation Check (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark - Manual Policy (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark - Request Processing (push) Has been cancelled
PR Benchmark (SMG Components) / Benchmark Summary (push) Has been cancelled
PR Test (SMG) / build-wheel (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on windows (x86_64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on macos (x86_64 - auto) (push) Has been cancelled
PR Test (SMG) / python-unit-tests (push) Has been cancelled
PR Test (SMG) / unit-tests (push) Has been cancelled
PR Test (SMG) / benchmarks (push) Has been cancelled
PR Test (SMG) / chat-completions (push) Has been cancelled
PR Test (SMG) / chat-completions-4gpu (push) Has been cancelled
PR Test (SMG) / e2e (push) Has been cancelled
PR Test (SMG) / docker-build-test (push) Has been cancelled
PR Test (SMG) / k8s-integration (push) Has been cancelled
PR Test (SMG) / finish (push) Has been cancelled
PR Test (SMG) / summarize-benchmarks (push) Has been cancelled
Release SGLang Model Gateway Docker Image / publish (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on macos (aarch64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - auto) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - musllinux_1_1) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - musllinux_1_1) (push) Has been cancelled
Release SGLang Model Gateway to PyPI / Build SDist (push) Has been cancelled
Release SGLang Model Gateway to PyPI / Upload to PyPI (push) Has been cancelled
Release SGLang Kernels / build-cu129-matrix (aarch64, 12.9, 3.10, arm-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / build-cu129-matrix (x86_64, 12.9, 3.10, x64-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / release-cu129 (push) Has been cancelled
Release SGLang Kernels / build-cu130-matrix (aarch64, 13.0, 3.10, arm-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / build-cu130-matrix (x86_64, 13.0, 3.10, x64-kernel-build-node) (push) Has been cancelled
Release SGLang Kernels / release-cu130 (push) Has been cancelled
Release SGLang Kernels / build-rocm-matrix (3.10, 700) (push) Has been cancelled
Release SGLang Kernels / build-rocm-matrix (3.10, 720) (push) Has been cancelled
Release SGLang Kernels / release-rocm700 (push) Has been cancelled
Release SGLang Kernels / release-rocm720 (push) Has been cancelled
Release SGLang Kernels / build-musa43 (43, 3.10) (push) Has been cancelled
Release SGLang Kernels / release-musa43 (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,17 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.diffusers_generic import (
|
||||
DiffusersGenericSamplingParams,
|
||||
)
|
||||
from sglang.multimodal_gen.configs.sample.ideogram import Ideogram4SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.pi05 import Pi05SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.vla import VLASamplingParams
|
||||
|
||||
__all__ = [
|
||||
"SamplingParams",
|
||||
"VLASamplingParams",
|
||||
"DiffusersGenericSamplingParams",
|
||||
"Ideogram4SamplingParams",
|
||||
"Pi05SamplingParams",
|
||||
]
|
||||
@@ -0,0 +1,80 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Cosmos3 sampling parameters.
|
||||
|
||||
A single ``SamplingParams`` class serves T2V, I2V, V2V, T2I, and
|
||||
action-conditioned variants. Per-request mode is dispatched in the pipeline
|
||||
from ``num_frames`` (``== 1`` → T2I), ``image_path`` (set → I2V),
|
||||
``video_path`` (set → V2V), and ``action_mode`` (set → action-conditioned).
|
||||
For ``num_frames == 1`` the output ``data_type`` flips to ``IMAGE``
|
||||
so the file extension and decode path agree.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
SamplingParams,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Cosmos3SamplingParams(SamplingParams):
|
||||
"""Cosmos3 sampling parameters (T2V defaults; also used for I2V / V2V / T2I)."""
|
||||
|
||||
height: int = 720
|
||||
width: int = 1280
|
||||
num_frames: int = 81
|
||||
fps: int = 24
|
||||
|
||||
guidance_scale: float = 4.0
|
||||
num_inference_steps: int = 35
|
||||
|
||||
negative_prompt: str = ""
|
||||
|
||||
# Optional CFG window — T2I requests typically pass e.g. ``(400, 1000)`` to
|
||||
# skip guidance at low noise levels. T2V / I2V / V2V leave it unset.
|
||||
guidance_interval: tuple[float, float] | None = None
|
||||
|
||||
# V2V conditioning: which latent-frame indices stay locked to the input
|
||||
# video. ``None`` resolves to ``[0]`` for I2V (single frame) and ``[0, 1]``
|
||||
# for V2V. ``condition_video_keep`` controls whether the first or last
|
||||
# source frames are used when the input video is longer than needed.
|
||||
condition_frame_indexes: list[int] | None = None
|
||||
condition_video_keep: str = "first"
|
||||
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 720),
|
||||
(720, 1280),
|
||||
(832, 480),
|
||||
(480, 832),
|
||||
(1024, 1024),
|
||||
]
|
||||
)
|
||||
|
||||
# Action modality (requires action_gen=True in the model checkpoint)
|
||||
# action_mode: "forward_dynamics" | "policy" | "inverse_dynamics"
|
||||
action_mode: str | None = None
|
||||
domain_id: int | None = None
|
||||
domain_name: str | None = None
|
||||
raw_action_dim: int | None = None
|
||||
action_fps: float | None = None
|
||||
# Action data for forward_dynamics: [T, D] nested list (API) or JSON string
|
||||
# (CLI via --action). Ignored by the other action modes.
|
||||
action: Any = None
|
||||
# Viewpoint phrasing for the structured action caption.
|
||||
action_view_point: str = "ego_view"
|
||||
# Optional dataset-derived action stats (JSON) for (de)normalization. When
|
||||
# set, input actions are normalized and predicted actions de-normalized
|
||||
# into physical units with ``action_normalization``.
|
||||
action_stats_path: str | None = None
|
||||
action_normalization: str = "quantile"
|
||||
|
||||
def _set_output_file_name(self) -> None:
|
||||
# The pipeline config's ``task_type=TI2V`` drives ``data_type`` to
|
||||
# VIDEO, but a single-frame request is a T2I and must pick the IMAGE
|
||||
# extension. Flip before the base derives the file name.
|
||||
if self.num_frames == 1:
|
||||
self.data_type = DataType.IMAGE
|
||||
super()._set_output_file_name()
|
||||
@@ -0,0 +1,50 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""
|
||||
Generic sampling parameters for diffusers backend.
|
||||
|
||||
This module provides generic sampling parameters that work with any diffusers pipeline.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
SamplingParams,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class DiffusersGenericSamplingParams(SamplingParams):
|
||||
"""
|
||||
Generic sampling parameters for diffusers backend.
|
||||
|
||||
These parameters cover the most common options across different diffusers pipelines.
|
||||
The diffusers pipeline will use whichever parameters it supports.
|
||||
|
||||
For pipeline-specific parameters, use `diffusers_kwargs` dict which will be
|
||||
passed directly to the diffusers pipeline call.
|
||||
"""
|
||||
|
||||
_default_height: ClassVar[int] = 1024
|
||||
_default_width: ClassVar[int] = 1024
|
||||
|
||||
# Override defaults with more conservative values that work across pipelines
|
||||
num_frames: int = 1 # default to image generation
|
||||
height: int = 1024
|
||||
width: int = 1024
|
||||
num_inference_steps: int = 30
|
||||
guidance_scale: float = 7.5
|
||||
negative_prompt: str = ""
|
||||
|
||||
# extra kwargs to pass directly to the diffusers pipeline
|
||||
# example: {"output_type": "latent", "return_dict": False}
|
||||
diffusers_kwargs: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.num_frames > 1:
|
||||
self.data_type = DataType.VIDEO
|
||||
else:
|
||||
self.data_type = DataType.IMAGE
|
||||
|
||||
super().__post_init__()
|
||||
@@ -0,0 +1,15 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Sampling parameters for ErnieImage."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class ErnieImageSamplingParams(SamplingParams):
|
||||
negative_prompt: str = " "
|
||||
num_frames: int = 1
|
||||
guidance_scale: float = 5.0
|
||||
num_inference_steps: int = 50
|
||||
use_pe: bool = True
|
||||
@@ -0,0 +1,39 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass
|
||||
from typing import ClassVar
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class FluxSamplingParams(SamplingParams):
|
||||
_default_height: ClassVar[int] = 128 * 8 # default_sample_size * vae_scale_factor
|
||||
_default_width: ClassVar[int] = 128 * 8
|
||||
|
||||
num_frames: int = 1
|
||||
# Denoising stage
|
||||
guidance_scale: float = 3.5
|
||||
negative_prompt: str = None
|
||||
num_inference_steps: int = 50
|
||||
|
||||
|
||||
@dataclass
|
||||
class Flux2SamplingParams(FluxSamplingParams):
|
||||
guidance_scale: float = 4.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class Flux2KleinSamplingParams(Flux2SamplingParams):
|
||||
# Klein is step-distilled, so default to 4 steps
|
||||
guidance_scale: float = 1.0
|
||||
num_inference_steps: int = 4
|
||||
|
||||
|
||||
@dataclass
|
||||
class Flux2KleinBaseSamplingParams(FluxSamplingParams):
|
||||
# Klein-base is undistilled
|
||||
num_inference_steps: int = 50
|
||||
guidance_scale: float = 4.0
|
||||
negative_prompt: str = ""
|
||||
@@ -0,0 +1,12 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class GlmImageSamplingParams(SamplingParams):
|
||||
negative_prompt = ""
|
||||
|
||||
num_frames: int = 1
|
||||
guidance_scale: float = 1.5
|
||||
num_inference_steps: int = 30
|
||||
@@ -0,0 +1,50 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class HeliosT2VSamplingParams(SamplingParams):
|
||||
# Video parameters
|
||||
height: int = 384
|
||||
width: int = 640
|
||||
num_frames: int = 99
|
||||
fps: int = 24
|
||||
|
||||
# Denoising stage
|
||||
guidance_scale: float = 5.0
|
||||
negative_prompt: str = (
|
||||
"Bright tones, overexposed, static, blurred details, subtitles, style, "
|
||||
"works, paintings, images, static, overall gray, worst quality, low quality, "
|
||||
"JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, "
|
||||
"poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, "
|
||||
"still picture, messy background, three legs, many people in the background, "
|
||||
"walking backwards"
|
||||
)
|
||||
num_inference_steps: int = 50
|
||||
|
||||
# Helios T2V supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(640, 384), # ~5:3
|
||||
(384, 640), # ~3:5
|
||||
(832, 480), # ~16:9-ish
|
||||
(480, 832), # ~9:16-ish
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class HeliosMidSamplingParams(HeliosT2VSamplingParams):
|
||||
"""Sampling params for Helios-Mid (Stage 2 pyramid SR)."""
|
||||
|
||||
num_inference_steps: int = 20
|
||||
|
||||
|
||||
@dataclass
|
||||
class HeliosDistilledSamplingParams(HeliosT2VSamplingParams):
|
||||
"""Sampling params for Helios-Distilled (DMD, no CFG needed)."""
|
||||
|
||||
guidance_scale: float = 1.0
|
||||
num_inference_steps: int = 10
|
||||
@@ -0,0 +1,56 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.teacache import TeaCacheParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class HunyuanSamplingParams(SamplingParams):
|
||||
num_inference_steps: int = 50
|
||||
|
||||
num_frames: int = 125
|
||||
height: int = 720
|
||||
width: int = 1280
|
||||
fps: int = 24
|
||||
|
||||
guidance_scale: float = 1.0
|
||||
|
||||
# HunyuanVideo supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
# 540p resolutions
|
||||
(960, 544), # 9:16
|
||||
(544, 960), # 16:9
|
||||
(832, 624), # 4:3
|
||||
(624, 832), # 3:4
|
||||
(720, 720), # 1:1
|
||||
# 720p resolutions (recommended)
|
||||
(1280, 720), # 9:16
|
||||
(720, 1280), # 16:9
|
||||
(832, 1104), # 4:3
|
||||
(1104, 832), # 3:4
|
||||
(960, 960), # 1:1
|
||||
]
|
||||
)
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.15,
|
||||
# from https://github.com/ali-vilab/TeaCache/blob/7c10efc4702c6b619f47805f7abe4a7a08085aa0/TeaCache4HunyuanVideo/teacache_sample_video.py#L222
|
||||
coefficients=[
|
||||
7.33226126e02,
|
||||
-4.01131952e02,
|
||||
6.75869174e01,
|
||||
-3.14987800e00,
|
||||
9.61237896e-02,
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class FastHunyuanSamplingParam(HunyuanSamplingParams):
|
||||
num_inference_steps: int = 6
|
||||
@@ -0,0 +1,29 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Sampling parameters for Hunyuan3D generation."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class Hunyuan3DSamplingParams(SamplingParams):
|
||||
"""Sampling parameters for Hunyuan3D image-to-mesh generation."""
|
||||
|
||||
negative_prompt: str = ""
|
||||
|
||||
shape_num_inference_steps: int = 50
|
||||
guidance_scale: float = 5.0
|
||||
|
||||
paint_num_inference_steps: int = 30
|
||||
paint_guidance_scale: float = 2.0
|
||||
|
||||
def __post_init__(self):
|
||||
if self.prompt is None:
|
||||
self.prompt = ""
|
||||
|
||||
if self.num_inference_steps is None:
|
||||
self.num_inference_steps = self.shape_num_inference_steps
|
||||
|
||||
self.guidance_scale = max(5.0, min(self.guidance_scale, 6.5))
|
||||
super().__post_init__()
|
||||
@@ -0,0 +1,78 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
SamplingParams,
|
||||
)
|
||||
|
||||
IDEOGRAM4_PRESETS: dict[str, dict[str, object]] = {
|
||||
"V4_QUALITY_48": {
|
||||
"num_steps": 48,
|
||||
"guidance_schedule": (3.0,) * 3 + (7.0,) * 45,
|
||||
"mu": 0.0,
|
||||
"std": 1.5,
|
||||
},
|
||||
"V4_DEFAULT_20": {
|
||||
"num_steps": 20,
|
||||
"guidance_schedule": (3.0,) * 2 + (7.0,) * 18,
|
||||
"mu": 0.0,
|
||||
"std": 1.75,
|
||||
},
|
||||
"V4_TURBO_12": {
|
||||
"num_steps": 12,
|
||||
"guidance_schedule": (3.0,) * 1 + (7.0,) * 11,
|
||||
"mu": 0.5,
|
||||
"std": 1.75,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Ideogram4SamplingParams(SamplingParams):
|
||||
data_type: DataType = DataType.IMAGE
|
||||
prompt: str = " "
|
||||
negative_prompt: str = " "
|
||||
height: int = 1024
|
||||
width: int = 1024
|
||||
num_frames: int = 1
|
||||
num_inference_steps: int | None = None
|
||||
guidance_scale: float | None = None
|
||||
preset: str = "V4_DEFAULT_20"
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.preset not in IDEOGRAM4_PRESETS:
|
||||
raise ValueError(
|
||||
f"Unknown Ideogram 4 preset {self.preset!r}; "
|
||||
f"expected one of {sorted(IDEOGRAM4_PRESETS)}"
|
||||
)
|
||||
preset_cfg = IDEOGRAM4_PRESETS[self.preset]
|
||||
preset_steps = int(preset_cfg["num_steps"])
|
||||
explicit_fields = getattr(self, "_explicit_fields", None)
|
||||
num_steps_is_explicit = (
|
||||
explicit_fields is None or "num_inference_steps" in explicit_fields
|
||||
)
|
||||
guidance_is_explicit = (
|
||||
explicit_fields is None or "guidance_scale" in explicit_fields
|
||||
)
|
||||
if (
|
||||
self.num_inference_steps is not None
|
||||
and self.num_inference_steps != preset_steps
|
||||
and num_steps_is_explicit
|
||||
):
|
||||
raise ValueError(
|
||||
"Ideogram 4 derives num_inference_steps from preset "
|
||||
f"{self.preset!r}; got {self.num_inference_steps}, expected "
|
||||
f"{preset_steps}."
|
||||
)
|
||||
if self.guidance_scale is not None and guidance_is_explicit:
|
||||
preset_guidance = float(preset_cfg["guidance_schedule"][-1])
|
||||
if self.guidance_scale != preset_guidance:
|
||||
raise ValueError(
|
||||
"Ideogram 4 derives guidance from the preset guidance_schedule; "
|
||||
"guidance_scale cannot be set directly."
|
||||
)
|
||||
self.num_inference_steps = preset_steps
|
||||
self.guidance_scale = float(preset_cfg["guidance_schedule"][-1])
|
||||
super().__post_init__()
|
||||
@@ -0,0 +1,34 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
import dataclasses
|
||||
from dataclasses import field
|
||||
|
||||
from sglang.multimodal_gen.configs.pipeline_configs.joy_echo import (
|
||||
JOY_ECHO_DEFAULT_SIGMAS,
|
||||
)
|
||||
from sglang.multimodal_gen.configs.sample.ltx_2 import LTX2SamplingParams
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class JoyEchoSamplingParams(LTX2SamplingParams):
|
||||
"""Sampling parameters for JoyEcho DMD inference."""
|
||||
|
||||
seed: int = 12345
|
||||
generator_device: str = "cuda"
|
||||
|
||||
height: int = 480
|
||||
width: int = 832
|
||||
num_frames: int = 121
|
||||
fps: int = 25
|
||||
|
||||
guidance_scale: float = 1.0
|
||||
num_inference_steps: int = 8
|
||||
|
||||
sigmas: tuple[float, ...] = field(default_factory=lambda: JOY_ECHO_DEFAULT_SIGMAS)
|
||||
|
||||
negative_prompt: str | None = None
|
||||
|
||||
video_cfg_scale: float = 1.0
|
||||
audio_cfg_scale: float = 1.0
|
||||
|
||||
enable_memory_bank: bool = True
|
||||
reset_memory_bank: bool = True
|
||||
@@ -0,0 +1,13 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class JoyImageEditSamplingParams(SamplingParams):
|
||||
"""Default sampling params for JoyImage Edit single-image I2I."""
|
||||
|
||||
negative_prompt: str = ""
|
||||
num_frames: int = 1
|
||||
guidance_scale: float = 4.0
|
||||
num_inference_steps: int = 40
|
||||
@@ -0,0 +1,29 @@
|
||||
# Krea-2 (K2) sampling defaults.
|
||||
#
|
||||
# `guidance_scale` is the SGLang classifier-free-guidance scale, which equals the
|
||||
# K2 reference `cfg + 1` (SGLang combines `uncond + scale*(cond-uncond)`, the K2
|
||||
# reference uses `cond + cfg*(cond-uncond)`). So K2 cfg=0 -> guidance_scale=1.0
|
||||
# (no CFG), K2 cfg=3.5 -> guidance_scale=4.5.
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class Krea2SamplingParams(SamplingParams):
|
||||
"""Distilled `oss_turbo` defaults: 8 steps, CFG disabled."""
|
||||
|
||||
negative_prompt: str = ""
|
||||
num_frames: int = 1
|
||||
height: int = 1024
|
||||
width: int = 1024
|
||||
guidance_scale: float = 1.0
|
||||
num_inference_steps: int = 8
|
||||
|
||||
|
||||
@dataclass
|
||||
class Krea2RawSamplingParams(Krea2SamplingParams):
|
||||
"""Base `oss_raw` defaults: full sampler with CFG."""
|
||||
|
||||
guidance_scale: float = 4.5
|
||||
num_inference_steps: int = 52
|
||||
@@ -0,0 +1,39 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
# Adapted from: https://github.com/Robbyant/lingbot-world
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.wan import Wan2_2_I2V_A14B_SamplingParam
|
||||
|
||||
|
||||
@dataclass
|
||||
class LingBotWorldSamplingParams(Wan2_2_I2V_A14B_SamplingParam):
|
||||
negative_prompt: str | None = None
|
||||
actions: list[list[str]] | None = None
|
||||
chunk_size: int | None = None
|
||||
guidance_scale: float = 5.0
|
||||
guidance_scale_2: float = 5.0
|
||||
num_inference_steps: int = 70
|
||||
num_frames: int = 117
|
||||
fps: int = 16
|
||||
|
||||
def _adjust(self, server_args):
|
||||
enable_sequence_shard = self.enable_sequence_shard
|
||||
if enable_sequence_shard is None or enable_sequence_shard:
|
||||
self.adjust_frames = False
|
||||
super()._adjust(server_args)
|
||||
if enable_sequence_shard is None or enable_sequence_shard:
|
||||
self.enable_sequence_shard = True
|
||||
self.adjust_frames = False
|
||||
if self.chunk_size is None:
|
||||
self.chunk_size = max(
|
||||
1,
|
||||
int(
|
||||
server_args.pipeline_config.dit_config.arch_config.num_frames_per_block
|
||||
),
|
||||
)
|
||||
if self.actions is not None:
|
||||
self.condition_inputs["camera_actions"] = self.actions
|
||||
if self.chunk_size is not None:
|
||||
self.realtime_chunk_size = self.chunk_size
|
||||
@@ -0,0 +1,126 @@
|
||||
import dataclasses
|
||||
from dataclasses import field
|
||||
from typing import Any
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class LTX2SamplingParams(SamplingParams):
|
||||
"""Sampling parameters for LTX-2."""
|
||||
|
||||
# Match the reference defaults used by ltx-pipelines (one-stage).
|
||||
# See: LTX-2/packages/ltx-pipelines/src/ltx_pipelines/utils/constants.py
|
||||
seed: int = 10
|
||||
generator_device: str = "cpu"
|
||||
|
||||
# Video parameters
|
||||
height: int = 512
|
||||
width: int = 768
|
||||
num_frames: int = 121
|
||||
fps: int = 24
|
||||
|
||||
# Audio specific
|
||||
generate_audio: bool = True
|
||||
|
||||
# Denoising parameters
|
||||
guidance_scale: float = 4.0
|
||||
num_inference_steps: int = 40
|
||||
|
||||
# Match ltx-pipelines default negative prompt (covers video + audio artifacts).
|
||||
negative_prompt: str = (
|
||||
"blurry, out of focus, overexposed, underexposed, low contrast, washed out colors, excessive noise, "
|
||||
"grainy texture, poor lighting, flickering, motion blur, distorted proportions, unnatural skin tones, "
|
||||
"deformed facial features, asymmetrical face, missing facial features, extra limbs, disfigured hands, "
|
||||
"wrong hand count, artifacts around text, inconsistent perspective, camera shake, incorrect depth of "
|
||||
"field, background too sharp, background clutter, distracting reflections, harsh shadows, inconsistent "
|
||||
"lighting direction, color banding, cartoonish rendering, 3D CGI look, unrealistic materials, uncanny "
|
||||
"valley effect, incorrect ethnicity, wrong gender, exaggerated expressions, wrong gaze direction, "
|
||||
"mismatched lip sync, silent or muted audio, distorted voice, robotic voice, echo, background noise, "
|
||||
"off-sync audio, incorrect dialogue, added dialogue, repetitive speech, jittery movement, awkward "
|
||||
"pauses, incorrect timing, unnatural transitions, inconsistent framing, tilted camera, flat lighting, "
|
||||
"inconsistent tone, cinematic oversaturation, stylized filters, or AI artifacts."
|
||||
)
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class LTX23SamplingParams(LTX2SamplingParams):
|
||||
"""Sampling parameters matching official LTX-2.3 one-stage defaults."""
|
||||
|
||||
seed: int = 42
|
||||
generator_device: str = "cuda"
|
||||
guidance_scale: float = 3.0
|
||||
num_inference_steps: int = 30
|
||||
|
||||
video_cfg_scale: float = 3.0
|
||||
video_stg_scale: float = 1.0
|
||||
video_rescale_scale: float = 0.7
|
||||
video_modality_scale: float = 3.0
|
||||
video_skip_step: int = 0
|
||||
video_stg_blocks: list[int] = field(default_factory=lambda: [28])
|
||||
|
||||
audio_cfg_scale: float = 7.0
|
||||
audio_stg_scale: float = 1.0
|
||||
audio_rescale_scale: float = 0.7
|
||||
audio_modality_scale: float = 3.0
|
||||
audio_skip_step: int = 0
|
||||
audio_stg_blocks: list[int] = field(default_factory=lambda: [28])
|
||||
skip_v2a_cross_attn_for_video_gt: bool = False
|
||||
|
||||
def build_request_extra(self) -> dict[str, Any]:
|
||||
extra = super().build_request_extra()
|
||||
# RL rollout uses the official CFG path (guidance_scale=1, no guider).
|
||||
if self.rollout:
|
||||
return extra
|
||||
extra["ltx2_stage1_guider_params"] = {
|
||||
"video_cfg_scale": self.video_cfg_scale,
|
||||
"video_stg_scale": self.video_stg_scale,
|
||||
"video_rescale_scale": self.video_rescale_scale,
|
||||
"video_modality_scale": self.video_modality_scale,
|
||||
"video_skip_step": self.video_skip_step,
|
||||
"video_stg_blocks": self.video_stg_blocks,
|
||||
"audio_cfg_scale": self.audio_cfg_scale,
|
||||
"audio_stg_scale": self.audio_stg_scale,
|
||||
"audio_rescale_scale": self.audio_rescale_scale,
|
||||
"audio_modality_scale": self.audio_modality_scale,
|
||||
"audio_skip_step": self.audio_skip_step,
|
||||
"audio_stg_blocks": self.audio_stg_blocks,
|
||||
}
|
||||
if self.skip_v2a_cross_attn_for_video_gt:
|
||||
extra["ltx2_skip_v2a_cross_attn_for_video_gt"] = True
|
||||
return extra
|
||||
|
||||
|
||||
@dataclasses.dataclass
|
||||
class LTX23HQSamplingParams(LTX23SamplingParams):
|
||||
"""Sampling parameters matching official LTX-2.3 HQ two-stage defaults."""
|
||||
|
||||
height: int = 1088
|
||||
width: int = 1920
|
||||
num_inference_steps: int = 15
|
||||
distilled_lora_strength_stage_1: float = 0.25
|
||||
distilled_lora_strength_stage_2: float = 0.5
|
||||
|
||||
video_cfg_scale: float = 3.0
|
||||
video_stg_scale: float = 0.0
|
||||
video_rescale_scale: float = 0.45
|
||||
video_modality_scale: float = 3.0
|
||||
video_skip_step: int = 0
|
||||
video_stg_blocks: list[int] = field(default_factory=list)
|
||||
|
||||
audio_cfg_scale: float = 7.0
|
||||
audio_stg_scale: float = 0.0
|
||||
audio_rescale_scale: float = 1.0
|
||||
audio_modality_scale: float = 3.0
|
||||
audio_skip_step: int = 0
|
||||
audio_stg_blocks: list[int] = field(default_factory=list)
|
||||
|
||||
def build_request_extra(self) -> dict[str, Any]:
|
||||
extra = super().build_request_extra()
|
||||
extra["ltx2_distilled_lora_strength_stage_1"] = float(
|
||||
self.distilled_lora_strength_stage_1
|
||||
)
|
||||
extra["ltx2_distilled_lora_strength_stage_2"] = float(
|
||||
self.distilled_lora_strength_stage_2
|
||||
)
|
||||
return extra
|
||||
@@ -0,0 +1,59 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class MOVASamplingParams(SamplingParams):
|
||||
# Video parameters (MOVA defaults)
|
||||
height: int = 352
|
||||
width: int = 640
|
||||
num_frames: int = 193
|
||||
fps: int = 24
|
||||
|
||||
# Denoising stage
|
||||
guidance_scale: float = 5.0
|
||||
num_inference_steps: int = 50
|
||||
sigma_shift: float = 5.0
|
||||
visual_shift: float = 5.0
|
||||
audio_shift: float = 5.0
|
||||
|
||||
adjust_frames: bool = False
|
||||
|
||||
negative_prompt: str = (
|
||||
"色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,"
|
||||
"整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,"
|
||||
"画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,"
|
||||
"静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走"
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MOVA_360P_SamplingParams(MOVASamplingParams):
|
||||
# Video parameters (MOVA 360P)
|
||||
height: int = 352
|
||||
width: int = 640
|
||||
|
||||
# MOVA 360P supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] = field(
|
||||
default_factory=lambda: [
|
||||
(352, 640),
|
||||
(640, 352),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MOVA_720P_SamplingParams(MOVASamplingParams):
|
||||
# Video parameters (MOVA 720P)
|
||||
height: int = 720
|
||||
width: int = 1280
|
||||
|
||||
# MOVA 720P supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] = field(
|
||||
default_factory=lambda: [
|
||||
(720, 1280),
|
||||
(1280, 720),
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,76 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.vla import VLASamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class Pi05SamplingParams(VLASamplingParams):
|
||||
"""Sampling parameters for Pi0.5 flow-matching action inference."""
|
||||
|
||||
num_inference_steps: int = 10
|
||||
|
||||
action_horizon: int = 50
|
||||
action_dim: int = 32
|
||||
output_format: str = "list"
|
||||
return_timing: bool = True
|
||||
enable_prefix_cache: bool = True
|
||||
enable_cuda_graph: bool = True
|
||||
|
||||
state: Any = field(default=None, metadata={"batch_sig_exclude": True})
|
||||
images: dict[str, Any] | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
image_masks: dict[str, bool] | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
camera_order: list[str] | tuple[str, ...] | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
noise: Any = field(default=None, metadata={"batch_sig_exclude": True})
|
||||
observation: dict[str, Any] | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
|
||||
def build_request_extra(self) -> dict[str, Any]:
|
||||
extra = super().build_request_extra()
|
||||
observation = dict(self.observation or {})
|
||||
if self.images is not None:
|
||||
observation["images"] = self.images
|
||||
if self.image_masks is not None:
|
||||
observation["image_masks"] = self.image_masks
|
||||
if self.state is not None:
|
||||
observation["state"] = self.state
|
||||
if self.camera_order is not None:
|
||||
observation["camera_order"] = tuple(self.camera_order)
|
||||
if self.prompt is not None:
|
||||
observation["prompt"] = self.prompt
|
||||
if self.noise is not None:
|
||||
observation["noise"] = self.noise
|
||||
|
||||
extra["vla"] = {
|
||||
"observation": observation,
|
||||
"options": {
|
||||
"output_format": self.output_format,
|
||||
"return_timing": self.return_timing,
|
||||
"enable_prefix_cache": self.enable_prefix_cache,
|
||||
"enable_cuda_graph": self.enable_cuda_graph,
|
||||
},
|
||||
}
|
||||
return extra
|
||||
|
||||
def _validate(self):
|
||||
super()._validate()
|
||||
if self.action_horizon <= 0:
|
||||
raise ValueError("action_horizon must be positive")
|
||||
if self.action_dim <= 0:
|
||||
raise ValueError("action_dim must be positive")
|
||||
if self.output_format not in ("list", "numpy"):
|
||||
raise ValueError("output_format must be 'list' or 'numpy'")
|
||||
|
||||
def _set_output_file_name(self):
|
||||
if self.output_file_name is None:
|
||||
self.output_file_name = "pi05_action"
|
||||
super()._set_output_file_name()
|
||||
@@ -0,0 +1,44 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class QwenImageSamplingParams(SamplingParams):
|
||||
negative_prompt: str = " "
|
||||
num_frames: int = 1
|
||||
# Denoising stage
|
||||
guidance_scale: float = 4.0
|
||||
num_inference_steps: int = 50
|
||||
|
||||
|
||||
@dataclass
|
||||
class QwenImage2512SamplingParams(QwenImageSamplingParams):
|
||||
negative_prompt: str = (
|
||||
"低分辨率,低画质,肢体畸形,手指畸形,画面过饱和,蜡像感,人脸无细节,过度光滑,画面具有AI感。构图混乱。文字模糊,扭曲。"
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class QwenImageEditPlusSamplingParams(QwenImageSamplingParams):
|
||||
# Denoising stage
|
||||
guidance_scale: float = 4.0
|
||||
# true_cfg_scale: float = 4.0
|
||||
num_inference_steps: int = 40
|
||||
|
||||
|
||||
@dataclass
|
||||
class QwenImageLayeredSamplingParams(QwenImageSamplingParams):
|
||||
# num_frames: int = 4
|
||||
height: int = 640
|
||||
width: int = 640
|
||||
prompt: str = " "
|
||||
negative_prompt: str = " "
|
||||
|
||||
guidance_scale: float = 4.0
|
||||
num_inference_steps: int = 50
|
||||
cfg_normalize: bool = True
|
||||
use_en_prompt: bool = True
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,29 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Sampling parameters for SANA image generation (T2I)."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
SamplingParams,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class SanaSamplingParams(SamplingParams):
|
||||
"""Defaults for SANA 1.5 1024px variant.
|
||||
|
||||
guidance_scale=4.5 enables standard classifier-free guidance.
|
||||
"""
|
||||
|
||||
data_type: DataType = DataType.IMAGE
|
||||
num_frames: int = 1
|
||||
guidance_scale: float = 4.5
|
||||
num_inference_steps: int = 20
|
||||
height: int = 1024
|
||||
width: int = 1024
|
||||
negative_prompt: str = (
|
||||
"low quality, low resolution, blurry, overexposed, underexposed, "
|
||||
"distorted, deformed, disfigured, bad anatomy, extra limbs, "
|
||||
"watermark, text, signature, ugly, noisy, artifacts"
|
||||
)
|
||||
@@ -0,0 +1,76 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Sampling parameters for SANA-WM TI2V world model generation."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Optional, Sequence, Union
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
SamplingParams,
|
||||
)
|
||||
|
||||
# Type alias for the camera tensor inputs. Accept torch.Tensor, numpy arrays,
|
||||
# or nested Python lists — coerced to torch.Tensor downstream in the stage.
|
||||
CameraTensorLike = Union[Any, Sequence[Sequence[Sequence[float]]]]
|
||||
|
||||
|
||||
@dataclass
|
||||
class SanaWMSamplingParams(SamplingParams):
|
||||
"""Default sampling parameters for SANA-WM 720p (704×1280) 16fps video.
|
||||
|
||||
Frame counts must satisfy (num_frames - 1) % 8 == 0.
|
||||
|
||||
Optional camera conditioning:
|
||||
camera_to_world: (T, 4, 4) extrinsics, one per output frame.
|
||||
intrinsics: (T, 3, 3) pinhole intrinsics, one per output frame.
|
||||
action: WASD/IJKL action DSL (e.g. "w-80,jw-40,w-40"), rolled
|
||||
out to camera_to_world before the camera branch.
|
||||
Omitted camera_to_world -> static identity camera. Omitted intrinsics ->
|
||||
centered heuristic; pass explicit intrinsics for closest NVlabs parity.
|
||||
"""
|
||||
|
||||
data_type: DataType = DataType.VIDEO
|
||||
|
||||
# Resolution: 720p landscape (LTX-2 VAE requires multiples of 32)
|
||||
height: int = 704
|
||||
width: int = 1280
|
||||
|
||||
# 49 = (49-1)/8 = 6 latent frames → ~3 seconds at 16fps
|
||||
num_frames: int = 49
|
||||
|
||||
# SANA-WM is trained at 16fps (override base default of 24).
|
||||
fps: int = 16
|
||||
|
||||
num_inference_steps: int = 20
|
||||
|
||||
guidance_scale: float = 4.5
|
||||
|
||||
# NVlabs' SANA-WM inference defaults to an empty negative prompt.
|
||||
negative_prompt: str = ""
|
||||
|
||||
# --- Camera trajectory (6-DoF) — optional ---
|
||||
camera_to_world: Optional[CameraTensorLike] = None
|
||||
intrinsics: Optional[CameraTensorLike] = None
|
||||
action: Optional[str] = None
|
||||
translation_speed: float = (
|
||||
0.04 # match official streaming (STREAMING_TRANSLATION_SPEED)
|
||||
)
|
||||
rotation_speed_deg: float = 1.2
|
||||
pitch_limit_deg: float = 85.0
|
||||
|
||||
def build_request_extra(self) -> dict[str, Any]:
|
||||
extra = super().build_request_extra()
|
||||
if self.action is not None and self.camera_to_world is not None:
|
||||
raise ValueError(
|
||||
"SANA-WM accepts either action or camera_to_world, not both."
|
||||
)
|
||||
if self.camera_to_world is not None:
|
||||
extra["camera_to_world"] = self.camera_to_world
|
||||
if self.intrinsics is not None:
|
||||
extra["intrinsics"] = self.intrinsics
|
||||
if self.action is not None:
|
||||
extra["action"] = self.action
|
||||
extra["translation_speed"] = self.translation_speed
|
||||
extra["rotation_speed_deg"] = self.rotation_speed_deg
|
||||
extra["pitch_limit_deg"] = self.pitch_limit_deg
|
||||
return extra
|
||||
@@ -0,0 +1,20 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""StableDiffusion3 sampling parameters configuration."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class StableDiffusion3SamplingParams(SamplingParams):
|
||||
"""Sampling parameters for StableDiffusion3."""
|
||||
|
||||
# A single space ensures tokenizers produce valid (non-empty) input for CFG.
|
||||
negative_prompt: str = " "
|
||||
num_frames: int = 1
|
||||
num_inference_steps: int = 50
|
||||
guidance_scale: float = 7.0
|
||||
guidance_rescale: float = 0.0
|
||||
@@ -0,0 +1,82 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Callable
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import CacheParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class TeaCacheParams(CacheParams):
|
||||
"""
|
||||
Parameters for [TeaCache](https://arxiv.org/abs/2411.14324).
|
||||
|
||||
Attributes:
|
||||
cache_type: (`str`, defaults to `teacache`):
|
||||
A string labeling these parameters as belonging to teacache.
|
||||
teacache_thresh (`float`, defaults to `0.0`):
|
||||
Threshold for accumulated relative L1 distance. When below this threshold, the
|
||||
forward pass is skipped. Recommended values: 0.25 for ~1.5x speedup, 0.4 for ~1.8x,
|
||||
0.6 for ~2.0x.
|
||||
start_skipping (`int` or `float`, defaults to `5`):
|
||||
The number of timesteps after which we may skip a forward pass. These early
|
||||
steps define the global structure and are too critical to not skip.
|
||||
int: The number of timesteps after which we can skip. If negative,
|
||||
this is an offset from the end of the schedule.
|
||||
float (0.0 - 1.0): A percentage of the total steps (e.g., 0.1
|
||||
computes the first 10%).
|
||||
end_skipping (`int` or `float`, defaults to `-1`):
|
||||
The number of timesteps after which we are no longer able to skip
|
||||
forward passes. The last steps refine fine textures and details.
|
||||
int: The number of timesteps after which skipping ends. If negative,
|
||||
this is an offset from the total number of steps.
|
||||
float (0.0 - 1.0): A percentage of the total steps (e.g., 0.1
|
||||
computes the first 10%).
|
||||
coefficients (`List[float]`, defaults to `[]`):
|
||||
Polynomial coefficients for rescaling the raw relative L1 distance,
|
||||
evaluated as `c[0]*x**4 + c[1]*x**3 + c[2]*x**2 + c[3]*x + c[4]`.
|
||||
coefficients_callback (`Callable[[TeaCacheParams], List[float]]`, *optional*):
|
||||
A function that receives this `TeaCacheParams` instance and returns
|
||||
the polynomial coefficients to use. When set, it takes precedence over
|
||||
the `coefficients` field, allowing dynamic coefficient selection based
|
||||
on any property of the params (e.g., `use_ret_steps` for Wan models).
|
||||
use_ret_steps: (`bool`, `None`, defaults to `None`):
|
||||
Used exclusively for wanvideo models to select different modulated inputs.
|
||||
"""
|
||||
|
||||
cache_type: str = "teacache"
|
||||
teacache_thresh: float = 0.0
|
||||
start_skipping: int | float = 5
|
||||
end_skipping: int | float = -1
|
||||
coefficients: list[float] = field(default_factory=list)
|
||||
coefficients_callback: Callable[[TeaCacheParams], list[float]] | None = field(
|
||||
default=None, repr=False
|
||||
)
|
||||
use_ret_steps: bool | None = None
|
||||
|
||||
def get_coefficients(self) -> list[float]:
|
||||
if self.coefficients_callback is not None:
|
||||
return self.coefficients_callback(self)
|
||||
return self.coefficients
|
||||
|
||||
def get_skip_boundaries(
|
||||
self, num_inference_steps: int, do_cfg: bool
|
||||
) -> tuple[int, int]:
|
||||
def _resolve_boundary(value: int | float) -> int:
|
||||
if isinstance(value, float):
|
||||
return int(num_inference_steps * value)
|
||||
if value < 0:
|
||||
return num_inference_steps + value
|
||||
return value
|
||||
|
||||
start_skipping = _resolve_boundary(self.start_skipping)
|
||||
end_skipping = _resolve_boundary(self.end_skipping)
|
||||
|
||||
if do_cfg:
|
||||
start_skipping *= 2
|
||||
end_skipping *= 2
|
||||
|
||||
return start_skipping, end_skipping
|
||||
@@ -0,0 +1,272 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import argparse
|
||||
import dataclasses
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import (
|
||||
DataType,
|
||||
_sanitize_filename,
|
||||
)
|
||||
from sglang.multimodal_gen.utils import StoreBoolean, expand_path_fields
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from sglang.multimodal_gen.runtime.server_args import ServerArgs
|
||||
|
||||
|
||||
@dataclass
|
||||
class VLASamplingParams:
|
||||
"""Sampling parameters for VLA/action-generation policies."""
|
||||
|
||||
data_type: DataType = DataType.ACTION
|
||||
request_id: str | None = field(default=None, metadata={"batch_sig_exclude": True})
|
||||
prompt: str | list[str] | None = field(
|
||||
default="", metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
num_outputs_per_prompt: int = 1
|
||||
seed: int | list[int] = field(default=42, metadata={"batch_sig_exclude": True})
|
||||
generator_device: str | None = None
|
||||
num_inference_steps: int = 10
|
||||
|
||||
output_path: str | None = field(default=None, metadata={"batch_sig_exclude": True})
|
||||
output_file_name: str | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
save_output: bool = False
|
||||
return_file_paths_only: bool = False
|
||||
|
||||
profile: bool = field(default=False, metadata={"batch_sig_exclude": True})
|
||||
num_profiled_timesteps: int = field(default=5, metadata={"batch_sig_exclude": True})
|
||||
profile_all_stages: bool = field(
|
||||
default=False, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
debug: bool = field(default=False, metadata={"batch_sig_exclude": True})
|
||||
perf_dump_path: str | None = field(
|
||||
default=None, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
suppress_logs: bool = field(default=False, metadata={"batch_sig_exclude": True})
|
||||
|
||||
enable_sequence_shard: bool | None = None
|
||||
max_sequence_length: int | None = None
|
||||
no_override_protected_fields: bool = field(
|
||||
default=False, metadata={"batch_sig_exclude": True}
|
||||
)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
self.data_type = DataType.ACTION
|
||||
self._validate()
|
||||
|
||||
env_steps = os.environ.get("SGLANG_TEST_NUM_INFERENCE_STEPS")
|
||||
if env_steps is not None and self.num_inference_steps is not None:
|
||||
self.num_inference_steps = int(env_steps)
|
||||
|
||||
def build_request_extra(self) -> dict[str, Any]:
|
||||
extra = {}
|
||||
diffusers_kwargs = getattr(self, "diffusers_kwargs", None)
|
||||
if diffusers_kwargs:
|
||||
extra["diffusers_kwargs"] = diffusers_kwargs
|
||||
explicit_fields = getattr(self, "_explicit_fields", None)
|
||||
if explicit_fields is not None:
|
||||
extra["explicit_fields"] = sorted(explicit_fields)
|
||||
return extra
|
||||
|
||||
def apply_request_extra(self, req: Any) -> None:
|
||||
req.extra.update(self.build_request_extra())
|
||||
|
||||
def _validate(self):
|
||||
if (
|
||||
not isinstance(self.num_outputs_per_prompt, int)
|
||||
or self.num_outputs_per_prompt <= 0
|
||||
):
|
||||
raise ValueError(
|
||||
"num_outputs_per_prompt must be a positive int, "
|
||||
f"got {self.num_outputs_per_prompt!r}"
|
||||
)
|
||||
|
||||
if isinstance(self.seed, list):
|
||||
if not self.seed:
|
||||
raise ValueError("seed list must not be empty")
|
||||
for seed in self.seed:
|
||||
if isinstance(seed, bool) or not isinstance(seed, int) or seed < 0:
|
||||
raise ValueError(
|
||||
f"seed list must contain non-negative ints, got {self.seed!r}"
|
||||
)
|
||||
elif (
|
||||
isinstance(self.seed, bool)
|
||||
or not isinstance(self.seed, int)
|
||||
or self.seed < 0
|
||||
):
|
||||
raise ValueError(
|
||||
f"seed must be a non-negative int or list of ints, got {self.seed!r}"
|
||||
)
|
||||
|
||||
if (
|
||||
not isinstance(self.num_inference_steps, int)
|
||||
or self.num_inference_steps <= 0
|
||||
):
|
||||
raise ValueError(
|
||||
"num_inference_steps must be a positive int, "
|
||||
f"got {self.num_inference_steps!r}"
|
||||
)
|
||||
|
||||
if self.generator_device not in (None, "cuda", "musa", "cpu"):
|
||||
raise ValueError(
|
||||
"generator_device must be one of None, 'cuda', 'musa', or 'cpu', "
|
||||
f"got {self.generator_device!r}"
|
||||
)
|
||||
|
||||
def _validate_with_pipeline_config(self, pipeline_config):
|
||||
if not pipeline_config.task_type.is_action_gen():
|
||||
raise ValueError(
|
||||
f"VLASamplingParams requires an ACTION pipeline, got {pipeline_config.task_type.name}"
|
||||
)
|
||||
|
||||
def _adjust(self, server_args: "ServerArgs"):
|
||||
expand_path_fields(self)
|
||||
self.data_type = DataType.ACTION
|
||||
self.return_file_paths_only = False
|
||||
if self.output_path is None and server_args.output_path is not None:
|
||||
self.output_path = server_args.output_path
|
||||
if self.output_path is None:
|
||||
self.save_output = False
|
||||
if self.save_output and not server_args.comfyui_mode:
|
||||
self._set_output_file_name()
|
||||
|
||||
def _set_output_file_ext(self):
|
||||
if self.output_file_name and not self.output_file_name.endswith(".json"):
|
||||
self.output_file_name = f"{self.output_file_name}.json"
|
||||
|
||||
def _set_output_file_name(self):
|
||||
if self.output_file_name is None:
|
||||
self.output_file_name = "vla_action"
|
||||
self.output_file_name = _sanitize_filename(self.output_file_name)
|
||||
self._set_output_file_ext()
|
||||
|
||||
def output_file_path(self):
|
||||
if self.output_path is None or self.output_file_name is None:
|
||||
return None
|
||||
return os.path.join(self.output_path, self.output_file_name)
|
||||
|
||||
def _merge_with_user_params(
|
||||
self,
|
||||
user_params: "VLASamplingParams",
|
||||
explicit_fields: set[str] | None = None,
|
||||
):
|
||||
if user_params is None:
|
||||
return
|
||||
|
||||
predefined_fields = set(type(self).__annotations__.keys())
|
||||
allow_override_protected = not user_params.no_override_protected_fields
|
||||
for field_info in dataclasses.fields(user_params):
|
||||
field_name = field_info.name
|
||||
user_value = getattr(user_params, field_name)
|
||||
if field_info.default is not dataclasses.MISSING:
|
||||
default_class_value = field_info.default
|
||||
elif field_info.default_factory is not dataclasses.MISSING:
|
||||
default_class_value = field_info.default_factory()
|
||||
else:
|
||||
default_class_value = dataclasses.MISSING
|
||||
|
||||
if explicit_fields is not None:
|
||||
is_user_modified = field_name in explicit_fields
|
||||
else:
|
||||
is_user_modified = user_value != default_class_value
|
||||
is_protected_field = field_name in predefined_fields
|
||||
if is_user_modified and (
|
||||
allow_override_protected or not is_protected_field
|
||||
):
|
||||
setattr(self, field_name, user_value)
|
||||
|
||||
if explicit_fields is not None:
|
||||
self._explicit_fields = set(explicit_fields)
|
||||
self.__post_init__()
|
||||
|
||||
@staticmethod
|
||||
def add_cli_args(parser: Any) -> Any:
|
||||
def add_argument(*name_or_flags, **kwargs):
|
||||
kwargs.setdefault("default", argparse.SUPPRESS)
|
||||
return parser.add_argument(*name_or_flags, **kwargs)
|
||||
|
||||
add_argument(
|
||||
"--prompt",
|
||||
type=str,
|
||||
nargs="+",
|
||||
help="Language instruction(s) for the VLA policy.",
|
||||
)
|
||||
add_argument(
|
||||
"--num-inference-steps",
|
||||
type=int,
|
||||
help="Number of action denoising steps.",
|
||||
)
|
||||
add_argument(
|
||||
"--num-outputs-per-prompt",
|
||||
type=int,
|
||||
help="Number of candidate actions to generate per observation.",
|
||||
)
|
||||
add_argument(
|
||||
"--seed",
|
||||
type=int,
|
||||
nargs="+",
|
||||
help="Random seed for action noise generation.",
|
||||
)
|
||||
add_argument(
|
||||
"--generator-device",
|
||||
type=str,
|
||||
choices=["cuda", "musa", "cpu"],
|
||||
help="Device for random generator. Default: use the model-specific setting.",
|
||||
)
|
||||
add_argument(
|
||||
"--profile",
|
||||
action="store_true",
|
||||
help="Enable torch profiler for action denoising.",
|
||||
)
|
||||
add_argument(
|
||||
"--num-profiled-timesteps",
|
||||
type=int,
|
||||
help="Number of denoising timesteps to profile after warmup.",
|
||||
)
|
||||
add_argument(
|
||||
"--profile-all-stages",
|
||||
action="store_true",
|
||||
dest="profile_all_stages",
|
||||
help="Used with --profile, profile all pipeline stages.",
|
||||
)
|
||||
add_argument("--debug", action="store_true")
|
||||
add_argument(
|
||||
"--enable-sequence-shard",
|
||||
action=StoreBoolean,
|
||||
help="Enable sequence dimension shard with sequence parallelism.",
|
||||
)
|
||||
add_argument(
|
||||
"--max-sequence-length",
|
||||
type=int,
|
||||
help="Maximum prefix sequence length.",
|
||||
)
|
||||
add_argument(
|
||||
"--no-override-protected-fields",
|
||||
action="store_true",
|
||||
help="If set, disallow user params to override subclass-defined fields.",
|
||||
)
|
||||
return parser
|
||||
|
||||
@classmethod
|
||||
def get_cli_args(cls, args: argparse.Namespace):
|
||||
sampling_params_fields = {attr.name for attr in dataclasses.fields(cls)}
|
||||
args_attrs = set(vars(args).keys())
|
||||
attrs = sampling_params_fields & args_attrs
|
||||
cli_args = {
|
||||
attr: getattr(args, attr)
|
||||
for attr in attrs
|
||||
if hasattr(args, attr) and getattr(args, attr) is not None
|
||||
}
|
||||
if isinstance(cli_args.get("seed"), list) and len(cli_args["seed"]) == 1:
|
||||
cli_args["seed"] = cli_args["seed"][0]
|
||||
return cli_args
|
||||
|
||||
def output_size_str(self) -> str:
|
||||
return "action"
|
||||
|
||||
def seconds(self) -> float:
|
||||
return 0.0
|
||||
@@ -0,0 +1,287 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.teacache import TeaCacheParams
|
||||
|
||||
|
||||
def _wan_1_3b_coefficients(p: TeaCacheParams) -> list[float]:
|
||||
if p.use_ret_steps:
|
||||
# from https://github.com/ali-vilab/TeaCache/blob/7c10efc4702c6b619f47805f7abe4a7a08085aa0/TeaCache4Wan2.1/teacache_generate.py#L883
|
||||
return [
|
||||
-5.21862437e04,
|
||||
9.23041404e03,
|
||||
-5.28275948e02,
|
||||
1.36987616e01,
|
||||
-4.99875664e-02,
|
||||
]
|
||||
# from https://github.com/ali-vilab/TeaCache/blob/7c10efc4702c6b619f47805f7abe4a7a08085aa0/TeaCache4Wan2.1/teacache_generate.py#L890
|
||||
return [
|
||||
2.39676752e03,
|
||||
-1.31110545e03,
|
||||
2.01331979e02,
|
||||
-8.29855975e00,
|
||||
1.37887774e-01,
|
||||
]
|
||||
|
||||
|
||||
def _wan_14b_coefficients(p: TeaCacheParams) -> list[float]:
|
||||
if p.use_ret_steps:
|
||||
# from https://github.com/ali-vilab/TeaCache/blob/7c10efc4702c6b619f47805f7abe4a7a08085aa0/TeaCache4Wan2.1/teacache_generate.py#L885
|
||||
return [
|
||||
-3.03318725e05,
|
||||
4.90537029e04,
|
||||
-2.65530556e03,
|
||||
5.87365115e01,
|
||||
-3.15583525e-01,
|
||||
]
|
||||
# from https://github.com/ali-vilab/TeaCache/blob/7c10efc4702c6b619f47805f7abe4a7a08085aa0/TeaCache4Wan2.1/teacache_generate.py#L892
|
||||
return [-5784.54975374, 5449.50911966, -1811.16591783, 256.27178429, -13.02252404]
|
||||
|
||||
|
||||
@dataclass
|
||||
class WanT2V_1_3B_SamplingParams(SamplingParams):
|
||||
# Video parameters
|
||||
height: int = 480
|
||||
width: int = 832
|
||||
num_frames: int = 81
|
||||
fps: int = 16
|
||||
|
||||
# Denoising stage
|
||||
guidance_scale: float = 3.0
|
||||
negative_prompt: str = (
|
||||
"Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
|
||||
)
|
||||
num_inference_steps: int = 50
|
||||
|
||||
# Wan T2V 1.3B supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.08,
|
||||
use_ret_steps=True,
|
||||
coefficients_callback=_wan_1_3b_coefficients,
|
||||
start_skipping=5,
|
||||
end_skipping=1.0,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class WanT2V_14B_SamplingParams(SamplingParams):
|
||||
# Video parameters
|
||||
height: int = 720
|
||||
width: int = 1280
|
||||
num_frames: int = 81
|
||||
fps: int = 16
|
||||
|
||||
# Denoising stage
|
||||
guidance_scale: float = 5.0
|
||||
negative_prompt: str = (
|
||||
"Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
|
||||
)
|
||||
num_inference_steps: int = 50
|
||||
|
||||
# Wan T2V 14B supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 720), # 16:9
|
||||
(720, 1280), # 9:16
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.20,
|
||||
use_ret_steps=False,
|
||||
coefficients_callback=_wan_14b_coefficients,
|
||||
start_skipping=1,
|
||||
end_skipping=-1,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class WanI2V_14B_480P_SamplingParam(WanT2V_1_3B_SamplingParams):
|
||||
# Denoising stage
|
||||
guidance_scale: float = 5.0
|
||||
num_inference_steps: int = 50
|
||||
# num_inference_steps: int = 40
|
||||
|
||||
# Wan I2V 480P supported resolutions (override parent)
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.26,
|
||||
use_ret_steps=True,
|
||||
coefficients_callback=_wan_14b_coefficients,
|
||||
start_skipping=5,
|
||||
end_skipping=1.0,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class WanI2V_14B_720P_SamplingParam(WanT2V_14B_SamplingParams):
|
||||
# Denoising stage
|
||||
guidance_scale: float = 5.0
|
||||
num_inference_steps: int = 50
|
||||
# num_inference_steps: int = 40
|
||||
|
||||
# Wan I2V 720P supported resolutions (override parent)
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 720), # 16:9
|
||||
(720, 1280), # 9:16
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.3,
|
||||
use_ret_steps=True,
|
||||
coefficients_callback=_wan_14b_coefficients,
|
||||
start_skipping=5,
|
||||
end_skipping=1.0,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class FastWanT2V480PConfig(WanT2V_1_3B_SamplingParams):
|
||||
# DMD parameters
|
||||
# dmd_denoising_steps: list[int] | None = field(default_factory=lambda: [1000, 757, 522])
|
||||
num_inference_steps: int = 3
|
||||
num_frames: int = 61
|
||||
height: int = 480
|
||||
width: int = 832
|
||||
fps: int = 16
|
||||
|
||||
|
||||
# =============================================
|
||||
# ============= Wan2.1 Fun Models =============
|
||||
# =============================================
|
||||
@dataclass
|
||||
class Wan2_1_Fun_1_3B_InP_SamplingParams(SamplingParams):
|
||||
"""Sampling parameters for Wan2.1 Fun 1.3B InP model."""
|
||||
|
||||
height: int = 480
|
||||
width: int = 832
|
||||
num_frames: int = 81
|
||||
fps: int = 16
|
||||
negative_prompt: str | None = (
|
||||
"色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走"
|
||||
)
|
||||
guidance_scale: float = 6.0
|
||||
num_inference_steps: int = 50
|
||||
|
||||
|
||||
# =============================================
|
||||
# ============= Wan2.2 TI2V Models =============
|
||||
# =============================================
|
||||
@dataclass
|
||||
class Wan2_2_Base_SamplingParams(SamplingParams):
|
||||
"""Sampling parameters for Wan2.2 TI2V 5B model."""
|
||||
|
||||
negative_prompt: str | None = (
|
||||
"色调艳丽,过曝,静态,细节模糊不清,字幕,风格,作品,画作,画面,静止,整体发灰,最差质量,低质量,JPEG压缩残留,丑陋的,残缺的,多余的手指,画得不好的手部,画得不好的脸部,畸形的,毁容的,形态畸形的肢体,手指融合,静止不动的画面,杂乱的背景,三条腿,背景人很多,倒着走"
|
||||
)
|
||||
|
||||
# TODO(Wan2.2): TeaCache coefficients need to be calibrated for Wan2.2 by
|
||||
# profiling L1 distances across timesteps. Until then, teacache_params is None
|
||||
# and enable_teacache will be accepted but silently no-op.
|
||||
# Consider using Cache-DiT (SGLANG_CACHE_DIT_ENABLED=1) as an alternative.
|
||||
|
||||
|
||||
@dataclass
|
||||
class Wan2_2_TI2V_5B_SamplingParam(Wan2_2_Base_SamplingParams):
|
||||
"""Sampling parameters for Wan2.2 TI2V 5B model."""
|
||||
|
||||
height: int = 704
|
||||
width: int = 1280
|
||||
num_frames: int = 121
|
||||
fps: int = 24
|
||||
guidance_scale: float = 5.0
|
||||
num_inference_steps: int = 50
|
||||
|
||||
# Wan2.2 TI2V 5B supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 704), # 16:9-ish
|
||||
(704, 1280), # 9:16-ish
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Wan2_2_T2V_A14B_SamplingParam(Wan2_2_Base_SamplingParams):
|
||||
guidance_scale: float = 4.0 # high_noise
|
||||
guidance_scale_2: float = 3.0 # low_noise
|
||||
num_inference_steps: int = 40
|
||||
fps: int = 16
|
||||
|
||||
num_frames: int = 81
|
||||
|
||||
# Wan2.2 T2V A14B supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 720), # 16:9
|
||||
(720, 1280), # 9:16
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Wan2_2_I2V_A14B_SamplingParam(Wan2_2_Base_SamplingParams):
|
||||
guidance_scale: float = 3.5 # high_noise
|
||||
guidance_scale_2: float = 3.5 # low_noise
|
||||
num_inference_steps: int = 40
|
||||
fps: int = 16
|
||||
|
||||
num_frames: int = 81
|
||||
|
||||
# Wan2.2 I2V A14B supported resolutions
|
||||
supported_resolutions: list[tuple[int, int]] | None = field(
|
||||
default_factory=lambda: [
|
||||
(1280, 720), # 16:9
|
||||
(720, 1280), # 9:16
|
||||
(832, 480), # 16:9
|
||||
(480, 832), # 9:16
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Turbo_Wan2_2_I2V_A14B_SamplingParam(Wan2_2_Base_SamplingParams):
|
||||
guidance_scale: float = 3.5 # high_noise
|
||||
guidance_scale_2: float = 3.5 # low_noise
|
||||
num_inference_steps: int = 4
|
||||
fps: int = 16
|
||||
|
||||
|
||||
# =============================================
|
||||
# ============= Causal Self-Forcing =============
|
||||
# =============================================
|
||||
@dataclass
|
||||
class SelfForcingWanT2V480PConfig(WanT2V_1_3B_SamplingParams):
|
||||
pass
|
||||
@@ -0,0 +1,45 @@
|
||||
# Copied and adapted from: https://github.com/hao-ai-lab/FastVideo
|
||||
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from sglang.multimodal_gen.configs.sample.sampling_params import SamplingParams
|
||||
from sglang.multimodal_gen.configs.sample.teacache import TeaCacheParams
|
||||
|
||||
|
||||
@dataclass
|
||||
class ZImageTurboSamplingParams(SamplingParams):
|
||||
num_inference_steps: int = 9
|
||||
|
||||
num_frames: int = 1
|
||||
negative_prompt: str = None
|
||||
# Z-Image officially recommends starting at 1024x1024
|
||||
height: int = 1024
|
||||
width: int = 1024
|
||||
# fps: int = 24
|
||||
|
||||
guidance_scale: float = 0.0
|
||||
cfg_normalization: float | bool = False
|
||||
|
||||
teacache_params: TeaCacheParams = field(
|
||||
default_factory=lambda: TeaCacheParams(
|
||||
teacache_thresh=0.15,
|
||||
coefficients=[
|
||||
7.33226126e02,
|
||||
-4.01131952e02,
|
||||
6.75869174e01,
|
||||
-3.14987800e00,
|
||||
9.61237896e-02,
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ZImageSamplingParams(SamplingParams):
|
||||
num_inference_steps: int = 50
|
||||
|
||||
num_frames: int = 1
|
||||
negative_prompt: str = " "
|
||||
guidance_scale: float = 5.0
|
||||
cfg_normalization: float | bool = True
|
||||
Reference in New Issue
Block a user