wehub/sgl-project--sglang
1
0
Fork 0
Code Issues Pull Requests Actions 8 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
PR Test (NPU) / check-changes (push) Has been cancelled
Details
PR Test (NPU) / pr-gate (push) Has been cancelled
Details
PR Test (NPU) / set-image-config (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-1-npu-a2 (0) (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-1-npu-a2 (1) (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-2-npu-a2 (0) (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-2-npu-a2 (1) (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-4-npu-a3 (push) Has been cancelled
Details
PR Test (NPU) / stage-b-test-16-npu-a3 (push) Has been cancelled
Details
PR Test (NPU) / multimodal-gen-test-1-npu-a3 (push) Has been cancelled
Details
PR Test (NPU) / multimodal-gen-test-2-npu-a3 (push) Has been cancelled
Details
PR Test (Arm64) / pr-gate (push) Has been cancelled
Details
PR Test (Arm64) / check-changes (push) Has been cancelled
Details
PR Test (Arm64) / build-test (push) Has been cancelled
Details
PR Test (sgl-router) / gate (push) Has been cancelled
Details
PR Test (sgl-router) / tier-1 — lint (push) Has been cancelled
Details
PR Test (sgl-router) / tier-2 — build + test (push) Has been cancelled
Details
PR Test (sgl-router) / tier-3 — docker (placeholder) (push) Has been cancelled
Details
PR Test (sgl-router) / tier-3 — k8s integration (push) Has been cancelled
Details
PR Test (sgl-router) / tier-3 — e2e (push) Has been cancelled
Details
PR Test (sgl-router) / finish (push) Has been cancelled
Details
PR Test (NPU) / single-node-poc (map[name:qwen3_6_27b_w8a8_1p_in64k_out1k_50ms runner:linux-aarch64-a3-2 test_case:test/registered/ascend/performance/qwen3_6_27b/test_npu_qwen3_6_27b_w8a8_1p_in64k_out1k_50ms.py test_type:perf]) (push) Has been cancelled
Details
PR Test (NPU) / pr-test-npu-finish (push) Has been cancelled
Details
PR Test (Xeon) / pr-gate (push) Has been cancelled
Details
PR Test (Xeon) / check-changes (push) Has been cancelled
Details
PR Test (Xeon) / build-test (, xeon-gnr, base-b-test-cpu) (push) Has been cancelled
Details
PR Test (XPU) / check-changes (push) Has been cancelled
Details
PR Test (XPU) / pr-gate (push) Has been cancelled
Details
PR Test (XPU) / stage-a-test-1-gpu-xpu (push) Has been cancelled
Details
PR Test (XPU) / wait-for-stage-a (push) Has been cancelled
Details
PR Test (XPU) / stage-b-test-1-gpu-xpu (push) Has been cancelled
Details
PR Test (XPU) / finish (push) Has been cancelled
Details
CI Model Inventory / build-inventory (push) Has been cancelled
Details
Lint / lint (push) Has been cancelled
Details
PR Benchmark (SMG Components) / Benchmark Compilation Check (push) Has been cancelled
Details
PR Benchmark (SMG Components) / Benchmark - Manual Policy (push) Has been cancelled
Details
PR Benchmark (SMG Components) / Benchmark - Request Processing (push) Has been cancelled
Details
PR Benchmark (SMG Components) / Benchmark Summary (push) Has been cancelled
Details
PR Test (SMG) / build-wheel (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on windows (x86_64 - auto) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on macos (x86_64 - auto) (push) Has been cancelled
Details
PR Test (SMG) / python-unit-tests (push) Has been cancelled
Details
PR Test (SMG) / unit-tests (push) Has been cancelled
Details
PR Test (SMG) / benchmarks (push) Has been cancelled
Details
PR Test (SMG) / chat-completions (push) Has been cancelled
Details
PR Test (SMG) / chat-completions-4gpu (push) Has been cancelled
Details
PR Test (SMG) / e2e (push) Has been cancelled
Details
PR Test (SMG) / docker-build-test (push) Has been cancelled
Details
PR Test (SMG) / k8s-integration (push) Has been cancelled
Details
PR Test (SMG) / finish (push) Has been cancelled
Details
PR Test (SMG) / summarize-benchmarks (push) Has been cancelled
Details
Release SGLang Model Gateway Docker Image / publish (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on macos (aarch64 - auto) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - auto) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - auto) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on linux (aarch64 - musllinux_1_1) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / build on linux (x86_64 - musllinux_1_1) (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / Build SDist (push) Has been cancelled
Details
Release SGLang Model Gateway to PyPI / Upload to PyPI (push) Has been cancelled
Details
Release SGLang Kernels / build-cu129-matrix (aarch64, 12.9, 3.10, arm-kernel-build-node) (push) Has been cancelled
Details
Release SGLang Kernels / build-cu129-matrix (x86_64, 12.9, 3.10, x64-kernel-build-node) (push) Has been cancelled
Details
Release SGLang Kernels / release-cu129 (push) Has been cancelled
Details
Release SGLang Kernels / build-cu130-matrix (aarch64, 13.0, 3.10, arm-kernel-build-node) (push) Has been cancelled
Details
Release SGLang Kernels / build-cu130-matrix (x86_64, 13.0, 3.10, x64-kernel-build-node) (push) Has been cancelled
Details
Release SGLang Kernels / release-cu130 (push) Has been cancelled
Details
Release SGLang Kernels / build-rocm-matrix (3.10, 700) (push) Has been cancelled
Details
Release SGLang Kernels / build-rocm-matrix (3.10, 720) (push) Has been cancelled
Details
Release SGLang Kernels / release-rocm700 (push) Has been cancelled
Details
Release SGLang Kernels / release-rocm720 (push) Has been cancelled
Details
Release SGLang Kernels / build-musa43 (43, 3.10) (push) Has been cancelled
Details
Release SGLang Kernels / release-musa43 (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:38:16 +08:00
commit 94057c3d3e
7152 changed files with 2120455 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/sglang/multimodal_gen/test/server/ascend/perf_baselines_npu.json
+703
View File
@@ -0,0 +1,703 @@
{
"metadata": {
"model": "Diffusion Server",
"hardware": "Ascend A3",
"description": "Reference numbers captured from the CI diffusion server baseline run"
},
"scenarios": {
"flux_image_t2i_npu": {
"stages_ms": {
"InputValidationStage": 0.06,
"TextEncodingStage": 468.6,
"TimestepPreparationStage": 41.44,
"LatentPreparationStage": 0.28,
"DenoisingStage": 19973.94,
"DecodingStage": 8.58
},
"denoise_step_ms": {
"0": 100.15,
"1": 95.5,
"2": 334.93,
"3": 413.4,
"4": 413.43,
"5": 413.33,
"6": 413.6,
"7": 413.2,
"8": 413.4,
"9": 413.15,
"10": 413.24,
"11": 413.37,
"12": 413.63,
"13": 413.39,
"14": 413.49,
"15": 413.69,
"16": 413.61,
"17": 413.38,
"18": 413.32,
"19": 413.3,
"20": 413.45,
"21": 413.49,
"22": 413.31,
"23": 413.14,
"24": 413.46,
"25": 413.31,
"26": 413.61,
"27": 413.58,
"28": 413.52,
"29": 413.37,
"30": 413.46,
"31": 413.83,
"32": 413.23,
"33": 413.49,
"34": 413.45,
"35": 413.41,
"36": 413.4,
"37": 413.46,
"38": 413.38,
"39": 413.43,
"40": 413.52,
"41": 413.56,
"42": 413.63,
"43": 413.46,
"44": 413.47,
"45": 413.38,
"46": 413.57,
"47": 413.37,
"48": 413.19,
"49": 413.38
},
"expected_e2e_ms": 20670.75,
"expected_avg_denoise_ms": 399.24,
"expected_median_denoise_ms": 413.41
},
"flux_2_image_t2i_2npu": {
"stages_ms": {
"InputValidationStage": 0.08,
"TextEncodingStage": 192.4,
"ImageVAEEncodingStage": 0.01,
"LatentPreparationStage": 0.97,
"TimestepPreparationStage": 34.65,
"DenoisingStage": 45390.41,
"DecodingStage": 5.32
},
"denoise_step_ms": {
"0": 84.23,
"1": 80.0,
"2": 874.6,
"3": 918.81,
"4": 900.4,
"5": 910.67,
"6": 903.36,
"7": 904.97,
"8": 906.84,
"9": 906.39,
"10": 904.99,
"11": 909.96,
"12": 901.67,
"13": 908.8,
"14": 902.93,
"15": 906.17,
"16": 906.67,
"17": 905.89,
"18": 906.9,
"19": 907.1,
"20": 905.31,
"21": 907.93,
"22": 903.68,
"23": 904.49,
"24": 905.73,
"25": 907.66,
"26": 906.71,
"27": 912.69,
"28": 901.4,
"29": 909.9,
"30": 901.72,
"31": 904.35,
"32": 905.61,
"33": 905.97,
"34": 906.5,
"35": 921.45,
"36": 892.58,
"37": 909.03,
"38": 903.82,
"39": 906.25,
"40": 905.18,
"41": 905.06,
"42": 906.21,
"43": 911.62,
"44": 901.04,
"45": 909.09,
"46": 904.29,
"47": 905.16,
"48": 907.35,
"49": 905.65
},
"expected_e2e_ms": 46557.7,
"expected_avg_denoise_ms": 872.7,
"expected_median_denoise_ms": 905.81
},
"wan2_1_t2v_1.3b_1_npu": {
"stages_ms": {
"InputValidationStage": 0.06,
"TextEncodingStage": 2386.55,
"LatentPreparationStage": 0.2,
"TimestepPreparationStage": 3.03,
"DenoisingStage": 26240.89,
"DecodingStage": 720.28,
"per_frame_generation": null
},
"denoise_step_ms": {
"0": 330.0,
"1": 303.37,
"2": 548.94,
"3": 542.02,
"4": 537.7,
"5": 537.26,
"6": 537.78,
"7": 537.27,
"8": 537.72,
"9": 537.65,
"10": 537.49,
"11": 537.41,
"12": 537.31,
"13": 537.43,
"14": 537.41,
"15": 537.51,
"16": 537.54,
"17": 537.56,
"18": 538.09,
"19": 537.09,
"20": 537.72,
"21": 537.56,
"22": 537.85,
"23": 537.54,
"24": 537.67,
"25": 537.66,
"26": 537.25,
"27": 537.71,
"28": 537.77,
"29": 537.16,
"30": 537.6,
"31": 537.58,
"32": 537.6,
"33": 537.51,
"34": 537.78,
"35": 537.43,
"36": 537.52,
"37": 537.59,
"38": 537.46,
"39": 537.77,
"40": 537.41,
"41": 538.11,
"42": 537.41,
"43": 537.18,
"44": 537.64,
"45": 537.32,
"46": 537.52,
"47": 537.52,
"48": 537.29,
"49": 547.88
},
"expected_e2e_ms": 29360.56,
"expected_avg_denoise_ms": 524.67,
"expected_median_denoise_ms": 537.54,
"estimated_full_test_time_s": 157.8
},
"wan2_2_t2v_14b_w8a8_2npu": {
"stages_ms": {
"InputValidationStage": 0.09,
"TextEncodingStage": 2789.3,
"LatentPreparationStage": 0.28,
"TimestepPreparationStage": 3.19,
"DenoisingStage": 187650.19,
"DecodingStage": 3491.88,
"per_frame_generation": null
},
"denoise_step_ms": {
"0": 1415.84,
"1": 4801.54,
"2": 4781.57,
"3": 4770.19,
"4": 4808.86,
"5": 4737.55,
"6": 4774.02,
"7": 4774.85,
"8": 4773.25,
"9": 4775.33,
"10": 4771.31,
"11": 4773.07,
"12": 4773.18,
"13": 4772.77,
"14": 4773.68,
"15": 4771.91,
"16": 4776.16,
"17": 4773.88,
"18": 4769.87,
"19": 4772.1,
"20": 4774.55,
"21": 4772.54,
"22": 4772.39,
"23": 4776.14,
"24": 4772.84,
"25": 4772.82,
"26": 4789.34,
"27": 4792.58,
"28": 4807.52,
"29": 4740.66,
"30": 4773.46,
"31": 4774.19,
"32": 4772.8,
"33": 4774.84,
"34": 4772.52,
"35": 4773.77,
"36": 4775.43,
"37": 4771.58,
"38": 4772.68,
"39": 4769.1
},
"expected_e2e_ms": 193947.19,
"expected_avg_denoise_ms": 4691.07,
"expected_median_denoise_ms": 4773.22
},
"qwen_image_t2i_2npu": {
"stages_ms": {
"InputValidationStage": 0.07,
"TextEncodingStage": 629.24,
"LatentPreparationStage": 0.69,
"TimestepPreparationStage": 35.29,
"DenoisingStage": 30529.83,
"DecodingStage": 428.21
},
"denoise_step_ms": {
"0": 477.43,
"1": 511.96,
"2": 607.78,
"3": 615.12,
"4": 616.29,
"5": 614.61,
"6": 623.04,
"7": 607.12,
"8": 615.32,
"9": 615.47,
"10": 616.93,
"11": 623.26,
"12": 607.12,
"13": 615.48,
"14": 615.07,
"15": 614.83,
"16": 623.18,
"17": 609.0,
"18": 614.8,
"19": 623.08,
"20": 607.64,
"21": 614.2,
"22": 615.58,
"23": 615.43,
"24": 623.59,
"25": 606.57,
"26": 616.02,
"27": 615.48,
"28": 615.76,
"29": 623.13,
"30": 608.73,
"31": 615.04,
"32": 616.08,
"33": 616.59,
"34": 623.77,
"35": 608.0,
"36": 616.1,
"37": 615.79,
"38": 615.34,
"39": 617.43,
"40": 610.99,
"41": 614.22,
"42": 623.27,
"43": 606.98,
"44": 615.87,
"45": 615.99,
"46": 614.66,
"47": 622.93,
"48": 607.97,
"49": 614.69
},
"expected_e2e_ms": 34362.34,
"expected_avg_denoise_ms": 610.41,
"expected_median_denoise_ms": 615.39
},
"ernie_image_t2i_1npu": {
"stages_ms": {
"InputValidationStage": 0.07,
"PromptEnhancementStage": 8787.2,
"TextEncodingStage": 35.87,
"TimestepPreparationStage": 32.74,
"LatentPreparationStage": 0.2,
"DenoisingStage": 47471.93,
"DecodingStage": 42.24
},
"denoise_step_ms": {
"0": 93.3,
"1": 229.78,
"2": 982.46,
"3": 984.1,
"4": 982.02,
"5": 982.42,
"6": 982.01,
"7": 982.12,
"8": 981.92,
"9": 982.08,
"10": 982.39,
"11": 982.14,
"12": 982.37,
"13": 982.06,
"14": 982.0,
"15": 982.53,
"16": 981.83,
"17": 982.41,
"18": 981.97,
"19": 981.93,
"20": 982.14,
"21": 981.71,
"22": 982.36,
"23": 981.97,
"24": 981.92,
"25": 982.12,
"26": 981.87,
"27": 982.38,
"28": 981.92,
"29": 981.95,
"30": 982.28,
"31": 981.94,
"32": 981.95,
"33": 982.2,
"34": 981.71,
"35": 982.29,
"36": 981.84,
"37": 982.2,
"38": 981.98,
"39": 981.93,
"40": 982.3,
"41": 982.27,
"42": 982.15,
"43": 981.96,
"44": 981.82,
"45": 982.11,
"46": 981.79,
"47": 982.17,
"48": 982.01,
"49": 981.89
},
"expected_e2e_ms": 56379.07,
"expected_avg_denoise_ms": 949.3,
"expected_median_denoise_ms": 982.02
},
"glm_image_t2i_1npu": {
"stages_ms": {
"GlmImageAR": 69299.14,
"GlmImageBeforeDenoisingStage": 94.01,
"DenoisingStage": 24325.65,
"DecodingStage": 290.05
},
"denoise_step_ms": {
"0": 548.52,
"1": 819.64,
"2": 819.1,
"3": 819.05,
"4": 819.51,
"5": 819.38,
"6": 818.79,
"7": 819.04,
"8": 818.91,
"9": 819.87,
"10": 819.65,
"11": 825.78,
"12": 812.68,
"13": 819.45,
"14": 820.37,
"15": 820.32,
"16": 819.2,
"17": 821.85,
"18": 817.57,
"19": 819.32,
"20": 819.99,
"21": 819.73,
"22": 819.84,
"23": 819.79,
"24": 820.7,
"25": 819.27,
"26": 819.59,
"27": 820.06,
"28": 819.45,
"29": 820.13
},
"expected_e2e_ms": 94017.76,
"expected_avg_denoise_ms": 810.55,
"expected_median_denoise_ms": 819.55
},
"z_image_t2i_1npu": {
"stages_ms": {
"InputValidationStage": 0.08,
"TextEncodingStage": 246.13,
"LatentPreparationStage": 0.23,
"TimestepPreparationStage": 29.16,
"DenoisingStage": 36649.71,
"DecodingStage": 6.28
},
"denoise_step_ms": {
"0": 732.97,
"1": 732.49,
"2": 732.47,
"3": 732.93,
"4": 732.5,
"5": 732.51,
"6": 732.18,
"7": 733.07,
"8": 733.17,
"9": 734.29,
"10": 734.49,
"11": 735.74,
"12": 733.51,
"13": 732.76,
"14": 733.39,
"15": 732.44,
"16": 731.72,
"17": 731.76,
"18": 732.76,
"19": 731.88,
"20": 732.23,
"21": 731.63,
"22": 731.59,
"23": 732.4,
"24": 732.18,
"25": 732.54,
"26": 732.08,
"27": 732.09,
"28": 732.08,
"29": 732.44,
"30": 732.18,
"31": 731.93,
"32": 732.05,
"33": 732.32,
"34": 733.97,
"35": 736.5,
"36": 735.63,
"37": 733.37,
"38": 734.09,
"39": 732.9,
"40": 732.78,
"41": 732.26,
"42": 731.74,
"43": 732.48,
"44": 732.14,
"45": 732.64,
"46": 733.53,
"47": 733.48,
"48": 732.59,
"49": 732.1
},
"expected_e2e_ms": 36940.12,
"expected_avg_denoise_ms": 732.82,
"expected_median_denoise_ms": 732.5
},
"flux_2_klein_4b_t2i_1npu": {
"stages_ms": {
"InputValidationStage": 0.07,
"TextEncodingStage": 232.96,
"ImageVAEEncodingStage": 0.01,
"LatentPreparationStage": 0.57,
"TimestepPreparationStage": 29.26,
"DenoisingStage": 992.79,
"DecodingStage": 10.76
},
"denoise_step_ms": {
"0": 37.78,
"1": 35.77,
"2": 34.34,
"3": 34.91
},
"expected_e2e_ms": 1317.11,
"expected_avg_denoise_ms": 35.7,
"expected_median_denoise_ms": 35.34
},
"joyai_image_edit_ti2i_1npu": {
"stages_ms": {
"InputValidationStage": 20.04,
"ImageEncodingStage": 1239.81,
"ImageVAEEncodingStage": 370.97,
"LatentPreparationStage": 0.33,
"TimestepPreparationStage": 34.12,
"DenoisingStage": 105471.23,
"DecodingStage": 2762.97
},
"denoise_step_ms": {
"0": 229.17,
"1": 2429.69,
"2": 2706.14,
"3": 2707.12,
"4": 2702.84,
"5": 2704.71,
"6": 2704.96,
"7": 2706.45,
"8": 2705.75,
"9": 2703.3,
"10": 2705.13,
"11": 2704.95,
"12": 2704.57,
"13": 2705.43,
"14": 2702.58,
"15": 2705.44,
"16": 2705.74,
"17": 2706.63,
"18": 2705.73,
"19": 2703.79,
"20": 2704.97,
"21": 2707.3,
"22": 2705.92,
"23": 2708.38,
"24": 2703.5,
"25": 2704.63,
"26": 2706.51,
"27": 2704.44,
"28": 2706.82,
"29": 2704.59,
"30": 2704.36,
"31": 2707.21,
"32": 2704.87,
"33": 2708.24,
"34": 2703.86,
"35": 2704.19,
"36": 2706.7,
"37": 2705.72,
"38": 2706.74,
"39": 2704.96
},
"expected_e2e_ms": 109909.47,
"expected_avg_denoise_ms": 2636.6,
"expected_median_denoise_ms": 2705.05
},
"mova_360p_ti2va_2npu": {
"stages_ms": {
"InputValidationStage": 19.09,
"TextEncodingStage": 512.61,
"ImageVAEEncodingStage": 588.51,
"MOVALatentPreparationStage": 0.34,
"MOVATimestepPreparationStage": 0.52,
"MOVADenoisingStage": 91033.05,
"MOVADecodingStage": 586.86
},
"denoise_step_ms": {
"0": 1816.84,
"1": 1818.1,
"2": 1819.39,
"3": 1817.58,
"4": 1820.35,
"5": 1816.92,
"6": 1811.79,
"7": 1826.0,
"8": 1816.3,
"9": 1817.59,
"10": 1816.24,
"11": 1815.25,
"12": 1815.69,
"13": 1817.23,
"14": 1817.59,
"15": 1820.19,
"16": 1820.93,
"17": 1823.82,
"18": 1848.39,
"19": 1819.37,
"20": 1818.51,
"21": 1815.18,
"22": 1821.84,
"23": 1816.08,
"24": 1817.98,
"25": 1817.91,
"26": 1818.12,
"27": 1816.79,
"28": 1815.43,
"29": 1818.44,
"30": 1817.64,
"31": 1818.73,
"32": 1814.81,
"33": 1818.24,
"34": 1817.88,
"35": 1816.15,
"36": 1818.26,
"37": 1814.76,
"38": 1816.17,
"39": 1817.48,
"40": 1817.83,
"41": 1817.71,
"42": 1811.81,
"43": 1820.49,
"44": 1817.3,
"45": 1814.35,
"46": 1820.92,
"47": 1824.75,
"48": 1822.46,
"49": 1822.08
},
"expected_e2e_ms": 92840.23,
"expected_avg_denoise_ms": 1818.63,
"expected_median_denoise_ms": 1817.77,
"estimated_full_test_time_s": 175.6
},
"ltx_2_ti2va_2npu": {
"stages_ms": {
"InputValidationStage": 3.8,
"TextEncodingStage": 1029.19,
"LTX2TextConnectorStage": 56.3,
"LTX2SigmaPreparationStage": 0.19,
"TimestepPreparationStage": 33.52,
"LTX2AVLatentPreparationStage": 0.45,
"LTX2ImageEncodingStage": 93.32,
"LTX2AVDenoisingStage": 29672.06,
"LTX2AVDecodingStage": 560.9
},
"denoise_step_ms": {
"0": 544.81,
"1": 746.36,
"2": 746.01,
"3": 774.46,
"4": 717.7,
"5": 749.83,
"6": 746.58,
"7": 784.74,
"8": 708.78,
"9": 746.72,
"10": 746.38,
"11": 784.14,
"12": 708.74,
"13": 746.85,
"14": 746.57,
"15": 751.14,
"16": 742.59,
"17": 747.19,
"18": 746.26,
"19": 750.58,
"20": 742.55,
"21": 746.34,
"22": 760.44,
"23": 736.36,
"24": 741.97,
"25": 746.3,
"26": 782.28,
"27": 714.3,
"28": 741.96,
"29": 746.25,
"30": 784.82,
"31": 711.85,
"32": 742.0,
"33": 746.24,
"34": 746.13,
"35": 750.84,
"36": 741.81,
"37": 746.2,
"38": 746.08,
"39": 752.12
},
"expected_e2e_ms": 31484.37,
"expected_avg_denoise_ms": 741.58,
"expected_median_denoise_ms": 746.28
}
}
}
python/sglang/multimodal_gen/test/server/ascend/test_server_1_npu.py
+23
View File
@@ -0,0 +1,23 @@
"""
Config-driven diffusion performance test with pytest parametrization.
If the actual run is significantly better than the baseline, the improved cases with their updated baseline will be printed
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.ascend.testcase_configs_npu import ONE_NPU_CASES
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneNpu(DiffusionServerBase):
"""Performance tests for 1-NPU diffusion cases."""
case = diffusion_case_fixture(ONE_NPU_CASES)
python/sglang/multimodal_gen/test/server/ascend/test_server_2_npu.py
+23
View File
@@ -0,0 +1,23 @@
"""
Config-driven diffusion performance test with pytest parametrization.
If the actual run is significantly better than the baseline, the improved cases with their updated baseline will be printed
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.ascend.testcase_configs_npu import TWO_NPU_CASES
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerTwoNpu(DiffusionServerBase):
"""Performance tests for 2-NPU diffusion cases."""
case = diffusion_case_fixture(TWO_NPU_CASES)
python/sglang/multimodal_gen/test/server/ascend/testcase_configs_npu.py
+200
View File
@@ -0,0 +1,200 @@
import os
from sglang.multimodal_gen.test.server.testcase_configs import (
T2V_PROMPT,
DiffusionSamplingParams,
DiffusionServerArgs,
DiffusionTestCase,
T2I_sampling_params,
TI2V_sampling_params,
)
MODELSCOPE_MODEL_WEIGHTS_DIR = "/root/.cache/modelscope/hub/models/"
def use_modelscope(name: str):
return os.path.join(MODELSCOPE_MODEL_WEIGHTS_DIR, name)
COSMOS3_NANO_WEIGHTS_PATH = use_modelscope("nv-community/Cosmos3-Nano")
ERNIE_IMAGE_WEIGHTS_PATH = use_modelscope("PaddlePaddle/ERNIE-Image")
FLUX_1_DEV_WEIGHTS_PATH = use_modelscope("black-forest-labs/FLUX.1-dev")
FLUX_2_DEV_WEIGHTS_PATH = use_modelscope("black-forest-labs/FLUX.2-dev")
FLUX_2_KLEIN_4B_WEIGHTS_PATH = use_modelscope("black-forest-labs/FLUX.2-klein-4B")
GLM_IMAGE_WEIGHTS_PATH = use_modelscope("ZhipuAI/GLM-Image")
JOYAI_IMAGE_EDIT_WEIGHTS_PATH = use_modelscope(
"jd-opensource/JoyAI-Image-Edit-Diffusers"
)
LTX_2_WEIGHTS_PATH = use_modelscope("Lightricks/LTX-2")
MOVA_360_WEIGHTS_PATH = use_modelscope("openmoss/MOVA-360p")
QWEN_IMAGE_WEIGHTS_PATH = use_modelscope("Qwen/Qwen-Image")
WAN2_1_T2V_1_3B_DIFFUSERS_WEIGHTS_PATH = use_modelscope(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
)
WAN2_2_T2V_A14B_DIFFUSERS_W8A8_WEIGHTS_PATH = use_modelscope(
"Eco-Tech/Wan2.2-T2V-A14B-Diffusers-w8a8"
)
Z_IMAGE_WEIGHTS_PATH = use_modelscope("Tongyi-MAI/Z-Image")
EXTRAS_DISABLE_WARMUP = ["--warmup-mode", "request"]
ONE_NPU_CASES: list[DiffusionTestCase] = [
# === Text to Image (T2I) ===
DiffusionTestCase(
"ernie_image_t2i_1npu",
DiffusionServerArgs(
model_path=ERNIE_IMAGE_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"glm_image_t2i_1npu",
DiffusionServerArgs(
model_path=GLM_IMAGE_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"flux_image_t2i_npu",
DiffusionServerArgs(
model_path=FLUX_1_DEV_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
),
DiffusionTestCase(
"flux_2_klein_4b_t2i_1npu",
DiffusionServerArgs(
model_path=FLUX_2_KLEIN_4B_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"z_image_t2i_1npu",
DiffusionServerArgs(
model_path=Z_IMAGE_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
run_consistency_check=False,
),
# === Text to Video (T2V) ===
DiffusionTestCase(
"wan2_1_t2v_1.3b_1_npu",
DiffusionServerArgs(
model_path=WAN2_1_T2V_1_3B_DIFFUSERS_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
DiffusionSamplingParams(
prompt=T2V_PROMPT,
),
),
# === Text+Image to Image (TI2I)
DiffusionTestCase(
"joyai_image_edit_ti2i_1npu",
DiffusionServerArgs(
model_path=JOYAI_IMAGE_EDIT_WEIGHTS_PATH,
extras=EXTRAS_DISABLE_WARMUP,
),
run_consistency_check=False,
run_component_accuracy_check=False,
),
]
TWO_NPU_CASES: list[DiffusionTestCase] = [
# === Text to Image (T2I) ===
DiffusionTestCase(
"flux_2_image_t2i_2npu",
DiffusionServerArgs(
model_path=FLUX_2_DEV_WEIGHTS_PATH,
num_gpus=2,
tp_size=2,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
),
DiffusionTestCase(
"qwen_image_t2i_2npu",
DiffusionServerArgs(
model_path=QWEN_IMAGE_WEIGHTS_PATH,
num_gpus=2,
# test ring attn
ulysses_degree=1,
ring_degree=2,
extras=EXTRAS_DISABLE_WARMUP,
),
T2I_sampling_params,
),
# === Text to Video (T2V) ===
DiffusionTestCase(
"wan2_2_t2v_14b_w8a8_2npu",
DiffusionServerArgs(
model_path=WAN2_2_T2V_A14B_DIFFUSERS_W8A8_WEIGHTS_PATH,
num_gpus=2,
tp_size=1,
ulysses_degree=2,
extras=EXTRAS_DISABLE_WARMUP,
),
DiffusionSamplingParams(
prompt=T2V_PROMPT,
),
),
# === Text+Image to Video+Audio (TI2V)
DiffusionTestCase(
"ltx_2_ti2va_2npu",
DiffusionServerArgs(
model_path=LTX_2_WEIGHTS_PATH,
num_gpus=2,
ulysses_degree=2,
extras=EXTRAS_DISABLE_WARMUP,
),
TI2V_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"mova_360p_ti2va_2npu",
DiffusionServerArgs(
model_path=MOVA_360_WEIGHTS_PATH,
num_gpus=2,
tp_size=2,
dit_layerwise_offload=True,
extras=EXTRAS_DISABLE_WARMUP,
),
run_consistency_check=False,
),
]
DEFAULT_EST_TIME_SECONDS = 300.0
STARTUP_OVERHEAD_SECONDS = 120.0
DEFAULT_STANDALONE_EST_TIME_SECONDS = 300.0
SUITES = {
"1-npu": [
"ascend/test_server_1_npu.py",
# add new 1-npu test files here
],
"2-npu": [
"ascend/test_server_2_npu.py",
# add new 2-npu test files here
],
}
PARAMETRIZED_CASE_GROUPS = {
"1-npu": [
("ascend/test_server_1_npu.py", ONE_NPU_CASES),
],
"2-npu": [
("ascend/test_server_2_npu.py", TWO_NPU_CASES),
],
}
FILE_SUITES = {}
STANDALONE_FILES = {}
COMPONENT_ACCURACY_SUITES = {}
_UPDATE_WEIGHTS_FROM_DISK_TEST_FILE = None
python/sglang/multimodal_gen/test/server/common/__init__.py
+1
View File
@@ -0,0 +1 @@
"""Common server test helpers."""
python/sglang/multimodal_gen/test/server/common/case_fixtures.py
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Sequence
import pytest
if TYPE_CHECKING:
from sglang.multimodal_gen.test.server.testcase_configs import DiffusionTestCase
def diffusion_case_fixture(cases: Sequence[DiffusionTestCase]):
@pytest.fixture(params=cases, ids=lambda case: case.id)
def case(self, request) -> DiffusionTestCase:
return request.param
return case
python/sglang/multimodal_gen/test/server/common/slack.py
+216
View File
@@ -0,0 +1,216 @@
"""
This file upload the media generated in diffusion-nightly-test to a slack channel of SGLang
"""
import logging
import os
import tempfile
from datetime import datetime
from typing import List, Union
from urllib.parse import urlparse
from urllib.request import urlopen
from sglang.multimodal_gen.runtime.utils.perf_logger import get_git_commit_hash
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
import inspect
try:
import sglang.multimodal_gen.test.server.testcase_configs as configs
from sglang.multimodal_gen.test.server.testcase_configs import DiffusionTestCase
ALL_CASES = []
for name, value in inspect.getmembers(configs):
if name.endswith("_CASES") or "_CASES_" in name:
if (
isinstance(value, list)
and len(value) > 0
and isinstance(value[0], DiffusionTestCase)
):
ALL_CASES.extend(value)
elif isinstance(value, list) and len(value) == 0:
# Assume empty list with matching name is a valid case list container
pass
# Deduplicate cases by ID
seen_ids = set()
unique_cases = []
for c in ALL_CASES:
if c.id not in seen_ids:
seen_ids.add(c.id)
unique_cases.append(c)
ALL_CASES = unique_cases
except Exception as e:
logger.warning(f"Failed to import test cases: {e}")
ALL_CASES = []
def _get_status_message(run_id, current_case_id, thread_messages=None):
date_str = datetime.now().strftime("%d/%m")
base_header = f"""🧵 for nightly test of {date_str}
*Git Revision:* {get_git_commit_hash()}
*GitHub Run ID:* {run_id}
*Total Tasks:* {len(ALL_CASES)}
"""
if not ALL_CASES:
return base_header
default_emoji_for_case_in_progress = "⏳"
status_map = {c.id: default_emoji_for_case_in_progress for c in ALL_CASES}
if thread_messages:
for msg in thread_messages:
text = msg.get("text", "")
# Look for case_id in the message (format: *Case ID:* `case_id`)
for c in ALL_CASES:
if f"*Case ID:* `{c.id}`" in text:
status_map[c.id] = "✅"
if current_case_id:
status_map[current_case_id] = "✅"
lines = [base_header, "", "*Tasks Status:*"]
# Calculate padding
max_len = max(len(c.id) for c in ALL_CASES) if ALL_CASES else 10
max_len = max(max_len, len("Case ID"))
# Build markdown table inside a code block
table_lines = ["```"]
table_lines.append(f"| {'Case ID'.ljust(max_len)} | Status |")
table_lines.append(f"| {'-' * max_len} | :----: |")
for c in ALL_CASES:
mark = status_map.get(c.id, default_emoji_for_case_in_progress)
table_lines.append(f"| {c.id.ljust(max_len)} | {mark} |")
table_lines.append("```")
lines.extend(table_lines)
return "\n".join(lines)
def upload_file_to_slack(
case_id: str = None,
model: str = None,
prompt: str = None,
file_path: str = None,
origin_file_path: Union[str, List[str]] = None,
) -> bool:
temp_paths = []
try:
from slack_sdk import WebClient
run_id = os.getenv("GITHUB_RUN_ID", "local")
token = os.environ.get("SGLANG_DIFFUSION_SLACK_TOKEN")
if not token:
logger.info("Slack upload failed: no token")
return False
if not file_path or not os.path.exists(file_path):
logger.info("Slack upload failed: no file path")
return False
origin_paths = []
if isinstance(origin_file_path, str):
if origin_file_path:
origin_paths.append(origin_file_path)
elif isinstance(origin_file_path, list):
origin_paths = [p for p in origin_file_path if p]
final_origin_paths = []
for path in origin_paths:
if path.startswith(("http", "https")):
try:
suffix = os.path.splitext(urlparse(path).path)[1] or ".tmp"
with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as tf:
with urlopen(path, timeout=30) as response:
tf.write(response.read())
temp_paths.append(tf.name)
final_origin_paths.append(tf.name)
except Exception as e:
logger.warning(f"Failed to download {path}: {e}")
else:
final_origin_paths.append(path)
uploads = []
for i, path in enumerate(final_origin_paths):
if os.path.exists(path):
title = (
"Original Image"
if len(final_origin_paths) == 1
else f"Original Image {i+1}"
)
uploads.append({"file": path, "title": title})
uploads.append({"file": file_path, "title": "Generated Image"})
message = (
f"*Case ID:* `{case_id}`\n" f"*Model:* `{model}`\n" f"*Prompt:* {prompt}"
)
client = WebClient(token=token, timeout=60)
channel_id = "C0A02NDF7UY"
thread_ts = None
parent_msg_text = None
try:
history = client.conversations_history(channel=channel_id, limit=100)
for msg in history.get("messages", []):
if f"*GitHub Run ID:* {run_id}" in msg.get("text", ""):
# Use thread_ts if it exists (msg is a reply), otherwise use ts (msg is a parent)
thread_ts = msg.get("thread_ts") or msg.get("ts")
parent_msg_text = msg.get("text", "")
logger.info(f"Found thread_ts: {thread_ts}")
break
except Exception as e:
logger.warning(f"Failed to search slack history: {e}")
if not thread_ts:
try:
text = _get_status_message(run_id, case_id)
response = client.chat_postMessage(channel=channel_id, text=text)
thread_ts = response["ts"]
except Exception as e:
logger.warning(f"Failed to create parent thread: {e}")
# Upload first to ensure it's in history
client.files_upload_v2(
channel=channel_id,
file_uploads=uploads,
initial_comment=message,
thread_ts=thread_ts,
)
# Then update status based on thread replies
if thread_ts:
try:
replies = client.conversations_replies(
channel=channel_id, ts=thread_ts, limit=200
)
messages = replies.get("messages", [])
new_text = _get_status_message(run_id, case_id, messages)
# Only update if changed significantly (ignoring timestamp diffs if any)
# But here we just check text content
if new_text != parent_msg_text:
client.chat_update(channel=channel_id, ts=thread_ts, text=new_text)
except Exception as e:
logger.warning(f"Failed to update parent message: {e}")
logger.info(f"File uploaded successfully: {os.path.basename(file_path)}")
return True
except Exception as e:
logger.info(f"Slack upload failed: {e}")
return False
finally:
for p in temp_paths:
if os.path.exists(p):
os.remove(p)
python/sglang/multimodal_gen/test/server/configs/cache_dit_scm_config.yaml
+13
View File
@@ -0,0 +1,13 @@
cache_config:
max_warmup_steps: 2
warmup_interval: 2
max_cached_steps: -1
max_continuous_cached_steps: 2
Fn_compute_blocks: 1
Bn_compute_blocks: 0
residual_diff_threshold: 0.12
enable_taylorseer: true
taylorseer_order: 1
num_inference_steps: 8
steps_computation_mask: "medium"
steps_computation_policy: dynamic
python/sglang/multimodal_gen/test/server/conftest.py
+165
View File
@@ -0,0 +1,165 @@
import json
import os
import pytest
print("[CONFTEST] Loading conftest.py at import time")
def pytest_configure(config):
"""
Create the perf results StashKey once and store it in config.
This hook runs once per test session, before module double-import issues.
"""
if not hasattr(config, "_diffusion_perf_key"):
config._diffusion_perf_key = pytest.StashKey[list]()
print(f"[CONFTEST] Created perf_results_key: {config._diffusion_perf_key}")
def add_perf_results(config, results: list):
"""Add performance results to the shared stash."""
# Get the shared key from config (created once in pytest_configure)
key = config._diffusion_perf_key
existing = config.stash.get(key, [])
existing.extend(results)
config.stash[key] = existing
print(f"[CONFTEST] Added {len(results)} results, total now: {len(existing)}")
@pytest.fixture(scope="session")
def perf_config(request):
"""Provide access to pytest config for storing perf results."""
return request.config
def _write_github_step_summary(content: str):
"""Write content to GitHub Step Summary if available."""
summary_file = os.environ.get("GITHUB_STEP_SUMMARY")
if summary_file:
with open(summary_file, "a") as f:
f.write(content)
def _write_results_json(results: list, output_path: str = "diffusion-results.json"):
"""Write performance results to JSON file for CI artifact collection."""
try:
with open(output_path, "w") as f:
json.dump(results, f, indent=2)
print(f"[CONFTEST] Wrote results to {output_path}")
except Exception as e:
print(f"[CONFTEST] Failed to write results JSON: {e}")
def _generate_diffusion_markdown_report(results: list) -> str:
"""Generate a markdown report for diffusion performance results."""
if not results:
return ""
gpu_config = os.environ.get("GPU_CONFIG", "")
header = "## Diffusion Performance Summary"
if gpu_config:
header += f" [{gpu_config}]"
header += "\n\n"
# Main performance table
markdown = header
markdown += "| Test Suite | Test Name | Modality | E2E (ms) | Avg Denoise (ms) | Median Denoise (ms) |\n"
markdown += "| ---------- | --------- | -------- | -------- | ---------------- | ------------------- |\n"
for entry in sorted(results, key=lambda x: (x["class_name"], x["test_name"])):
modality = entry.get("modality", "image")
markdown += (
f"| {entry['class_name']} | {entry['test_name']} | {modality} | "
f"{entry['e2e_ms']:.2f} | {entry['avg_denoise_ms']:.2f} | "
f"{entry['median_denoise_ms']:.2f} |\n"
)
# Video-specific metrics table (if any video tests)
video_results = [r for r in results if r.get("modality") == "video"]
if video_results:
markdown += "\n### Video Generation Metrics\n\n"
markdown += "| Test Name | FPS | Total Frames | Avg Frame Time (ms) |\n"
markdown += "| --------- | --- | ------------ | ------------------- |\n"
for entry in video_results:
fps = entry.get("frames_per_second", "N/A")
frames = entry.get("total_frames", "N/A")
avg_frame = entry.get("avg_frame_time_ms", "N/A")
if isinstance(fps, float):
fps = f"{fps:.2f}"
if isinstance(avg_frame, float):
avg_frame = f"{avg_frame:.2f}"
markdown += f"| {entry['test_name']} | {fps} | {frames} | {avg_frame} |\n"
return markdown
def pytest_sessionfinish(session):
"""
This hook is called by pytest at the end of the entire test session.
It prints a consolidated summary of all performance results.
"""
# Get results from stash using the shared key from config
key = session.config._diffusion_perf_key
results = session.config.stash.get(key, [])
print(f"\n[DEBUG] pytest_sessionfinish called, has {len(results)} entries")
if not results:
print("[DEBUG] No results collected, skipping summary output")
return
sorted_results = sorted(results, key=lambda x: (x["class_name"], x["test_name"]))
# Print to stdout (existing behavior)
print("\n\n" + "=" * 35 + " Performance Summary " + "=" * 35)
print(
f"{'Test Suite':<30} | {'Test Name':<20} | {'E2E (ms)':>12} | {'Avg Denoise (ms)':>18} | {'Median Denoise (ms)':>20}"
)
print(
"-" * 30
+ "-+-"
+ "-" * 20
+ "-+-"
+ "-" * 12
+ "-+-"
+ "-" * 18
+ "-+-"
+ "-" * 20
)
for entry in sorted_results:
print(
f"{entry['class_name']:<30} | {entry['test_name']:<20} | {entry['e2e_ms']:>12.2f} | "
f"{entry['avg_denoise_ms']:>18.2f} | {entry['median_denoise_ms']:>20.2f}"
)
print("=" * 91)
print("\n\n" + "=" * 36 + " Detailed Reports " + "=" * 37)
for entry in sorted_results:
print(f"\n--- Details for {entry['class_name']} / {entry['test_name']} ---")
stage_report = ", ".join(
f"{name}:{duration:.2f}ms"
for name, duration in entry.get("stage_metrics", {}).items()
)
if stage_report:
print(f" Stages: {stage_report}")
sampled_steps = entry.get("sampled_steps") or {}
if sampled_steps:
step_report = ", ".join(
f"{idx}:{duration:.2f}ms"
for idx, duration in sorted(sampled_steps.items())
)
print(f" Sampled Steps: {step_report}")
print("=" * 91)
print("\n\n" + "=" * 34 + " Performance Data JSON " + "=" * 34)
print(json.dumps(sorted_results, indent=2, sort_keys=True))
print("=" * 91)
# Write to GitHub Step Summary (new behavior for CI monitoring)
markdown_report = _generate_diffusion_markdown_report(sorted_results)
if markdown_report:
_write_github_step_summary(markdown_report)
# Write results to JSON file for CI artifact collection
_write_results_json(sorted_results)
python/sglang/multimodal_gen/test/server/consistency_thresholds/5090.json
+10
View File
@@ -0,0 +1,10 @@
{
"cases": {
"zimage_image_t2i": {
"clip_threshold": 0.92,
"ssim_threshold": 0.86,
"psnr_threshold": 19.5,
"mean_abs_diff_threshold": 8.5
}
}
}
python/sglang/multimodal_gen/test/server/consistency_thresholds/b200.json
+3
View File
@@ -0,0 +1,3 @@
{
"cases": {}
}
python/sglang/multimodal_gen/test/server/consistency_thresholds/h100.json
+289
View File
@@ -0,0 +1,289 @@
{
"_comment": "Some cases use lower thresholds; raise them if quality/perf improves later.",
"cases": {
"qwen_image_t2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.83,
"psnr_threshold": 16.0,
"mean_abs_diff_threshold": 13.3
},
"flux_image_t2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 24.0,
"mean_abs_diff_threshold": 5.5
},
"flux1_modelopt_fp8_t2i": {
"clip_threshold": 0.96,
"ssim_threshold": 0.94,
"psnr_threshold": 28.0,
"mean_abs_diff_threshold": 6.3
},
"ideogram4_nvfp4_t2i": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 18.0,
"mean_abs_diff_threshold": 4.0
},
"qwen_image_2512_modelopt_nvfp4_t2i": {
"clip_threshold": 0.95,
"ssim_threshold": 0.88,
"psnr_threshold": 23.0,
"mean_abs_diff_threshold": 9.0
},
"flux_2_klein_image_t2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.83,
"psnr_threshold": 17.0,
"mean_abs_diff_threshold": 12.5
},
"zimage_image_t2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.86,
"psnr_threshold": 19.9,
"mean_abs_diff_threshold": 8.5
},
"zimage_image_t2i_multi_lora": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 19.9,
"mean_abs_diff_threshold": 4.0
},
"qwen_image_t2i_cache_dit_enabled": {
"clip_threshold": 0.98,
"ssim_threshold": 0.78,
"psnr_threshold": 13.5,
"mean_abs_diff_threshold": 17.2
},
"qwen_image_t2i_2_gpus": {
"clip_threshold": 0.98,
"ssim_threshold": 0.79,
"psnr_threshold": 15.7,
"mean_abs_diff_threshold": 17.2
},
"flux_2_image_t2i": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 14.9,
"mean_abs_diff_threshold": 4.0
},
"flux_2_ti2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.88,
"psnr_threshold": 19.5,
"mean_abs_diff_threshold": 13.5
},
"zimage_image_t2i_fp8": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"qwen_image_edit_2509_ti2i": {
"clip_threshold": 0.91,
"ssim_threshold": 0.68,
"psnr_threshold": 11.3,
"mean_abs_diff_threshold": 26.4
},
"qwen_image_edit_ti2i": {
"clip_threshold": 0.96,
"ssim_threshold": 0.94,
"psnr_threshold": 25.4,
"mean_abs_diff_threshold": 10.0
},
"qwen_image_edit_2511_ti2i": {
"clip_threshold": 0.96,
"ssim_threshold": 0.83,
"psnr_threshold": 20.5,
"mean_abs_diff_threshold": 17.0
},
"qwen_image_layered_i2i": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.2
},
"wan2_1_t2v_1_3b_lora_1gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.94,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"wan2_1_t2v_1.3b": {
"clip_threshold": 0.97,
"ssim_threshold": 0.85,
"psnr_threshold": 25.0,
"mean_abs_diff_threshold": 8.0
},
"cosmos3_nano_t2v": {
"clip_threshold": 0.91,
"ssim_threshold": 0.89,
"psnr_threshold": 24.0,
"mean_abs_diff_threshold": 9.9
},
"ltx_2_two_stage_t2v": {
"clip_threshold": 0.97,
"ssim_threshold": 0.89,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 5.9
},
"ltx_2.3_one_stage_ti2v": {
"clip_threshold": 0.87,
"ssim_threshold": 0.78,
"psnr_threshold": 20.5,
"mean_abs_diff_threshold": 11.0
},
"ltx_2.3_two_stage_t2v_2gpus": {
"clip_threshold": 0.94,
"ssim_threshold": 0.45,
"psnr_threshold": 16.0,
"mean_abs_diff_threshold": 27.5
},
"wan2_1_t2v_1.3b_teacache_enabled": {
"clip_threshold": 0.97,
"ssim_threshold": 0.92,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 5.5
},
"wan2_1_t2v_1.3b_upscaling_4x": {
"clip_threshold": 0.97,
"ssim_threshold": 0.94,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.6
},
"wan2_2_ti2v_5b": {
"clip_threshold": 0.94,
"ssim_threshold": 0.88,
"psnr_threshold": 27.1,
"mean_abs_diff_threshold": 7.2
},
"fastwan2_2_ti2v_5b": {
"clip_threshold": 0.97,
"ssim_threshold": 0.92,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.7
},
"turbo_wan2_1_t2v_1.3b": {
"clip_threshold": 0.90,
"ssim_threshold": 0.52,
"psnr_threshold": 9.5,
"mean_abs_diff_threshold": 46.0
},
"fsdp-inference": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"zimage_image_t2i_2_gpus_non_square": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"flux_2_image_t2i_2_gpus": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 19,
"mean_abs_diff_threshold": 4.0
},
"flux_2_image_t2i_upscaling_4x": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"flux_2_t2i_customized_vae_path": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 28.0,
"mean_abs_diff_threshold": 4.0
},
"flux_2_ti2i_multi_image_cache_dit": {
"clip_threshold": 0.94,
"ssim_threshold": 0.90,
"psnr_threshold": 22.0,
"mean_abs_diff_threshold": 9.0
},
"zimage_image_t2i_2_gpus": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 4.0
},
"flux_image_t2i_2_gpus": {
"clip_threshold": 0.97,
"ssim_threshold": 0.95,
"psnr_threshold": 25.8,
"mean_abs_diff_threshold": 4.0
},
"wan2_2_t2v_a14b_teacache_2gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.90,
"psnr_threshold": 29.1,
"mean_abs_diff_threshold": 6.8
},
"wan2_1_t2v_14b_2gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.91,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 5.6
},
"wan2_1_i2v_14b_lora_2gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.90,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 5.3
},
"wan2_2_t2v_a14b_2gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.90,
"psnr_threshold": 29.1,
"mean_abs_diff_threshold": 6.8
},
"wan2_2_t2v_a14b_lora_2gpu": {
"clip_threshold": 0.98,
"ssim_threshold": 0.95,
"psnr_threshold": 22.2,
"mean_abs_diff_threshold": 4.0
},
"wan2_1_i2v_14b_480P_2gpu": {
"clip_threshold": 0.92,
"ssim_threshold": 0.75,
"psnr_threshold": 21.4,
"mean_abs_diff_threshold": 9.7
},
"wan2_1_i2v_14b_720P_2gpu": {
"clip_threshold": 0.97,
"ssim_threshold": 0.90,
"psnr_threshold": 30.0,
"mean_abs_diff_threshold": 5.5
},
"ltx_2_3_hq_pipeline": {
"clip_threshold": 0.995,
"ssim_threshold": 0.995,
"psnr_threshold": 50.0,
"mean_abs_diff_threshold": 0.5
},
"ltx_2_3_two_stage_ti2v_2gpus": {
"clip_threshold": 0.55,
"ssim_threshold": 0.29,
"psnr_threshold": 11.7,
"mean_abs_diff_threshold": 47.0
},
"joy_echo_t2v_2gpu": {
"clip_threshold": 0.78,
"ssim_threshold": 0.48,
"psnr_threshold": 13.0,
"mean_abs_diff_threshold": 45.0
}
},
"default_clip_threshold_image": 0.92,
"default_clip_threshold_video": 0.90,
"default_ssim_threshold_image": 0.95,
"default_psnr_threshold_image": 28.0,
"default_mean_abs_diff_threshold_image": 8.0,
"default_ssim_threshold_video": 0.92,
"default_psnr_threshold_video": 24.0,
"default_mean_abs_diff_threshold_video": 10.0
}
python/sglang/multimodal_gen/test/server/gpu_cases.py
+1025
View File
File diff suppressed because it is too large Load Diff
python/sglang/multimodal_gen/test/server/musa/perf_baselines_musa.json
+564
View File
@@ -0,0 +1,564 @@
{
"metadata": {
"model": "Diffusion Server",
"hardware": "CI S5000 pool",
"description": "Reference numbers captured from the CI diffusion server baseline run"
},
"scenarios": {
"zimage_image_t2i_musa": {
"stages_ms": {
"InputValidationStage": 0.08,
"TextEncodingStage": 3761.4,
"LatentPreparationStage": 4.06,
"TimestepPreparationStage": 21.94,
"DenoisingStage": 4873.8,
"DecodingStage": 252.49
},
"denoise_step_ms": {
"0": 3281.35,
"2": 209.69,
"3": 199.26,
"5": 200.42,
"6": 207.8,
"8": 205.19
},
"expected_e2e_ms": 12161.85,
"expected_avg_denoise_ms": 540.67,
"expected_median_denoise_ms": 205.19,
"estimated_full_test_time_s": 64.6
},
"qwen_image_layered_i2i_musa": {
"stages_ms": {
"QwenImageLayeredBeforeDenoisingStage": 287.12,
"TimestepPreparationStage": 0.01,
"DenoisingStage": 79962.68,
"DecodingStage": 232.86
},
"denoise_step_ms": {
"0": 1377.49,
"1": 1608.67,
"2": 1606.57,
"3": 1601.06,
"4": 1606.65,
"5": 1605.0,
"6": 1596.31,
"7": 1609.13,
"8": 1599.48,
"9": 1601.81,
"10": 1606.7,
"11": 1601.7,
"12": 1607.68,
"13": 1606.42,
"14": 1599.13,
"15": 1611.78,
"16": 1598.31,
"17": 1600.97,
"18": 1611.38,
"19": 1598.6,
"20": 1599.72,
"21": 1608.24,
"22": 1599.4,
"23": 1613.52,
"24": 1600.56,
"25": 1605.91,
"26": 1605.47,
"27": 1598.85,
"28": 1607.93,
"29": 1603.65,
"30": 1598.64,
"31": 1607.1,
"32": 1595.55,
"33": 1608.09,
"34": 1606.47,
"35": 1596.18,
"36": 1599.88,
"37": 1607.12,
"38": 1595.63,
"39": 1612.07,
"40": 1596.19,
"41": 1602.35,
"42": 1604.17,
"43": 1598.26,
"44": 1602.67,
"45": 1611.54,
"46": 1599.01,
"47": 1619.72,
"48": 1593.45,
"49": 1602.33
},
"expected_e2e_ms": 80490.53,
"expected_avg_denoise_ms": 1599.09,
"expected_median_denoise_ms": 1602.51,
"estimated_full_test_time_s": 159.4
},
"fast_hunyuan_video_musa": {
"stages_ms": {
"InputValidationStage": 0.08,
"TextEncodingStage": 4146.14,
"TimestepPreparationStage": 18.33,
"LatentPreparationStage": 2.42,
"DenoisingStage": 17236.59,
"DecodingStage": 7019.29
},
"denoise_step_ms": {
"0": 5462.17,
"1": 2338.96,
"2": 2355.67,
"3": 2353.45,
"4": 2359.64,
"5": 2363.04
},
"expected_e2e_ms": 32799.37,
"expected_avg_denoise_ms": 2872.16,
"expected_median_denoise_ms": 2357.66,
"estimated_full_test_time_s": 111.3
},
"qwen_image_t2i_musa": {
"stages_ms": {
"InputValidationStage": 0.09,
"TextEncodingStage": 658.4,
"LatentPreparationStage": 0.33,
"TimestepPreparationStage": 24.39,
"DenoisingStage": 36196.6,
"DecodingStage": 40.44
},
"denoise_step_ms": {
"0": 666.68,
"1": 732.33,
"2": 721.29,
"3": 729.27,
"4": 725.05,
"5": 721.71,
"6": 722.22,
"7": 725.93,
"8": 724.94,
"9": 724.14,
"10": 730.43,
"11": 719.92,
"12": 726.24,
"13": 722.04,
"14": 727.68,
"15": 720.31,
"16": 721.75,
"17": 725.65,
"18": 720.23,
"19": 724.12,
"20": 726.35,
"21": 723.27,
"22": 731.58,
"23": 724.97,
"24": 721.48,
"25": 722.0,
"26": 722.37,
"27": 719.81,
"28": 721.64,
"29": 724.81,
"30": 723.9,
"31": 725.42,
"32": 719.86,
"33": 728.04,
"34": 728.55,
"35": 723.13,
"36": 722.0,
"37": 730.11,
"38": 724.06,
"39": 728.35,
"40": 728.04,
"41": 726.62,
"42": 728.47,
"43": 728.11,
"44": 728.59,
"45": 721.5,
"46": 724.59,
"47": 729.26,
"48": 726.05,
"49": 721.13
},
"expected_e2e_ms": 37190.98,
"expected_avg_denoise_ms": 723.72,
"expected_median_denoise_ms": 724.7,
"estimated_full_test_time_s": 137.0
},
"qwen_image_2512_t2i_musa": {
"stages_ms": {
"InputValidationStage": 0.07,
"TextEncodingStage": 849.1,
"LatentPreparationStage": 0.26,
"TimestepPreparationStage": 21.95,
"DenoisingStage": 31935.97,
"DecodingStage": 98.62
},
"denoise_step_ms": {
"0": 553.87,
"1": 639.78,
"2": 640.39,
"3": 638.78,
"4": 640.13,
"5": 639.69,
"6": 640.19,
"7": 639.67,
"8": 638.95,
"9": 640.06,
"10": 639.8,
"11": 640.81,
"12": 638.7,
"13": 640.16,
"14": 639.76,
"15": 640.17,
"16": 639.26,
"17": 640.33,
"18": 639.91,
"19": 640.47,
"20": 640.1,
"21": 639.81,
"22": 639.05,
"23": 639.7,
"24": 640.02,
"25": 640.67,
"26": 639.04,
"27": 640.21,
"28": 639.78,
"29": 643.32,
"30": 640.2,
"31": 639.54,
"32": 640.14,
"33": 640.14,
"34": 639.54,
"35": 639.22,
"36": 639.94,
"37": 639.87,
"38": 639.41,
"39": 639.6,
"40": 638.5,
"41": 639.5,
"42": 639.35,
"43": 641.43,
"44": 638.73,
"45": 639.74,
"46": 639.72,
"47": 639.72,
"48": 638.64,
"49": 638.72
},
"expected_e2e_ms": 32915.75,
"expected_avg_denoise_ms": 638.55,
"expected_median_denoise_ms": 640.05,
"estimated_full_test_time_s": 137.8
},
"wan2_1_t2v_1.3b_musa": {
"stages_ms": {
"InputValidationStage": 0.06,
"TextEncodingStage": 1381.91,
"LatentPreparationStage": 0.18,
"TimestepPreparationStage": 3.99,
"DenoisingStage": 23679.86,
"DecodingStage": 1059.53,
"per_frame_generation": null
},
"denoise_step_ms": {
"0": 331.71,
"1": 478.93,
"2": 481.37,
"3": 483.42,
"4": 476.27,
"5": 488.73,
"6": 486.79,
"7": 467.89,
"8": 465.47,
"9": 472.81,
"10": 478.41,
"11": 488.84,
"12": 474.68,
"13": 468.48,
"14": 483.79,
"15": 476.05,
"16": 483.77,
"17": 476.04,
"18": 484.27,
"19": 486.43,
"20": 483.04,
"21": 473.76,
"22": 464.09,
"23": 474.14,
"24": 470.76,
"25": 487.99,
"26": 477.77,
"27": 465.76,
"28": 483.96,
"29": 484.14,
"30": 471.99,
"31": 483.41,
"32": 486.66,
"33": 467.07,
"34": 478.04,
"35": 476.9,
"36": 462.4,
"37": 476.53,
"38": 485.65,
"39": 478.89,
"40": 465.22,
"41": 472.03,
"42": 479.65,
"43": 479.74,
"44": 479.48,
"45": 474.19,
"46": 464.47,
"47": 463.09,
"48": 463.77,
"49": 463.49
},
"expected_e2e_ms": 26134.51,
"expected_avg_denoise_ms": 473.44,
"expected_median_denoise_ms": 476.72,
"estimated_full_test_time_s": 101.5
},
"wan2_2_t2v_a14b_2gpu_musa": {
"stages_ms": {
"per_frame_generation": null,
"TimestepPreparationStage": 4.8,
"DenoisingStage": 281769.53,
"DecodingStage": 1728.76,
"InputValidationStage": 0.13,
"TextEncodingStage": 1098.56,
"LatentPreparationStage": 0.33
},
"denoise_step_ms": {
"0": 6262.73,
"1": 7103.05,
"2": 7039.74,
"3": 7052.97,
"4": 7046.74,
"5": 7059.9,
"6": 7048.75,
"7": 7054.99,
"8": 7052.75,
"9": 7059.59,
"10": 7056.92,
"11": 7053.61,
"12": 7063.48,
"13": 7045.34,
"14": 7051.22,
"15": 7051.25,
"16": 7048.83,
"17": 7053.91,
"18": 7060.47,
"19": 7056.4,
"20": 7055.39,
"21": 7052.72,
"22": 7054.84,
"23": 7058.17,
"24": 7052.26,
"25": 7057.8,
"26": 7362.31,
"27": 7053.57,
"28": 7044.42,
"29": 7044.03,
"30": 7056.59,
"31": 7045.81,
"32": 7051.59,
"33": 7048.66,
"34": 7050.84,
"35": 7048.57,
"36": 7056.21,
"37": 7056.08,
"38": 7053.32,
"39": 7047.35
},
"expected_e2e_ms": 285727.55,
"expected_avg_denoise_ms": 7043.96,
"expected_median_denoise_ms": 7056.07
},
"wan2_1_i2v_14b_480P_2gpu_musa": {
"stages_ms": {
"InputValidationStage": 10.35,
"TextEncodingStage": 1098.55,
"ImageEncodingStage": 730.6,
"ImageVAEEncodingStage": 1188.4,
"LatentPreparationStage": 0.5,
"TimestepPreparationStage": 7.99,
"DenoisingStage": 134355.99,
"DecodingStage": 1200.0,
"per_frame_generation": null
},
"denoise_step_ms": {
"0": 2441.69,
"1": 2711.82,
"2": 2701.26,
"3": 2692.91,
"4": 2686.94,
"5": 2685.35,
"6": 2689.45,
"7": 2685.49,
"8": 2687.74,
"9": 2690.0,
"10": 2691.31,
"11": 2691.65,
"12": 2698.52,
"13": 2689.37,
"14": 2690.18,
"15": 2691.48,
"16": 2695.74,
"17": 2691.77,
"18": 2689.77,
"19": 2690.6,
"20": 2686.84,
"21": 2694.9,
"22": 2689.56,
"23": 2696.38,
"24": 2689.13,
"25": 2686.38,
"26": 2689.53,
"27": 2695.91,
"28": 2691.9,
"29": 2691.55,
"30": 2700.09,
"31": 2691.45,
"32": 2696.8,
"33": 2689.42,
"34": 2695.87,
"35": 2690.31,
"36": 2687.65,
"37": 2697.62,
"38": 2683.46,
"39": 2692.41,
"40": 2699.33,
"41": 2695.25,
"42": 2691.16,
"43": 2687.13,
"44": 2692.92,
"45": 2690.4,
"46": 2696.53,
"47": 2689.51,
"48": 2692.06,
"49": 2680.93
},
"expected_e2e_ms": 138624.98,
"expected_avg_denoise_ms": 2686.91,
"expected_median_denoise_ms": 2691.24
},
"qwen_image_edit_t2i_musa": {
"stages_ms": {
"InputValidationStage": 35.9,
"ImageEncodingStage": 7483.38,
"ImageVAEEncodingStage": 160.82,
"LatentPreparationStage": 3.0,
"TimestepPreparationStage": 19.02,
"DenoisingStage": 84367.86,
"DecodingStage": 36.82
},
"denoise_step_ms": {
"0": 7612.62,
"1": 1567.7,
"2": 1559.81,
"3": 1570.75,
"4": 1568.7,
"5": 1572.44,
"6": 1565.71,
"7": 1559.33,
"8": 1567.63,
"9": 1566.83,
"10": 1560.77,
"11": 1561.03,
"12": 1565.61,
"13": 1560.43,
"14": 1562.47,
"15": 1562.22,
"16": 1564.1,
"17": 1571.28,
"18": 1563.26,
"19": 1561.37,
"20": 1559.04,
"21": 1556.07,
"22": 1577.83,
"23": 1564.54,
"24": 1564.28,
"25": 1573.79,
"26": 1572.94,
"27": 1568.81,
"28": 1568.73,
"29": 1571.71,
"30": 1557.83,
"31": 1568.7,
"32": 1570.3,
"33": 1567.36,
"34": 1566.47,
"35": 1567.2,
"36": 1560.98,
"37": 1563.43,
"38": 1570.74,
"39": 1568.01,
"40": 1560.57,
"41": 1572.64,
"42": 1564.01,
"43": 1566.34,
"44": 1569.09,
"45": 1573.18,
"46": 1566.5,
"47": 1567.04,
"48": 1570.02,
"49": 1559.91
},
"expected_e2e_ms": 92355.9,
"expected_avg_denoise_ms": 1687.04,
"expected_median_denoise_ms": 1566.67,
"estimated_full_test_time_s": 217.0
},
"qwen_image_edit_2509_ti2i_musa": {
"stages_ms": {
"InputValidationStage": 125.7,
"ImageEncodingStage": 1018.93,
"ImageVAEEncodingStage": 311.41,
"LatentPreparationStage": 0.24,
"TimestepPreparationStage": 33.39,
"DenoisingStage": 88792.03,
"DecodingStage": 320.64
},
"denoise_step_ms": {
"0": 1914.75,
"1": 2230.29,
"2": 2216.93,
"3": 2231.22,
"4": 2230.63,
"5": 2222.13,
"6": 2224.43,
"7": 2235.47,
"8": 2220.55,
"9": 2239.83,
"10": 2239.29,
"11": 2216.95,
"12": 2221.39,
"13": 2229.65,
"14": 2231.94,
"15": 2222.23,
"16": 2230.03,
"17": 2236.55,
"18": 2217.18,
"19": 2231.48,
"20": 2236.88,
"21": 2226.74,
"22": 2224.26,
"23": 2231.1,
"24": 2214.29,
"25": 2224.57,
"26": 2233.64,
"27": 2217.0,
"28": 2226.08,
"29": 2229.47,
"30": 2230.21,
"31": 2224.45,
"32": 2230.89,
"33": 2232.82,
"34": 2219.97,
"35": 2228.74,
"36": 2231.6,
"37": 2225.25,
"38": 2223.72,
"39": 2228.41
},
"expected_e2e_ms": 90612.84,
"expected_avg_denoise_ms": 2219.58,
"expected_median_denoise_ms": 2228.58,
"estimated_full_test_time_s": 220.8
}
}
}
python/sglang/multimodal_gen/test/server/musa/run_suite.py
+172
View File
@@ -0,0 +1,172 @@
"""
Test runner for multimodal_gen MUSA suites that manages partitioned execution.
Usage:
python3 -m sglang.multimodal_gen.test.server.musa.run_suite --suite <suite_name>
"""
from __future__ import annotations
import argparse
import os
import sys
from pathlib import Path
import tabulate
TEST_ROOT = Path(__file__).resolve().parents[2]
if str(TEST_ROOT) not in sys.path:
sys.path.insert(0, str(TEST_ROOT))
from runner.pytest_runner import ( # noqa: E402
collect_test_items,
partition_items_by_index,
run_pytest,
)
SUITES = {
"1-gpu-musa": [
"test_server_1_gpu_musa.py",
],
"1-gpu-musa-nightly": [
"test_server_1_gpu_musa_nightly.py",
],
"2-gpu-musa": [
"test_server_2_gpu_musa.py",
],
}
def parse_args():
parser = argparse.ArgumentParser(description="Run multimodal_gen MUSA test suite")
parser.add_argument(
"--suite",
type=str,
required=True,
choices=list(SUITES.keys()),
help="The test suite to run (valid names are defined in SUITES)",
)
parser.add_argument(
"--partition-id",
type=int,
default=0,
help="Index of the current partition (for parallel execution)",
)
parser.add_argument(
"--total-partitions",
type=int,
default=1,
help="Total number of partitions",
)
parser.add_argument(
"--base-dir",
type=str,
default=None,
help=(
"Base directory for tests relative to multimodal_gen/test. "
"Defaults to server/musa."
),
)
parser.add_argument(
"-k",
"--filter",
type=str,
default=None,
help="Pytest filter expression (passed to pytest -k)",
)
parser.add_argument(
"--continue-on-error",
action="store_true",
default=False,
help="Continue running remaining tests even if one fails.",
)
return parser.parse_args()
def _resolve_suite_files(
suite: str, test_root_dir: Path, musa_dir: Path, base_dir: str | None
) -> tuple[Path, list[str]]:
target_dir = test_root_dir / base_dir if base_dir else musa_dir
suite_files_rel = SUITES[suite]
if target_dir == musa_dir:
return target_dir, suite_files_rel
musa_rel = musa_dir.relative_to(target_dir)
return target_dir, [str(musa_rel / rel_path) for rel_path in suite_files_rel]
def main():
args = parse_args()
musa_dir = Path(__file__).resolve().parent
test_root_dir = musa_dir.parent.parent
target_dir, suite_files_rel = _resolve_suite_files(
args.suite, test_root_dir, musa_dir, args.base_dir
)
if not target_dir.exists():
print(f"Error: Target directory {target_dir} does not exist.")
sys.exit(1)
suite_files_abs = []
for rel_path in suite_files_rel:
abs_path = target_dir / rel_path
if not abs_path.exists():
print(f"Warning: Test file {rel_path} not found in {target_dir}. Skipping.")
continue
suite_files_abs.append(str(abs_path))
if not suite_files_abs:
print(f"No valid test files found for suite '{args.suite}'.")
sys.exit(0)
all_test_items = collect_test_items(suite_files_abs, filter_expr=args.filter)
if not all_test_items:
print(f"No test items found for suite '{args.suite}'.")
sys.exit(0)
my_items = partition_items_by_index(
all_test_items, args.partition_id, args.total_partitions
)
partition_info = (
f"{args.partition_id + 1}/{args.total_partitions} "
f"(0-based id={args.partition_id})"
)
rows = [[args.suite, partition_info]]
msg = (
tabulate.tabulate(rows, headers=["Suite", "Partition"], tablefmt="psql") + "\n"
)
msg += f"Enabled {len(my_items)} test(s):\n"
for item in my_items:
msg += f" - {item}\n"
print(msg, flush=True)
print(
f"Suite: {args.suite} | Partition: {args.partition_id}/{args.total_partitions}"
)
print(f"Selected {len(suite_files_abs)} files:")
for file_path in suite_files_abs:
print(f" - {os.path.basename(file_path)}")
if not my_items:
print("No items assigned to this partition. Exiting success.")
sys.exit(0)
print(f"Running {len(my_items)} items in this shard: {', '.join(my_items)}")
exit_code, _, _ = run_pytest(my_items, exitfirst=not args.continue_on_error)
msg = (
"\n"
+ tabulate.tabulate(rows, headers=["Suite", "Partition"], tablefmt="psql")
+ "\n"
)
msg += f"Executed {len(my_items)} test(s):\n"
for item in my_items:
msg += f" - {item}\n"
print(msg, flush=True)
sys.exit(exit_code)
if __name__ == "__main__":
main()
python/sglang/multimodal_gen/test/server/musa/test_server_1_gpu_musa.py
+22
View File
@@ -0,0 +1,22 @@
"""
MUSA-specific 1-GPU diffusion performance tests.
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.musa.testcase_configs_musa import (
ONE_GPU_MUSA_CASES,
)
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneGpuMusa(DiffusionServerBase):
"""Performance tests for 1-GPU diffusion cases on MUSA."""
case = diffusion_case_fixture(ONE_GPU_MUSA_CASES)
python/sglang/multimodal_gen/test/server/musa/test_server_1_gpu_musa_nightly.py
+22
View File
@@ -0,0 +1,22 @@
"""
MUSA-specific 1-GPU diffusion performance tests for nightly suite.
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.musa.testcase_configs_musa import (
ONE_GPU_NIGHTLY_MUSA_CASES,
)
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneGpuMusaNightly(DiffusionServerBase):
"""Performance tests for 1-GPU diffusion cases on MUSA (nightly-only)."""
case = diffusion_case_fixture(ONE_GPU_NIGHTLY_MUSA_CASES)
python/sglang/multimodal_gen/test/server/musa/test_server_2_gpu_musa.py
+22
View File
@@ -0,0 +1,22 @@
"""
MUSA-specific 2-GPU diffusion performance tests.
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.musa.testcase_configs_musa import (
TWO_GPU_MUSA_CASES,
)
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerTwoGpuMusa(DiffusionServerBase):
"""Performance tests for 2-GPU diffusion cases on MUSA."""
case = diffusion_case_fixture(TWO_GPU_MUSA_CASES)
python/sglang/multimodal_gen/test/server/musa/testcase_configs_musa.py
+134
View File
@@ -0,0 +1,134 @@
from __future__ import annotations
from dataclasses import replace
from functools import lru_cache
from sglang.multimodal_gen.test.server.testcase_configs import (
T2V_PROMPT,
DiffusionSamplingParams,
DiffusionServerArgs,
DiffusionTestCase,
MULTI_FRAME_I2I_sampling_params,
MULTI_IMAGE_TI2I_sampling_params,
T2I_sampling_params,
T2V_sampling_params,
TI2I_sampling_params,
TI2V_sampling_params,
)
@lru_cache(maxsize=None)
def hf_cached_model(repo_id: str) -> str:
"""Resolve an HF repo id to the local cache snapshot prepared on MUSA runners."""
from huggingface_hub import snapshot_download
return snapshot_download(repo_id, local_files_only=True)
MUSA_TI2I_sampling_params = replace(
TI2I_sampling_params,
image_path="/hf-cache/hub/musa-test-assets/TI2I_Qwen_Image_Edit_Input.jpg",
)
ONE_GPU_MUSA_CASES: list[DiffusionTestCase] = [
DiffusionTestCase(
"qwen_image_t2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Qwen/Qwen-Image"),
modality="image",
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"wan2_1_t2v_1.3b_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Wan-AI/Wan2.1-T2V-1.3B-Diffusers"),
modality="video",
custom_validator="video",
),
DiffusionSamplingParams(
prompt=T2V_PROMPT,
),
run_consistency_check=False,
),
]
NIGHTLY_1_GPU_MUSA_CASES: list[DiffusionTestCase] = [
DiffusionTestCase(
"zimage_image_t2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Tongyi-MAI/Z-Image-Turbo"),
modality="image",
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"qwen_image_layered_i2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Qwen/Qwen-Image-Layered"),
modality="image",
),
MULTI_FRAME_I2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"fast_hunyuan_video_musa",
DiffusionServerArgs(
model_path=hf_cached_model("FastVideo/FastHunyuan-diffusers"),
modality="video",
custom_validator="video",
),
T2V_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"qwen_image_2512_t2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Qwen/Qwen-Image-2512"),
modality="image",
),
T2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"qwen_image_edit_t2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Qwen/Qwen-Image-Edit"),
modality="image",
),
MUSA_TI2I_sampling_params,
run_consistency_check=False,
),
DiffusionTestCase(
"qwen_image_edit_2509_ti2i_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Qwen/Qwen-Image-Edit-2509"),
modality="image",
),
MULTI_IMAGE_TI2I_sampling_params,
run_consistency_check=False,
),
]
ONE_GPU_NIGHTLY_MUSA_CASES: list[DiffusionTestCase] = (
ONE_GPU_MUSA_CASES + NIGHTLY_1_GPU_MUSA_CASES
)
TWO_GPU_MUSA_CASES: list[DiffusionTestCase] = [
DiffusionTestCase(
"wan2_1_i2v_14b_480P_2gpu_musa",
DiffusionServerArgs(
model_path=hf_cached_model("Wan-AI/Wan2.1-I2V-14B-480P-Diffusers"),
modality="video",
custom_validator="video",
num_gpus=2,
),
TI2V_sampling_params,
run_consistency_check=False,
),
]
python/sglang/multimodal_gen/test/server/perf_baselines/5090.json
+106
View File
@@ -0,0 +1,106 @@
{
"metadata": {
"model": "Diffusion Server",
"hardware": "CI RTX 5090 pool",
"description": "Reference numbers captured from real 5090 CI runner history (PR 29791 run 28492141274, job 84450974283).",
"last_updated": "2026-07-01"
},
"tolerances": {
"long_term": {
"e2e": 0.15,
"denoise_stage": 0.1,
"non_denoise_stage": 0.5,
"denoise_step": 0.25,
"denoise_agg": 0.15
},
"pr_test": {
"e2e": 0.25,
"denoise_stage": 0.25,
"non_denoise_stage": 0.8,
"denoise_step": 0.3,
"denoise_agg": 0.2
}
},
"improvement_reporting": {
"threshold": 0.2
},
"sampling": {
"step_fractions": [
0.0,
0.2,
0.4,
0.6,
0.8,
1.0
]
},
"scenarios": {
"flux_2_klein_base_image_t2i": {
"stages_ms": {
"DecodingStage": 18.19,
"DenoisingStage": 18213.48,
"ImageVAEEncodingStage": 0.01,
"InputValidationStage": 0.08,
"LatentPreparationStage": 4.87,
"TextEncodingStage": 104.46,
"TimestepPreparationStage": 142.18
},
"denoise_step_ms": {
"0": 219.58,
"10": 359.79,
"20": 361.55,
"29": 361.99,
"39": 362.57,
"49": 362.89
},
"expected_e2e_ms": 18641.09,
"expected_avg_denoise_ms": 357.93,
"expected_median_denoise_ms": 361.8,
"estimated_full_test_time_s": 94.0
},
"wan2_1_t2v_1.3b": {
"stages_ms": {
"DecodingStage": 1202.43,
"DenoisingStage": 21452.59,
"InputValidationStage": 0.09,
"LatentPreparationStage": 0.23,
"TextEncodingStage": 1210.22,
"TimestepPreparationStage": 3.91
},
"denoise_step_ms": {
"0": 480.39,
"10": 426.11,
"20": 427.32,
"29": 428.09,
"39": 428.04,
"49": 422.31
},
"expected_e2e_ms": 23877.73,
"expected_avg_denoise_ms": 428.74,
"expected_median_denoise_ms": 427.94,
"estimated_full_test_time_s": 160.9
},
"zimage_image_t2i": {
"stages_ms": {
"DecodingStage": 7.11,
"DenoisingStage": 2229.48,
"InputValidationStage": 0.04,
"LatentPreparationStage": 0.17,
"TextEncodingStage": 252.73,
"TimestepPreparationStage": 38.57
},
"denoise_step_ms": {
"0": 39.34,
"2": 277.35,
"3": 267.94,
"5": 279.01,
"6": 273.01,
"8": 275.12
},
"expected_e2e_ms": 2533.95,
"expected_avg_denoise_ms": 246.97,
"expected_median_denoise_ms": 273.01,
"estimated_full_test_time_s": 329.8
}
}
}
python/sglang/multimodal_gen/test/server/perf_baselines/b200.json
+79
View File
@@ -0,0 +1,79 @@
{
"metadata": {
"model": "Diffusion Server",
"hardware": "CI B200 pool",
"description": "Reference estimates for B200-only diffusion cases, split out from the shared diffusion baseline file.",
"last_updated": "2026-07-01"
},
"tolerances": {
"long_term": {
"e2e": 0.15,
"denoise_stage": 0.1,
"non_denoise_stage": 0.5,
"denoise_step": 0.25,
"denoise_agg": 0.15
},
"pr_test": {
"e2e": 0.25,
"denoise_stage": 0.25,
"non_denoise_stage": 0.8,
"denoise_step": 0.3,
"denoise_agg": 0.2
}
},
"improvement_reporting": {
"threshold": 0.2
},
"sampling": {
"step_fractions": [
0.0,
0.2,
0.4,
0.6,
0.8,
1.0
]
},
"scenarios": {
"flux1_modelopt_nvfp4_t2i": {
"stages_ms": {},
"denoise_step_ms": {},
"expected_e2e_ms": 0.0,
"expected_avg_denoise_ms": 0.0,
"expected_median_denoise_ms": 0.0,
"estimated_full_test_time_s": 71.2
},
"flux2_modelopt_nvfp4_t2i": {
"stages_ms": {},
"denoise_step_ms": {},
"expected_e2e_ms": 0.0,
"expected_avg_denoise_ms": 0.0,
"expected_median_denoise_ms": 0.0,
"estimated_full_test_time_s": 592.3
},
"qwen_image_2512_modelopt_nvfp4_t2i": {
"stages_ms": {},
"denoise_step_ms": {},
"expected_e2e_ms": 0.0,
"expected_avg_denoise_ms": 0.0,
"expected_median_denoise_ms": 0.0,
"estimated_full_test_time_s": 120.0
},
"wan22_modelopt_nvfp4_t2v": {
"stages_ms": {},
"denoise_step_ms": {},
"expected_e2e_ms": 0.0,
"expected_avg_denoise_ms": 0.0,
"expected_median_denoise_ms": 0.0,
"estimated_full_test_time_s": 181.8
},
"ideogram4_nvfp4_t2i": {
"stages_ms": {},
"denoise_step_ms": {},
"expected_e2e_ms": 0.0,
"expected_avg_denoise_ms": 0.0,
"expected_median_denoise_ms": 0.0,
"estimated_full_test_time_s": 300.0
}
}
}
python/sglang/multimodal_gen/test/server/perf_baselines/h100.json
+2834
View File
File diff suppressed because it is too large Load Diff
python/sglang/multimodal_gen/test/server/realtime_consistency.py
+403
View File
@@ -0,0 +1,403 @@
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
import asyncio
import os
import statistics
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any
import imageio
import msgspec.msgpack
import numpy as np
import pytest
from openai import Client
from sglang.multimodal_gen.runtime.utils.realtime_video import (
RAW_RGB_CONTENT_TYPE,
RAW_RGB_DELTA_GZIP_CONTENT_TYPE,
RAW_RGBA_DELTA_GZIP_CONTENT_TYPE,
restore_delta_gzip_raw_rgb_payload,
)
from sglang.multimodal_gen.test.server.testcase_configs import DiffusionSamplingParams
from sglang.multimodal_gen.test.test_utils import is_image_url
_REALTIME_WS_TIMEOUT_SECS = float(
os.environ.get("SGLANG_TEST_REALTIME_WS_TIMEOUT_SECS", "1200")
)
@dataclass(frozen=True)
class RealtimeChunkStats:
chunk_index: int
request_id: str | None
content_type: str
num_frames: int
raw_bytes: int
ws_payload_bytes: int
request_prepare_ms: float
scheduler_forward_ms: float
raw_payload_build_ms: float
raw_write_ms: float
ws_write_ms: float
chunk_total_ms: float
@dataclass(frozen=True)
class RealtimeCollectionResult:
frames: list[np.ndarray]
chunk_stats: list[RealtimeChunkStats]
_REALTIME_CHUNK_STATS_BY_CASE: dict[str, list[RealtimeChunkStats]] = {}
_REALTIME_KEY_FRAMES_BY_CASE: dict[str, list[np.ndarray]] = {}
def realtime_ws_url(client: Client) -> str:
base_url = str(client.base_url).rstrip("/")
if base_url.startswith("https://"):
return "wss://" + base_url[len("https://") :] + "/realtime_video/generate"
if base_url.startswith("http://"):
return "ws://" + base_url[len("http://") :] + "/realtime_video/generate"
raise ValueError(f"Unsupported realtime client base_url: {base_url}")
def prepare_realtime_first_frame(
image_path: Path | str | list[Path | str] | None,
) -> bytes | str | None:
if image_path is None:
return None
if isinstance(image_path, list):
image_path = image_path[0]
if isinstance(image_path, str) and is_image_url(image_path):
return image_path
path = Path(image_path)
if not path.exists():
raise FileNotFoundError(f"Realtime first frame file missing: {path}")
return path.read_bytes()
def build_realtime_init_payload(
*,
model_path: str,
sampling_params: DiffusionSamplingParams,
output_size: str,
first_frame: bytes | str | None,
) -> dict[str, Any]:
payload: dict[str, Any] = {
"type": "init",
"model": model_path,
"prompt": sampling_params.prompt,
"size": output_size,
"seconds": sampling_params.seconds,
"first_frame": first_frame,
}
optional_fields = {
"fps": sampling_params.fps,
"num_frames": sampling_params.num_frames,
"realtime_output_format": sampling_params.realtime_output_format,
}
payload.update({k: v for k, v in optional_fields.items() if v is not None})
payload.update(dict(sampling_params.extras))
return {k: v for k, v in payload.items() if v is not None}
def build_realtime_event_payload(event: dict[str, Any]) -> dict[str, Any]:
payload = dict(event)
payload.pop("after_chunk", None)
payload.setdefault("type", "event")
if "kind" not in payload:
raise ValueError("realtime event config must include kind")
return payload
def parse_realtime_chunk_stats(header: dict[str, Any]) -> RealtimeChunkStats:
if header.get("type") != "chunk_stats":
raise ValueError(f"Unexpected realtime chunk stats message: {header}")
return RealtimeChunkStats(
chunk_index=int(header["chunk_index"]),
request_id=header.get("request_id"),
content_type=str(header.get("content_type", "")),
num_frames=int(header.get("num_frames", 0)),
raw_bytes=int(header.get("raw_bytes", 0)),
ws_payload_bytes=int(header.get("ws_payload_bytes", 0)),
request_prepare_ms=float(header.get("request_prepare_ms", 0.0)),
scheduler_forward_ms=float(header.get("scheduler_forward_ms", 0.0)),
raw_payload_build_ms=float(header.get("raw_payload_build_ms", 0.0)),
raw_write_ms=float(header.get("raw_write_ms", 0.0)),
ws_write_ms=float(header.get("ws_write_ms", 0.0)),
chunk_total_ms=float(header.get("chunk_total_ms", 0.0)),
)
def summarize_realtime_perf_stats(
chunk_stats: list[RealtimeChunkStats],
*,
ignore_initial_chunks: int = 0,
) -> dict[str, float]:
if not chunk_stats:
return {}
if ignore_initial_chunks < 0:
raise ValueError("ignore_initial_chunks must be non-negative")
if ignore_initial_chunks >= len(chunk_stats):
raise ValueError(
"ignore_initial_chunks must leave at least one realtime chunk to guard"
)
ignored_stats = chunk_stats[:ignore_initial_chunks]
guarded_stats = chunk_stats[ignore_initial_chunks:]
metrics = {
"request_prepare_ms": [s.request_prepare_ms for s in guarded_stats],
"scheduler_forward_ms": [s.scheduler_forward_ms for s in guarded_stats],
"raw_payload_build_ms": [s.raw_payload_build_ms for s in guarded_stats],
"raw_write_ms": [s.raw_write_ms for s in guarded_stats],
"ws_write_ms": [s.ws_write_ms for s in guarded_stats],
"chunk_total_ms": [s.chunk_total_ms for s in guarded_stats],
"ws_payload_mb": [s.ws_payload_bytes / (1024 * 1024) for s in guarded_stats],
}
summary: dict[str, float] = {
"num_chunks": float(len(chunk_stats)),
"total_frames": float(sum(s.num_frames for s in chunk_stats)),
"ignored_initial_chunks": float(ignore_initial_chunks),
"guarded_chunks": float(len(guarded_stats)),
}
if ignored_stats:
summary["ignored_max_chunk_total_ms"] = max(
s.chunk_total_ms for s in ignored_stats
)
summary["ignored_max_scheduler_forward_ms"] = max(
s.scheduler_forward_ms for s in ignored_stats
)
for name, values in metrics.items():
sorted_values = sorted(values)
p95_idx = min(len(sorted_values) - 1, int(len(sorted_values) * 0.95))
summary[f"avg_{name}"] = statistics.fmean(values)
summary[f"p95_{name}"] = sorted_values[p95_idx]
summary[f"max_{name}"] = max(values)
return summary
def validate_realtime_perf_stats(
case_id: str,
chunk_stats: list[RealtimeChunkStats],
thresholds: dict[str, float],
*,
ignore_initial_chunks: int = 0,
) -> None:
if not thresholds:
return
summary = summarize_realtime_perf_stats(
chunk_stats, ignore_initial_chunks=ignore_initial_chunks
)
if not summary:
pytest.fail(f"{case_id}: no realtime chunk stats were received")
failures = []
for metric_name, threshold in thresholds.items():
if metric_name not in summary:
raise ValueError(
f"{case_id}: unknown realtime perf metric {metric_name!r}; "
f"available metrics: {sorted(summary)}"
)
actual = summary[metric_name]
if actual > threshold:
failures.append(
f"{metric_name}: actual={actual:.2f}, limit={threshold:.2f}"
)
if failures:
pytest.fail(
f"Realtime performance guard failed for {case_id}:\n"
+ "\n".join(f" - {failure}" for failure in failures)
)
def record_realtime_perf_stats(
case_id: str, chunk_stats: list[RealtimeChunkStats]
) -> None:
_REALTIME_CHUNK_STATS_BY_CASE[case_id] = list(chunk_stats)
def pop_realtime_perf_stats(case_id: str) -> list[RealtimeChunkStats]:
return _REALTIME_CHUNK_STATS_BY_CASE.pop(case_id, [])
def select_realtime_key_frames(frames: list[np.ndarray]) -> list[np.ndarray]:
if not frames:
return []
key_indices = [0, len(frames) // 2, len(frames) - 1]
return [frames[idx].copy() for idx in key_indices]
def record_realtime_key_frames(case_id: str, frames: list[np.ndarray]) -> None:
_REALTIME_KEY_FRAMES_BY_CASE[case_id] = select_realtime_key_frames(frames)
def pop_realtime_key_frames(case_id: str) -> list[np.ndarray] | None:
return _REALTIME_KEY_FRAMES_BY_CASE.pop(case_id, None)
def decode_realtime_raw_rgb_frames(
header: dict[str, Any],
payload: bytes,
previous_frame: bytes | None = None,
) -> list[np.ndarray]:
content_type = header.get("content_type")
if content_type not in (
RAW_RGB_CONTENT_TYPE,
RAW_RGB_DELTA_GZIP_CONTENT_TYPE,
RAW_RGBA_DELTA_GZIP_CONTENT_TYPE,
):
raise ValueError(f"Unsupported realtime frame content type: {content_type}")
width = int(header["width"])
height = int(header["height"])
channels = int(header["channels"])
num_frames = int(header["num_frames"])
bytes_per_frame = int(header["bytes_per_frame"])
expected_size = num_frames * bytes_per_frame
if content_type in (
RAW_RGB_DELTA_GZIP_CONTENT_TYPE,
RAW_RGBA_DELTA_GZIP_CONTENT_TYPE,
):
if header.get("delta_reference") != "previous-frame":
previous_frame = None
payload = restore_delta_gzip_raw_rgb_payload(
payload,
bytes_per_frame=bytes_per_frame,
num_frames=num_frames,
reference_frame=previous_frame,
)
if len(payload) != expected_size:
raise ValueError(
f"Realtime payload size mismatch: expected {expected_size}, got {len(payload)}"
)
frames = []
for frame_idx in range(num_frames):
offset = frame_idx * bytes_per_frame
frame = np.frombuffer(
payload[offset : offset + bytes_per_frame], dtype=np.uint8
)
frame = frame.reshape(height, width, channels)
if channels > 3:
frame = frame[:, :, :3]
frames.append(frame.copy())
return frames
def encode_realtime_frames_to_mp4(frames: list[np.ndarray], fps: int) -> bytes:
if not frames:
raise ValueError("Cannot encode empty realtime frame list")
with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmp:
output_path = tmp.name
try:
imageio.mimsave(
output_path,
frames,
fps=fps,
format="mp4",
codec="libx264",
quality=5,
)
return Path(output_path).read_bytes()
finally:
try:
os.remove(output_path)
except OSError:
pass
async def collect_realtime_frames(
*,
ws_url: str,
init_payload: dict[str, Any],
events: list[dict[str, Any]],
num_chunks: int,
) -> list[np.ndarray]:
return (
await collect_realtime_output(
ws_url=ws_url,
init_payload=init_payload,
events=events,
num_chunks=num_chunks,
)
).frames
async def collect_realtime_output(
*,
ws_url: str,
init_payload: dict[str, Any],
events: list[dict[str, Any]],
num_chunks: int,
require_chunk_stats: bool = False,
) -> RealtimeCollectionResult:
try:
import websockets
except ImportError:
pytest.skip("websockets is required for realtime consistency checks")
frames: list[np.ndarray] = []
chunk_stats: list[RealtimeChunkStats] = []
sent_event_indices: set[int] = set()
async def send_events_for_boundary(ws, completed_chunk: int) -> None:
for event_idx, event in enumerate(events):
if event_idx in sent_event_indices:
continue
if int(event.get("after_chunk", 0)) != completed_chunk:
continue
await ws.send(msgspec.msgpack.encode(build_realtime_event_payload(event)))
sent_event_indices.add(event_idx)
async with websockets.connect(ws_url, max_size=None, ping_interval=None) as ws:
await ws.send(msgspec.msgpack.encode(init_payload))
await send_events_for_boundary(ws, -1)
received_chunks: set[int] = set()
previous_frame: bytes | None = None
while len(received_chunks) < num_chunks or (
require_chunk_stats and len(chunk_stats) < len(received_chunks)
):
header_payload = await asyncio.wait_for(
ws.recv(), timeout=_REALTIME_WS_TIMEOUT_SECS
)
header = msgspec.msgpack.decode(header_payload)
message_type = header.get("type")
if message_type == "error":
pytest.fail(f"Realtime generation failed: {header.get('content')}")
if message_type == "chunk_stats":
chunk_stats.append(parse_realtime_chunk_stats(header))
continue
if message_type == "frame_batch":
raw_payload = header.pop("payload", None)
header["type"] = "frame_batch_header"
elif message_type == "frame_batch_header":
raw_payload = await asyncio.wait_for(
ws.recv(), timeout=_REALTIME_WS_TIMEOUT_SECS
)
else:
raise ValueError(f"Unexpected realtime message: {header}")
if not isinstance(raw_payload, bytes):
raise ValueError("Realtime frame payload must be bytes")
chunk_frames = decode_realtime_raw_rgb_frames(
header,
raw_payload,
previous_frame,
)
frames.extend(chunk_frames)
if chunk_frames:
previous_frame = chunk_frames[-1].tobytes()
chunk_index = int(header["chunk_index"])
if header.get("is_final_frame_batch", True):
received_chunks.add(chunk_index)
await send_events_for_boundary(ws, chunk_index)
return RealtimeCollectionResult(frames=frames, chunk_stats=chunk_stats)
python/sglang/multimodal_gen/test/server/test_request_logger.py
+233
View File
@@ -0,0 +1,233 @@
"""
Test request logging for diffusion models.
Tests the --log-requests CLI flags for diffusion model serving,
verifying that request logs are correctly written to stdout and files.
"""
import json
import os
import shutil
import tempfile
import time
from pathlib import Path
import pytest
from openai import OpenAI
from sglang.multimodal_gen.test.server.test_server_utils import ServerManager
from sglang.multimodal_gen.test.test_utils import get_dynamic_server_port
# Test models and prompts
IMAGE_MODEL = "Efficient-Large-Model/Sana_600M_512px_diffusers"
IMAGE_PROMPT = "A beautiful sunset over mountains, oil painting style"
VIDEO_MODEL = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
VIDEO_PROMPT = "A cat playing with a ball"
# Timeout settings
BASE_TIMEOUT = float(os.environ.get("SGLANG_TEST_OPENAI_REQUEST_TIMEOUT_SECS", "600"))
POLL_INTERVAL = 1.0
def _start_server(model: str, log_format: str):
"""Start server with request logging enabled."""
temp_dir = tempfile.mkdtemp()
port = get_dynamic_server_port()
extra_args = (
f"--log-requests "
f"--log-requests-level 2 "
f"--log-requests-format {log_format} "
f"--log-requests-target stdout {temp_dir} "
f"--strict-ports"
)
wait_deadline = float(os.environ.get("SGLANG_TEST_WAIT_SECS", "1200"))
manager = ServerManager(
model=model,
port=port,
wait_deadline=wait_deadline,
extra_args=extra_args,
)
ctx = manager.start()
ctx.temp_dir = temp_dir
return ctx
def _cleanup_server(ctx):
"""Cleanup server and temp directory."""
ctx.cleanup()
shutil.rmtree(ctx.temp_dir, ignore_errors=True)
def _create_client(ctx) -> OpenAI:
"""Create OpenAI client for the server."""
return OpenAI(
api_key="test",
base_url=f"http://localhost:{ctx.port}/v1",
timeout=BASE_TIMEOUT,
)
def _wait_for_video_completion(client: OpenAI, video_id: str, timeout: float):
"""Poll video job until completion."""
deadline = time.time() + timeout
while time.time() < deadline:
page = client.videos.list()
item = next((v for v in page.data if v.id == video_id), None)
status = getattr(item, "status", None) if item else None
if status == "completed":
return True
if status in ("failed", "cancelled", "deleted"):
pytest.fail(f"Video job {video_id} ended with status={status}")
time.sleep(POLL_INTERVAL)
pytest.fail(f"Video job {video_id} did not complete in {timeout}s")
def _verify_json_logs(content: str):
"""Verify JSON logs contain request.received and request.finished events."""
has_received = False
has_finished = False
for line in content.splitlines():
idx = line.find("{")
if idx == -1:
continue
try:
data = json.loads(line[idx:])
except json.JSONDecodeError:
continue
if data.get("event") == "request.received":
has_received = True
elif data.get("event") == "request.finished":
has_finished = True
assert has_received, "request.received event not found"
assert has_finished, "request.finished event not found"
def _verify_text_logs(content: str, prompt: str):
"""Verify text logs contain Receive, prompt, and Finish markers."""
assert "Receive:" in content, "'Receive:' not found"
assert prompt in content, f"Prompt '{prompt}' not found"
assert "Finish:" in content, "'Finish:' not found"
@pytest.fixture(scope="class")
def image_text_server():
"""Server with text-format logging for image model."""
ctx = _start_server(IMAGE_MODEL, "text")
yield ctx
_cleanup_server(ctx)
@pytest.fixture(scope="class")
def image_json_server():
"""Server with JSON-format logging for image model."""
ctx = _start_server(IMAGE_MODEL, "json")
yield ctx
_cleanup_server(ctx)
@pytest.fixture(scope="class")
def video_text_server():
"""Server with text-format logging for video model."""
ctx = _start_server(VIDEO_MODEL, "text")
yield ctx
_cleanup_server(ctx)
@pytest.fixture(scope="class")
def video_json_server():
"""Server with JSON-format logging for video model."""
ctx = _start_server(VIDEO_MODEL, "json")
yield ctx
_cleanup_server(ctx)
class TestImageRequestLoggerText:
"""Test text-format request logging for image models."""
def test_request_logging(self, image_text_server):
ctx = image_text_server
client = _create_client(ctx)
# Image generation is synchronous, waits for completion
client.images.generate(prompt=IMAGE_PROMPT, size="256x256", n=1)
# Verify stdout and file logs
stdout = ctx.log_tail(lines=500)
_verify_text_logs(stdout, IMAGE_PROMPT[:30])
logs = list(Path(ctx.temp_dir).glob("*.log"))
assert logs, "No log files found"
_verify_text_logs("".join(f.read_text() for f in logs), IMAGE_PROMPT[:30])
class TestImageRequestLoggerJson:
"""Test JSON-format request logging for image models."""
def test_request_logging(self, image_json_server):
ctx = image_json_server
client = _create_client(ctx)
# Image generation is synchronous, waits for completion
client.images.generate(prompt=IMAGE_PROMPT, size="256x256", n=1)
# Verify stdout and file logs
stdout = ctx.log_tail(lines=500)
_verify_json_logs(stdout)
logs = list(Path(ctx.temp_dir).glob("*.log"))
assert logs, "No log files found"
_verify_json_logs("".join(f.read_text() for f in logs))
class TestVideoRequestLoggerText:
"""Test text-format request logging for video models."""
def test_request_logging(self, video_text_server):
ctx = video_text_server
client = _create_client(ctx)
# Video generation is async - create job and poll until completion
job = client.videos.create(
prompt=VIDEO_PROMPT,
size="832x480",
extra_body={"num_frames": 5, "num_inference_steps": 10},
)
_wait_for_video_completion(client, job.id, BASE_TIMEOUT * 2)
# Verify stdout and file logs
stdout = ctx.log_tail(lines=500)
_verify_text_logs(stdout, VIDEO_PROMPT[:20])
logs = list(Path(ctx.temp_dir).glob("*.log"))
assert logs, "No log files found"
_verify_text_logs("".join(f.read_text() for f in logs), VIDEO_PROMPT[:20])
class TestVideoRequestLoggerJson:
"""Test JSON-format request logging for video models."""
def test_request_logging(self, video_json_server):
ctx = video_json_server
client = _create_client(ctx)
# Video generation is async - create job and poll until completion
job = client.videos.create(
prompt=VIDEO_PROMPT,
size="832x480",
extra_body={"num_frames": 5, "num_inference_steps": 10},
)
_wait_for_video_completion(client, job.id, BASE_TIMEOUT * 2)
# Verify stdout and file logs
stdout = ctx.log_tail(lines=500)
_verify_json_logs(stdout)
logs = list(Path(ctx.temp_dir).glob("*.log"))
assert logs, "No log files found"
_verify_json_logs("".join(f.read_text() for f in logs))
python/sglang/multimodal_gen/test/server/test_server_1_gpu.py
+23
View File
@@ -0,0 +1,23 @@
"""
Config-driven diffusion performance test with pytest parametrization.
If the actual run is significantly better than the baseline, the improved cases with their updated baseline will be printed
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.gpu_cases import ONE_GPU_CASES
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneGpu(DiffusionServerBase):
"""Performance tests for 1-GPU diffusion cases."""
case = diffusion_case_fixture(ONE_GPU_CASES)
python/sglang/multimodal_gen/test/server/test_server_1_gpu_5090.py
+20
View File
@@ -0,0 +1,20 @@
"""
Config-driven diffusion canary tests for the 1-GPU 5090 PR runner.
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.gpu_cases import ONE_GPU_5090_CASES
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneGpu5090(DiffusionServerBase):
"""Canary tests for lightweight 1-GPU diffusion cases on 5090."""
case = diffusion_case_fixture(ONE_GPU_5090_CASES)
python/sglang/multimodal_gen/test/server/test_server_2_gpu.py
+20
View File
@@ -0,0 +1,20 @@
"""
2 GPU tests
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.gpu_cases import TWO_GPU_CASES
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerTwoGpu(DiffusionServerBase):
"""Performance tests for 2-GPU diffusion cases."""
case = diffusion_case_fixture(TWO_GPU_CASES)
python/sglang/multimodal_gen/test/server/test_server_b200.py
+20
View File
@@ -0,0 +1,20 @@
"""
Config-driven diffusion performance test with pytest parametrization.
"""
from __future__ import annotations
from sglang.multimodal_gen.test.server.common.case_fixtures import (
diffusion_case_fixture,
)
from sglang.multimodal_gen.test.server.gpu_cases import ONE_GPU_B200_CASES
from sglang.multimodal_gen.test.server.test_server_common import ( # noqa: F401
DiffusionServerBase,
diffusion_server,
)
class TestDiffusionServerOneGpuB200(DiffusionServerBase):
"""B200-targeted CI tests for 1-GPU Blackwell-only diffusion cases."""
case = diffusion_case_fixture(ONE_GPU_B200_CASES)
python/sglang/multimodal_gen/test/server/test_server_common.py
+1458
View File
File diff suppressed because it is too large Load Diff
python/sglang/multimodal_gen/test/server/test_server_utils.py
+1628
View File
File diff suppressed because it is too large Load Diff
python/sglang/multimodal_gen/test/server/testcase_configs.py
+790
View File
@@ -0,0 +1,790 @@
"""
Configuration and data structures for diffusion performance tests.
Usage:
pytest python/sglang/multimodal_gen/test/server/test_server_1_gpu.py
# for a single testcase, look for the name of the testcase in ONE_GPU_CASES,
# ONE_GPU_MODELOPT_FP8_CASES, ONE_GPU_B200_CASES, or TWO_GPU_CASES
pytest python/sglang/multimodal_gen/test/server/test_server_1_gpu.py -k qwen_image_t2i
To add a new testcase:
1. add your testcase with case-id: `my_new_test_case_id` to `ONE_GPU_CASES`, `ONE_GPU_MODELOPT_FP8_CASES`, `ONE_GPU_B200_CASES`, or `TWO_GPU_CASES`
2. run `SGLANG_GEN_BASELINE=1 pytest -s python/sglang/multimodal_gen/test/server/ -k my_new_test_case_id`
3. insert or override the corresponding scenario in the platform JSON under `perf_baselines/`
"""
from __future__ import annotations
import json
import os
import shlex
import statistics
from dataclasses import dataclass, field, replace
from functools import lru_cache
from pathlib import Path
from typing import Any, Sequence
from sglang.multimodal_gen.configs.pipeline_configs.base import ModelTaskType
from sglang.multimodal_gen.registry import (
get_model_info,
get_pipeline_config_classes,
)
from sglang.multimodal_gen.runtime.platforms import current_platform
from sglang.multimodal_gen.runtime.utils.perf_logger import RequestPerfRecord
@dataclass
class ToleranceConfig:
"""Tolerance ratios for performance validation."""
e2e: float
denoise_stage: float
non_denoise_stage: float
denoise_step: float
denoise_agg: float
@classmethod
def load_profile(cls, all_tolerances: dict, profile_name: str) -> ToleranceConfig:
"""Load a specific tolerance profile from a dictionary of profiles."""
# Support both flat structure (backward compatibility) and profiled structure
if "e2e" in all_tolerances and not isinstance(all_tolerances["e2e"], dict):
tol_data = all_tolerances
actual_profile = "legacy/flat"
else:
tol_data = all_tolerances.get(
profile_name, all_tolerances.get("pr_test", {})
)
actual_profile = (
profile_name if profile_name in all_tolerances else "pr_test"
)
if not tol_data:
raise ValueError(
f"No tolerance profile found for '{profile_name}' and no default 'pr_test' profile exists."
)
print(f"--- Performance Tolerance Profile: {actual_profile} ---")
return cls(
e2e=float(os.getenv("SGLANG_E2E_TOLERANCE", tol_data["e2e"])),
denoise_stage=float(
os.getenv("SGLANG_STAGE_TIME_TOLERANCE", tol_data["denoise_stage"])
),
non_denoise_stage=float(
os.getenv(
"SGLANG_NON_DENOISE_STAGE_TIME_TOLERANCE",
tol_data["non_denoise_stage"],
)
),
denoise_step=float(
os.getenv("SGLANG_DENOISE_STEP_TOLERANCE", tol_data["denoise_step"])
),
denoise_agg=float(
os.getenv("SGLANG_DENOISE_AGG_TOLERANCE", tol_data["denoise_agg"])
),
)
@dataclass
class ScenarioConfig:
"""Expected performance metrics for a test scenario."""
stages_ms: dict[str, float]
denoise_step_ms: dict[int, float]
expected_e2e_ms: float
expected_avg_denoise_ms: float
expected_median_denoise_ms: float
estimated_full_test_time_s: float | None = None
@dataclass
class BaselineConfig:
"""Full baseline configuration."""
scenarios: dict[str, ScenarioConfig]
step_fractions: Sequence[float]
tolerances: ToleranceConfig
improvement_threshold: float
@classmethod
def load(cls, path: Path) -> BaselineConfig:
"""Load baseline configuration from JSON file."""
with path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
# Get tolerance profile, defaulting to 'pr_test'
profile_name = "pr_test"
tolerances = ToleranceConfig.load_profile(
data.get("tolerances", {}), profile_name
)
scenarios = {}
for name, cfg in data["scenarios"].items():
scenarios[name] = ScenarioConfig(
stages_ms=cfg["stages_ms"],
denoise_step_ms={int(k): v for k, v in cfg["denoise_step_ms"].items()},
expected_e2e_ms=float(cfg["expected_e2e_ms"]),
expected_avg_denoise_ms=float(cfg["expected_avg_denoise_ms"]),
expected_median_denoise_ms=float(cfg["expected_median_denoise_ms"]),
estimated_full_test_time_s=cfg.get("estimated_full_test_time_s"),
)
return cls(
scenarios=scenarios,
step_fractions=tuple(data["sampling"]["step_fractions"]),
tolerances=tolerances,
improvement_threshold=data.get("improvement_reporting", {}).get(
"threshold", 0.2
),
)
def update(self, path: Path):
"""Load baseline configuration from JSON file."""
with path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
scenarios_new = {}
for name, cfg in data["scenarios"].items():
scenarios_new[name] = ScenarioConfig(
stages_ms=cfg["stages_ms"],
denoise_step_ms={int(k): v for k, v in cfg["denoise_step_ms"].items()},
expected_e2e_ms=float(cfg["expected_e2e_ms"]),
expected_avg_denoise_ms=float(cfg["expected_avg_denoise_ms"]),
expected_median_denoise_ms=float(cfg["expected_median_denoise_ms"]),
estimated_full_test_time_s=cfg.get("estimated_full_test_time_s"),
)
self.scenarios.update(scenarios_new)
return self
@dataclass
class DiffusionServerArgs:
"""Configuration for a single model/scenario test case."""
model_path: str # HF repo or local path
modality: str | None = None # auto-inferred: "image", "video", "3d", or "action"
custom_validator: str | None = None # auto-derived unless explicitly overridden
# resources
num_gpus: int = 1
tp_size: int | None = None
ulysses_degree: int | None = None
ring_degree: int | None = None
cfg_parallel: bool | None = None
# LoRA
lora_path: str | None = (
None # LoRA adapter path (HF repo or local path, loaded at startup)
)
dynamic_lora_path: str | None = (
None # LoRA path for dynamic loading test (loaded via set_lora after startup)
)
second_lora_path: str | None = (
None # Second LoRA adapter path for multi-LoRA testing
)
dit_layerwise_offload: bool = False
dit_offload_prefetch_size: int | float | None = None
enable_cache_dit: bool = False
text_encoder_cpu_offload: bool = False
extras: list[str] = field(default_factory=lambda: [])
env_vars: dict[str, str] = field(default_factory=dict)
def __post_init__(self):
if self.modality is None:
self.modality = _infer_modality_from_model_path(self.model_path)
if self.custom_validator is not None:
return
if self.modality == "image":
self.custom_validator = "image"
elif self.modality == "video":
self.custom_validator = "video"
elif self.modality == "3d":
self.custom_validator = "mesh"
elif self.modality == "action":
self.custom_validator = "action"
@lru_cache(maxsize=None)
def _infer_modality_from_model_path(model_path: str) -> str:
model_info = get_model_info(model_path)
if model_info is None:
raise ValueError(f"Could not resolve model info for {model_path!r}")
task_type = model_info.pipeline_config_cls.task_type
if task_type == ModelTaskType.I2M:
return "3d"
if task_type.is_action_gen():
return "action"
if task_type.is_image_gen():
return "image"
return "video"
@dataclass(frozen=True)
class DiffusionSamplingParams:
"""Configuration for a single model/scenario test case."""
output_size: str = ""
# inputs and conditioning
prompt: str | None = None # text prompt for generation
image_path: Path | str | None = None # input image/video for editing (Path or URL)
# duration
seconds: int = 1 # for video: duration in seconds
num_frames: int | None = None # for video: number of frames
fps: int | None = None # for video: frames per second
# URL direct test flag - if True, don't pre-download URL images
direct_url_test: bool = False
# output format
output_format: str | None = None # "png", "jpeg", "mp4", etc.
num_outputs_per_prompt: int = 1
# Realtime video consistency harness. When set, server tests use
# /v1/realtime_video/generate and fold streamed chunks back into mp4 bytes.
realtime_num_chunks: int | None = None
realtime_events: list[dict[str, Any]] = field(default_factory=list)
realtime_perf_thresholds: dict[str, float] = field(default_factory=dict)
realtime_perf_ignore_initial_chunks: int = 0
# None keeps the lossless/raw transport used by GT-backed consistency checks.
realtime_output_format: str | None = None
# Additional request-level parameters (e.g. enable_teacache, enable_upscaling, …)
# merged directly into the OpenAI extra_body dict.
extras: dict = field(default_factory=dict)
@dataclass(frozen=True)
class DiffusionTestCase:
"""Configuration for a single model/scenario test case."""
id: str # pytest test id and scenario name
server_args: DiffusionServerArgs
sampling_params: DiffusionSamplingParams | None = None
run_perf_check: bool = True
run_consistency_check: bool = True
run_component_accuracy_check: bool = True
run_models_api_check: bool = True
run_t2v_input_reference_check: bool = True
run_lora_basic_api_check: bool = False
run_lora_dynamic_load_check: bool = False
run_lora_dynamic_switch_check: bool = False
run_multi_lora_api_check: bool = False
def __post_init__(self) -> None:
if self.sampling_params is None:
object.__setattr__(
self,
"sampling_params",
get_default_sampling_params_for_server_args(self.server_args),
)
has_startup_lora = self.server_args.lora_path is not None
has_dynamic_lora = self.server_args.dynamic_lora_path is not None
has_second_lora = self.server_args.second_lora_path is not None
if self.run_lora_basic_api_check and not (has_startup_lora or has_dynamic_lora):
raise ValueError(
f"{self.id}: run_lora_basic_api_check requires lora_path or dynamic_lora_path"
)
if self.run_lora_dynamic_load_check and not has_dynamic_lora:
raise ValueError(
f"{self.id}: run_lora_dynamic_load_check requires dynamic_lora_path"
)
if self.run_lora_dynamic_switch_check and not has_second_lora:
raise ValueError(
f"{self.id}: run_lora_dynamic_switch_check requires second_lora_path"
)
if self.run_multi_lora_api_check and not (has_startup_lora and has_second_lora):
raise ValueError(
f"{self.id}: run_multi_lora_api_check requires lora_path and second_lora_path"
)
LINGBOT_WORLD_REALTIME_sampling_params = DiffusionSamplingParams(
prompt=(
"A slow aerial orbit around a pastel floating island hotel in the open "
"ocean, hazy sunlight, turquoise water, toy-like architectural detail, "
"clean horizon, cinematic but playful."
),
image_path=(
"https://is1-ssl.mzstatic.com/image/thumb/Music/v4/b8/f9/b9/"
"b8f9b9f8-a609-bde2-0302-349436ffc508/825646291038.jpg/600x600bb.jpg"
),
output_size="832x480",
num_frames=9,
fps=16,
realtime_num_chunks=4,
realtime_perf_thresholds={
"p95_chunk_total_ms": 5000.0,
"p95_scheduler_forward_ms": 4500.0,
"p95_ws_payload_mb": 16.0,
},
realtime_perf_ignore_initial_chunks=2,
extras={
"seed": 42,
"num_inference_steps": 4,
"guidance_scale": 1.0,
"realtime_causal_sink_size": 9,
"realtime_causal_kv_cache_num_frames": 18,
"condition_inputs": {
"camera_actions": [
["w"],
["w"],
["w"],
["w"],
["w"],
["w"],
[],
[],
[],
[],
[],
[],
]
},
},
)
PI05_ACTION_CI_sampling_params = DiffusionSamplingParams(
prompt="pick up the blue block",
extras={
"action_horizon": 50,
"action_dim": 32,
"state_dim": 32,
"image_size": 64,
"num_inference_steps": 2,
"seed": 0,
"enable_prefix_cache": False,
"enable_cuda_graph": True,
"action_max_abs_diff_threshold": 0.05,
"action_mean_abs_diff_threshold": 0.005,
},
)
def sample_step_indices(
step_map: dict[int, float], fractions: Sequence[float]
) -> list[int]:
if not step_map:
return []
max_idx = max(step_map.keys())
indices = set()
for fraction in fractions:
idx = min(max_idx, max(0, int(round(fraction * max_idx))))
if idx in step_map:
indices.add(idx)
return sorted(indices)
@dataclass
class PerformanceSummary:
"""Summary of performance of a request, built from RequestPerfRecord"""
e2e_ms: float
avg_denoise_ms: float
median_denoise_ms: float
# { "stage_1": time_1, "stage_2": time_2 }
stage_metrics: dict[str, float]
step_metrics: list[float]
sampled_steps: dict[int, float]
all_denoise_steps: dict[int, float]
frames_per_second: float | None = None
total_frames: int | None = None
avg_frame_time_ms: float | None = None
@staticmethod
def from_req_perf_record(
record: RequestPerfRecord, step_fractions: Sequence[float]
):
"""Collect all performance metrics into a summary without validation."""
e2e_ms = record.total_duration_ms
step_durations = record.steps
avg_denoise = 0.0
median_denoise = 0.0
if step_durations:
avg_denoise = sum(step_durations) / len(step_durations)
median_denoise = statistics.median(step_durations)
per_step = {index: s for index, s in enumerate(step_durations)}
sample_indices = sample_step_indices(per_step, step_fractions)
sampled_steps = {idx: per_step[idx] for idx in sample_indices}
# convert from list to dict
stage_metrics = {}
for item in record.stages:
if isinstance(item, dict) and "name" in item:
val = item.get("execution_time_ms", 0.0)
stage_metrics[item["name"]] = val
return PerformanceSummary(
e2e_ms=e2e_ms,
avg_denoise_ms=avg_denoise,
median_denoise_ms=median_denoise,
stage_metrics=stage_metrics,
step_metrics=step_durations,
sampled_steps=sampled_steps,
all_denoise_steps=per_step,
)
T2I_sampling_params = DiffusionSamplingParams(
prompt="Doraemon is eating dorayaki",
output_size="1024x1024",
)
IDEOGRAM4_CI_TEXT_PROMPT = "A cat sitting on a bench"
IDEOGRAM4_CI_PROMPT = json.dumps(
{
"high_level_description": IDEOGRAM4_CI_TEXT_PROMPT,
"style_description": {
"aesthetics": "warm, peaceful, vibrant",
"lighting": "bright afternoon sunlight, long soft shadows",
"photo": "shallow depth of field, eye-level, 85mm lens",
"medium": "photograph",
"color_palette": [
"#F5C542",
"#87CEEB",
"#4A4A4A",
"#FFFFFF",
"#2E8B57",
],
},
"compositional_deconstruction": {
"background": (
"A sunlit garden path with green hedges and a wooden bench. "
"Dappled light filters through overhead trees."
),
"elements": [
{
"type": "obj",
"bbox": [260, 260, 760, 780],
"desc": (
"A small tabby cat sitting calmly on a wooden bench, "
"looking toward the camera."
),
},
{
"type": "obj",
"bbox": [180, 580, 840, 840],
"desc": (
"A weathered wooden garden bench with soft sunlight "
"falling across the seat."
),
},
],
},
},
separators=(",", ":"),
ensure_ascii=False,
)
COSMOS3_NANO_CI_sampling_params = DiffusionSamplingParams(
prompt="A red cube on a white table, product photo.",
output_size="832x480",
output_format="png",
extras={
"num_inference_steps": 35,
"seed": 0,
"max_sequence_length": 128,
"extra_args": {
"guardrails": False,
"use_resolution_template": False,
},
},
)
IDEOGRAM4_CI_sampling_params = replace(
T2I_sampling_params,
prompt=IDEOGRAM4_CI_PROMPT,
output_size="1024x1024",
output_format="png",
extras={"preset": "V4_QUALITY_48", "seed": 0},
)
MODELOPT_T2I_CI_sampling_params = DiffusionSamplingParams(
prompt="Doraemon is eating dorayaki",
output_size="768x768",
extras={"num_inference_steps": 12, "seed": 0},
)
MODELOPT_QWEN_IMAGE_2512_NVFP4_CI_sampling_params = replace(
MODELOPT_T2I_CI_sampling_params,
extras={"num_inference_steps": 50, "seed": 0},
)
MODELOPT_TI2I_CI_sampling_params = DiffusionSamplingParams(
prompt="Convert 2D style to 3D style",
image_path="https://github.com/lm-sys/lm-sys.github.io/releases/download/test/TI2I_Qwen_Image_Edit_Input.jpg",
output_size="512x512",
extras={"num_inference_steps": 8, "seed": 0},
)
TI2I_sampling_params = DiffusionSamplingParams(
prompt="Convert 2D style to 3D style",
image_path="https://github.com/lm-sys/lm-sys.github.io/releases/download/test/TI2I_Qwen_Image_Edit_Input.jpg",
)
MULTI_IMAGE_TI2I_sampling_params = DiffusionSamplingParams(
prompt="The magician bear is on the left, the alchemist bear is on the right, facing each other in the central park square.",
image_path=[
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/edit2509/edit2509_1.jpg",
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/edit2509/edit2509_2.jpg",
],
direct_url_test=True,
)
MULTI_IMAGE_TI2I_UPLOAD_sampling_params = DiffusionSamplingParams(
prompt="The magician bear is on the left, the alchemist bear is on the right, facing each other in the central park square.",
image_path=[
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/edit2509/edit2509_1.jpg",
"https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/edit2509/edit2509_2.jpg",
],
)
MULTI_FRAME_I2I_sampling_params = DiffusionSamplingParams(
prompt="a high quality, cute halloween themed illustration, consistent style and lighting",
image_path=[
"https://raw.githubusercontent.com/QwenLM/Qwen-Image-Layered/main/assets/test_images/4.png"
],
num_frames=4,
direct_url_test=True,
output_format="png",
)
T2V_PROMPT = "A curious raccoon"
T2V_sampling_params = DiffusionSamplingParams(
prompt=T2V_PROMPT,
)
JOY_ECHO_T2V_CI_sampling_params = DiffusionSamplingParams(
prompt=T2V_PROMPT,
output_size="640x384",
num_frames=33,
extras={
"num_inference_steps": 8,
"seed": 42,
"enable_memory_bank": False,
},
)
MODELOPT_T2V_CI_sampling_params = DiffusionSamplingParams(
prompt=T2V_PROMPT,
output_size="640x384",
seconds=5,
num_frames=17,
extras={"num_inference_steps": 12, "seed": 0},
)
TI2V_sampling_params = DiffusionSamplingParams(
prompt="The man in the picture slowly turns his head, his expression enigmatic and otherworldly. The camera performs a slow, cinematic dolly out, focusing on his face. Moody lighting, neon signs glowing in the background, shallow depth of field.",
image_path="https://is1-ssl.mzstatic.com/image/thumb/Music114/v4/5f/fa/56/5ffa56c2-ea1f-7a17-6bad-192ff9b6476d/825646124206.jpg/600x600bb.jpg",
direct_url_test=True,
)
SANA_WM_TI2V_CI_sampling_params = DiffusionSamplingParams(
prompt=TI2V_sampling_params.prompt,
image_path=TI2V_sampling_params.image_path,
direct_url_test=True,
output_size="384x640",
num_frames=17,
extras={"num_inference_steps": 12, "seed": 0, "guidance_scale": 4.5},
)
TURBOWAN_I2V_sampling_params = DiffusionSamplingParams(
prompt="The man in the picture slowly turns his head, his expression enigmatic and otherworldly. The camera performs a slow, cinematic dolly out, focusing on his face. Moody lighting, neon signs glowing in the background, shallow depth of field.",
image_path="https://is1-ssl.mzstatic.com/image/thumb/Music114/v4/5f/fa/56/5ffa56c2-ea1f-7a17-6bad-192ff9b6476d/825646124206.jpg/600x600bb.jpg",
direct_url_test=True,
output_size="960x960",
num_frames=4,
fps=4,
)
HUNYUAN3D_SHAPE_sampling_params = DiffusionSamplingParams(
prompt="",
image_path="https://raw.githubusercontent.com/sgl-project/sgl-test-files/main/diffusion-ci/consistency_gt/1-gpu/hunyuan3d_2_0/hunyuan3d.png",
)
def _get_extra_arg_value(extras: Sequence[str], option_name: str) -> str | None:
tokens: list[str] = []
for item in extras:
tokens.extend(shlex.split(item))
option_prefix = f"{option_name}="
for index, token in enumerate(tokens):
if token.startswith(option_prefix):
return token[len(option_prefix) :]
if token == option_name and index + 1 < len(tokens):
return tokens[index + 1]
return None
def get_model_task_type_for_server_args(
server_args: DiffusionServerArgs,
) -> ModelTaskType:
pipeline_class_name = _get_extra_arg_value(
server_args.extras, "--pipeline-class-name"
)
if pipeline_class_name:
config_classes = get_pipeline_config_classes(pipeline_class_name)
if config_classes is not None:
pipeline_config_cls, _ = config_classes
return pipeline_config_cls.task_type
model_info = get_model_info(server_args.model_path)
if model_info is None:
raise ValueError(f"Could not resolve model info for {server_args.model_path!r}")
return model_info.pipeline_config_cls.task_type
def get_default_sampling_params_for_model_task(
task_type: ModelTaskType,
) -> DiffusionSamplingParams:
if task_type == ModelTaskType.T2I:
return T2I_sampling_params
if task_type in (ModelTaskType.I2I, ModelTaskType.TI2I):
return TI2I_sampling_params
if task_type == ModelTaskType.T2V:
return T2V_sampling_params
if task_type in (ModelTaskType.I2V, ModelTaskType.TI2V):
return TI2V_sampling_params
if task_type == ModelTaskType.I2M:
return HUNYUAN3D_SHAPE_sampling_params
if task_type.is_action_gen():
return PI05_ACTION_CI_sampling_params
raise ValueError(f"No default sampling params for model task {task_type!r}")
def get_default_sampling_params_for_server_args(
server_args: DiffusionServerArgs,
) -> DiffusionSamplingParams:
task_type = get_model_task_type_for_server_args(server_args)
return get_default_sampling_params_for_model_task(task_type)
MODELOPT_FLUX1_FP8_TRANSFORMER = "lmsys/flux1-dev-modelopt-fp8-sglang-transformer"
MODELOPT_FLUX2_FP8_TRANSFORMER = "lmsys/flux2-dev-modelopt-fp8-sglang-transformer"
MODELOPT_WAN22_FP8_MODEL = "nvidia/Wan2.2-T2V-A14B-Diffusers-FP8"
MODELOPT_HUNYUANVIDEO_FP8_TRANSFORMER = (
"lmsys/hunyuanvideo-modelopt-fp8-sglang-transformer"
)
MODELOPT_QWEN_IMAGE_FP8_TRANSFORMER = "lmsys/qwen-image-modelopt-fp8-sglang-transformer"
MODELOPT_QWEN_IMAGE_EDIT_FP8_TRANSFORMER = (
"lmsys/qwen-image-edit-modelopt-fp8-sglang-transformer"
)
MODELOPT_FLUX1_NVFP4_TRANSFORMER = "lmsys/flux1-dev-modelopt-nvfp4-sglang-transformer"
MODELOPT_FLUX2_NVFP4_WEIGHTS = "black-forest-labs/FLUX.2-dev-NVFP4"
MODELOPT_QWEN_IMAGE_2512_NVFP4_MODEL = "lmsys/qwen-image-2512-modelopt-nvfp4-sglang"
MODELOPT_WAN22_NVFP4_MODEL = "nvidia/Wan2.2-T2V-A14B-Diffusers-NVFP4"
MODELOPT_NVFP4_B200_ENV_VARS = {}
MODELOPT_WAN22_NVFP4_B200_ENV_VARS = {}
PERF_BASELINE_PLATFORM_ENV = "SGLANG_DIFFUSION_PERF_BASELINE_PLATFORM"
PERF_BASELINE_DIR = Path(__file__).with_name("perf_baselines")
PERF_BASELINE_FILE_BY_PLATFORM = {
"h100": "h100.json",
"b200": "b200.json",
"5090": "5090.json",
}
PERF_BASELINE_PLATFORM_ALIASES = {
"sm90": "h100",
"hopper": "h100",
"h100": "h100",
"sm100": "b200",
"blackwell": "b200",
"b200": "b200",
"sm120": "5090",
"rtx5090": "5090",
"5090": "5090",
}
def _normalize_perf_baseline_platform(platform: str) -> str:
normalized = platform.strip().lower().replace("_", "-")
normalized = normalized.replace("-", "")
if normalized not in PERF_BASELINE_PLATFORM_ALIASES:
valid = ", ".join(sorted(PERF_BASELINE_FILE_BY_PLATFORM))
raise ValueError(
f"Invalid diffusion perf baseline platform {platform!r}. "
f"Expected one of: {valid}"
)
return PERF_BASELINE_PLATFORM_ALIASES[normalized]
def get_perf_baseline_platform() -> str:
override = os.getenv(PERF_BASELINE_PLATFORM_ENV)
if override:
return _normalize_perf_baseline_platform(override)
if current_platform.is_sm120():
return "5090"
if current_platform.is_blackwell():
return "b200"
return "h100"
def get_perf_baseline_path(platform: str | None = None) -> Path:
baseline_platform = (
_normalize_perf_baseline_platform(platform)
if platform is not None
else get_perf_baseline_platform()
)
return PERF_BASELINE_DIR / PERF_BASELINE_FILE_BY_PLATFORM[baseline_platform]
def _make_modelopt_ci_case(
case_id: str,
*,
model_path: str,
modality: str,
sampling_params: DiffusionSamplingParams,
extras: list[str],
env_vars: dict[str, str] | None = None,
run_consistency_check: bool = False,
) -> DiffusionTestCase:
return DiffusionTestCase(
case_id,
DiffusionServerArgs(
model_path=model_path,
modality=modality,
extras=extras,
env_vars=env_vars or {},
),
sampling_params,
run_perf_check=False,
run_consistency_check=run_consistency_check,
run_component_accuracy_check=False,
)
def _with_default_num_gpus(
cases: list[DiffusionTestCase], num_gpus: int
) -> list[DiffusionTestCase]:
return [
replace(case, server_args=replace(case.server_args, num_gpus=num_gpus))
for case in cases
]
# Load global configuration
BASELINE_CONFIG = (
BaselineConfig.load(get_perf_baseline_path())
.update(Path(__file__).parent / "ascend" / "perf_baselines_npu.json")
.update(Path(__file__).parent / "musa" / "perf_baselines_musa.json")
)