From 296f9921e489b381fb35e8693bc49ae95775e8fb Mon Sep 17 00:00:00 2001 From: wehub-resource-sync Date: Mon, 13 Jul 2026 10:29:31 +0000 Subject: [PATCH] docs: make Chinese README the default --- README.md | 130 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 68 insertions(+), 62 deletions(-) diff --git a/README.md b/README.md index e5cfd3d..81eca07 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,9 @@ + +> [!NOTE] +> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。 +> [English](./README.en.md) · [原始项目](https://github.com/sgl-project/sglang) · [上游 README](https://github.com/sgl-project/sglang/blob/HEAD/README.md) +> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。 +
logo @@ -13,83 +19,83 @@ --------------------------------------------------------------------------------

-Blog | -Documentation | -Roadmap | -Join Slack | -Weekly Dev Meeting | -Slides +博客 | +文档 | +路线图 | +加入 Slack | +每周开发会议 | +幻灯片

-## News -- [2026/06] 🔥 The next generation of speculative decoding: DFlash and Spec V2 ([blog](https://lmsys.org/blog/2026-06-15-next-generation-speculative-decoding-dflash-v2/)). -- [2026/04] 🔥 DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles ([blog](https://lmsys.org/blog/2026-04-25-deepseek-v4/)). -- [2026/06] SGLang provides day-0 support for latest open models ([Nemotron 3 Ultra](https://lmsys.org/blog/2026-06-04-nvidia-run-nemotron-3-ultra/), [Nemotron 3 Super](https://lmsys.org/blog/2026-03-11-run-nvidia-nemotron-3-super/), [Higgs Audio v3 TTS](https://lmsys.org/blog/2026-06-04-higgs-audio-v3-tts/)). -- [2026/02] 🔥 Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72 ([blog](https://lmsys.org/blog/2026-02-20-gb300-inferencex/)). -- [2026/01] SGLang Diffusion accelerates video and image generation ([blog](https://lmsys.org/blog/2026-01-16-sglang-diffusion/)). -- [2025/12] SGLang provides day-0 support for latest open models ([MiMo-V2-Flash](https://lmsys.org/blog/2025-12-16-mimo-v2-flash/), [Nemotron 3 Nano](https://lmsys.org/blog/2025-12-15-run-nvidia-nemotron-3-nano/), [Mistral Large 3](https://github.com/sgl-project/sglang/pull/14213), [LLaDA 2.0 Diffusion LLM](https://lmsys.org/blog/2025-12-19-diffusion-llm/), [MiniMax M2](https://lmsys.org/blog/2025-11-04-miminmax-m2/)). -- [2025/10] SGLang now runs natively on TPU with the SGLang-Jax backend ([blog](https://lmsys.org/blog/2025-10-29-sglang-jax/)). +## 新闻 +- [2026/06] 🔥 下一代推测解码（Speculative Decoding）：DFlash 与 Spec V2（[博客](https://lmsys.org/blog/2026-06-15-next-generation-speculative-decoding-dflash-v2/)). +- [2026/04] 🔥 DeepSeek-V4 首日上线：从 SGLang 与 Miles 实现快速推理到可验证强化学习（[博客](https://lmsys.org/blog/2026-04-25-deepseek-v4/)). +- [2026/06] SGLang 为最新开源模型提供首日（day-0）支持（[Nemotron 3 Ultra](https://lmsys.org/blog/2026-06-04-nvidia-run-nemotron-3-ultra/), [Nemotron 3 Super](https://lmsys.org/blog/2026-03-11-run-nvidia-nemotron-3-super/), [Higgs Audio v3 TTS](https://lmsys.org/blog/2026-06-04-higgs-audio-v3-tts/)). +- [2026/02] 🔥 在 NVIDIA GB300 NVL72 上借助 SGLang 解锁 25 倍推理性能（[博客](https://lmsys.org/blog/2026-02-20-gb300-inferencex/)). +- [2026/01] SGLang Diffusion 加速视频与图像生成（[博客](https://lmsys.org/blog/2026-01-16-sglang-diffusion/)). +- [2025/12] SGLang 为最新开源模型提供首日（day-0）支持（[MiMo-V2-Flash](https://lmsys.org/blog/2025-12-16-mimo-v2-flash/), [Nemotron 3 Nano](https://lmsys.org/blog/2025-12-15-run-nvidia-nemotron-3-nano/), [Mistral Large 3](https://github.com/sgl-project/sglang/pull/14213), [LLaDA 2.0 Diffusion LLM](https://lmsys.org/blog/2025-12-19-diffusion-llm/), [MiniMax M2](https://lmsys.org/blog/2025-11-04-miminmax-m2/)). +- [2025/10] SGLang 现可通过 SGLang-Jax 后端原生运行于 TPU（[博客](https://lmsys.org/blog/2025-10-29-sglang-jax/)).
-More +更多 -- [2025/09] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part II): 3.8x Prefill, 4.8x Decode Throughput ([blog](https://lmsys.org/blog/2025-09-25-gb200-part-2/)). -- [2025/09] SGLang Day 0 Support for DeepSeek-V3.2 with Sparse Attention ([blog](https://lmsys.org/blog/2025-09-29-deepseek-V32/)). -- [2025/08] SGLang x AMD SF Meetup on 8/22: Hands-on GPU workshop, tech talks by AMD/xAI/SGLang, and networking ([Roadmap](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_sglang_roadmap.pdf), [Large-scale EP](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_sglang_ep.pdf), [Highlights](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_highlights.pdf), [AITER/MoRI](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_aiter_mori.pdf), [Wave](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_wave.pdf)). +- [2025/09] 在 GB200 NVL72 上部署 DeepSeek，采用 PD 与大规模 EP（第二部分）：Prefill 提升 3.8 倍，Decode 吞吐量提升 4.8 倍（[博客](https://lmsys.org/blog/2025-09-25-gb200-part-2/)). +- [2025/09] SGLang 首日支持带稀疏注意力（Sparse Attention）的 DeepSeek-V3.2（[博客](https://lmsys.org/blog/2025-09-29-deepseek-V32/)). +- [2025/08] SGLang x AMD SF 线下聚会（8/22）：GPU 动手工作坊、AMD/xAI/SGLang 技术分享与社交交流（[路线图](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_sglang_roadmap.pdf), [大规模 EP](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_sglang_ep.pdf), [亮点](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_highlights.pdf), [AITER/MoRI](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_aiter_mori.pdf), [Wave](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/amd_meetup_wave.pdf)). -- [2025/11] SGLang Diffusion accelerates video and image generation ([blog](https://lmsys.org/blog/2025-11-07-sglang-diffusion/)). -- [2025/10] PyTorch Conference 2025 SGLang Talk ([slide](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/sglang_pytorch_2025.pdf)). -- [2025/10] SGLang x Nvidia SF Meetup on 10/2 ([recap](https://x.com/lmsysorg/status/1975339501934510231)). -- [2025/08] SGLang provides day-0 support for OpenAI gpt-oss model ([instructions](https://github.com/sgl-project/sglang/issues/8833)) -- [2025/06] SGLang, the high-performance serving infrastructure powering trillions of tokens daily, has been awarded the third batch of the Open Source AI Grant by a16z ([a16z blog](https://a16z.com/advancing-open-source-ai-through-benchmarks-and-bold-experimentation/)). -- [2025/05] Deploying DeepSeek with PD Disaggregation and Large-scale Expert Parallelism on 96 H100 GPUs ([blog](https://lmsys.org/blog/2025-05-05-large-scale-ep/)). -- [2025/06] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part I): 2.7x Higher Decoding Throughput ([blog](https://lmsys.org/blog/2025-06-16-gb200-part-1/)). -- [2025/03] Supercharge DeepSeek-R1 Inference on AMD Instinct MI300X ([AMD blog](https://rocm.blogs.amd.com/artificial-intelligence/DeepSeekR1-Part2/README.html)) -- [2025/03] SGLang Joins PyTorch Ecosystem: Efficient LLM Serving Engine ([PyTorch blog](https://pytorch.org/blog/sglang-joins-pytorch/)) -- [2025/02] Unlock DeepSeek-R1 Inference Performance on AMD Instinct™ MI300X GPU ([AMD blog](https://rocm.blogs.amd.com/artificial-intelligence/DeepSeekR1_Perf/README.html)) -- [2025/01] SGLang provides day one support for DeepSeek V3/R1 models on NVIDIA and AMD GPUs with DeepSeek-specific optimizations. ([instructions](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3), [AMD blog](https://www.amd.com/en/developer/resources/technical-articles/amd-instinct-gpus-power-deepseek-v3-revolutionizing-ai-development-with-sglang.html), [10+ other companies](https://x.com/lmsysorg/status/1887262321636221412)) -- [2024/12] v0.4 Release: Zero-Overhead Batch Scheduler, Cache-Aware Load Balancer, Faster Structured Outputs ([blog](https://lmsys.org/blog/2024-12-04-sglang-v0-4/)). -- [2024/10] The First SGLang Online Meetup ([slides](https://github.com/sgl-project/sgl-learning-materials?tab=readme-ov-file#the-first-sglang-online-meetup)). -- [2024/09] v0.3 Release: 7x Faster DeepSeek MLA, 1.5x Faster torch.compile, Multi-Image/Video LLaVA-OneVision ([blog](https://lmsys.org/blog/2024-09-04-sglang-v0-3/)). -- [2024/07] v0.2 Release: Faster Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM) ([blog](https://lmsys.org/blog/2024-07-25-sglang-llama3/)). -- [2024/02] SGLang enables **3x faster JSON decoding** with compressed finite state machine ([blog](https://lmsys.org/blog/2024-02-05-compressed-fsm/)). -- [2024/01] SGLang provides up to **5x faster inference** with RadixAttention ([blog](https://lmsys.org/blog/2024-01-17-sglang/)). -- [2024/01] SGLang powers the serving of the official **LLaVA v1.6** release demo ([usage](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)). +- [2025/11] SGLang Diffusion 加速视频与图像生成（[博客](https://lmsys.org/blog/2025-11-07-sglang-diffusion/)). +- [2025/10] PyTorch Conference 2025 SGLang 演讲（[幻灯片](https://github.com/sgl-project/sgl-learning-materials/blob/main/slides/sglang_pytorch_2025.pdf)). +- [2025/10] SGLang x Nvidia SF 线下聚会（10/2）（[回顾](https://x.com/lmsysorg/status/1975339501934510231)). +- [2025/08] SGLang 为 OpenAI gpt-oss 模型提供首日（day-0）支持（[说明](https://github.com/sgl-project/sglang/issues/8833)) +- [2025/06] SGLang 作为每日驱动数万亿 token 的高性能服务基础设施，荣获 a16z 第三批开源 AI 资助（[a16z 博客](https://a16z.com/advancing-open-source-ai-through-benchmarks-and-bold-experimentation/)). +- [2025/05] 在 96 张 H100 GPU 上通过 PD 分离与大规模专家并行（Expert Parallelism）部署 DeepSeek（[博客](https://lmsys.org/blog/2025-05-05-large-scale-ep/)). +- [2025/06] 在 GB200 NVL72 上部署 DeepSeek，采用 PD 与大规模 EP（第一部分）：Decode 吞吐量提升 2.7 倍（[博客](https://lmsys.org/blog/2025-06-16-gb200-part-1/)). +- [2025/03] 在 AMD Instinct MI300X 上为 DeepSeek-R1 推理加速（[AMD 博客](https://rocm.blogs.amd.com/artificial-intelligence/DeepSeekR1-Part2/README.html)) +- [2025/03] SGLang 加入 PyTorch 生态：高效 LLM 服务引擎（[PyTorch 博客](https://pytorch.org/blog/sglang-joins-pytorch/)) +- [2025/02] 在 AMD Instinct™ MI300X GPU 上解锁 DeepSeek-R1 推理性能（[AMD 博客](https://rocm.blogs.amd.com/artificial-intelligence/DeepSeekR1_Perf/README.html)) +- [2025/01] SGLang 在 NVIDIA 与 AMD GPU 上为 DeepSeek V3/R1 模型提供首日支持，并包含 DeepSeek 专属优化。（[说明](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3), [AMD 博客](https://www.amd.com/en/developer/resources/technical-articles/amd-instinct-gpus-power-deepseek-v3-revolutionizing-ai-development-with-sglang.html), [10+ 其他公司](https://x.com/lmsysorg/status/1887262321636221412)) +- [2024/12] v0.4 发布：零开销批处理调度器、缓存感知负载均衡器、更快的结构化输出（[博客](https://lmsys.org/blog/2024-12-04-sglang-v0-4/)). +- [2024/10] 首届 SGLang 线上聚会（[幻灯片](https://github.com/sgl-project/sgl-learning-materials?tab=readme-ov-file#the-first-sglang-online-meetup)). +- [2024/09] v0.3 发布：DeepSeek MLA 快 7 倍、torch.compile 快 1.5 倍、多图/视频 LLaVA-OneVision（[博客](https://lmsys.org/blog/2024-09-04-sglang-v0-3/)). +- [2024/07] v0.2 发布：借助 SGLang Runtime 更快服务 Llama3（对比 TensorRT-LLM、vLLM）（[博客](https://lmsys.org/blog/2024-07-25-sglang-llama3/)). +- [2024/02] SGLang 借助压缩有限状态机实现 **JSON 解码快 3 倍**（[博客](https://lmsys.org/blog/2024-02-05-compressed-fsm/)). +- [2024/01] SGLang 借助 RadixAttention 实现最高 **5 倍更快推理**（[博客](https://lmsys.org/blog/2024-01-17-sglang/)). +- [2024/01] SGLang 为官方 **LLaVA v1.6** 发布演示提供推理服务（[用法](https://github.com/haotian-liu/LLaVA?tab=readme-ov-file#demo)).
-## About -SGLang is a high-performance serving framework for large language models and multimodal models. -It is designed to deliver low-latency and high-throughput inference across a wide range of setups, from a single GPU to large distributed clusters. -Its core features include: +## 简介 +SGLang 是一个面向大语言模型（LLM）与多模态模型的高性能服务框架。 +它旨在在从单 GPU 到大型分布式集群的广泛部署场景中，提供低延迟、高吞吐量的推理能力。 +其核心特性包括： -- **Fast Runtime**: Provides efficient serving with RadixAttention for prefix caching, a zero-overhead CPU scheduler, prefill-decode disaggregation, speculative decoding, continuous batching, paged attention, tensor/pipeline/expert/data parallelism, structured outputs, chunked prefill, quantization (FP4/FP8/INT4/AWQ/GPTQ), and multi-LoRA batching. -- **Broad Model Support**: Supports a wide range of language models (Llama, Qwen, DeepSeek, Kimi, GLM, GPT, Gemma, Mistral, etc.), embedding models (e5-mistral, gte, mcdse), reward models (Skywork), and diffusion models (WAN, Qwen-Image), with easy extensibility for adding new models. Compatible with most Hugging Face models and OpenAI APIs. -- **Extensive Hardware Support**: Runs on NVIDIA GPUs (GB200/B300/H100/A100/Spark/5090), AMD GPUs (MI355/MI300), Intel Xeon CPUs, Google TPUs, Ascend NPUs, and more. -- **Active Community**: SGLang is open-source and supported by a vibrant community with widespread industry adoption, powering over 400,000 GPUs worldwide. -- **RL & Post-Training Backbone**: SGLang is a proven rollout backend used for training many frontier models, with native RL integrations and adoption by well-known post-training frameworks such as [**AReaL**](https://github.com/inclusionAI/AReaL), [**Miles**](https://github.com/radixark/miles), [**slime**](https://github.com/THUDM/slime), [**Tunix**](https://github.com/google/tunix), [**verl**](https://github.com/volcengine/verl) and more. +- **快速运行时（Fast Runtime）**：通过 RadixAttention 实现前缀缓存（prefix caching）、零开销 CPU 调度器、prefill-decode 分离（prefill-decode disaggregation）、推测解码（speculative decoding）、连续批处理（continuous batching）、分页注意力（paged attention）、张量/流水线/专家/数据并行（tensor/pipeline/expert/data parallelism）、结构化输出（structured outputs）、分块 prefill（chunked prefill）、量化（FP4/FP8/INT4/AWQ/GPTQ）以及多 LoRA 批处理，提供高效服务。 +- **广泛的模型支持（Broad Model Support）**：支持多种语言模型（Llama、Qwen、DeepSeek、Kimi、GLM、GPT、Gemma、Mistral 等）、嵌入模型（e5-mistral、gte、mcdse）、奖励模型（Skywork）以及扩散模型（WAN、Qwen-Image），并易于扩展以接入新模型。兼容大多数 Hugging Face 模型与 OpenAI API。 +- **广泛的硬件支持（Extensive Hardware Support）**：可运行于 NVIDIA GPU（GB200/B300/H100/A100/Spark/5090）、AMD GPU（MI355/MI300）、Intel Xeon CPU、Google TPU、Ascend NPU 等。 +- **活跃的社区（Active Community）**：SGLang 是开源项目，拥有活跃社区与广泛的行业采用，全球范围内驱动超过 400,000 张 GPU。 +- **强化学习与后训练骨干（RL & Post-Training Backbone）**：SGLang 是经实战验证的 rollout 后端，已用于训练众多前沿模型，具备原生 RL 集成，并被知名后训练框架广泛采用，例如 [**AReaL**](https://github.com/inclusionAI/AReaL), [**Miles**](https://github.com/radixark/miles), [**slime**](https://github.com/THUDM/slime), [**Tunix**](https://github.com/google/tunix), [**verl**](https://github.com/volcengine/verl) 等。 -## Getting Started -- [Install SGLang](https://docs.sglang.io/get_started/install.html) -- [Quick Start](https://docs.sglang.io/basic_usage/send_request.html) -- [Backend Tutorial](https://docs.sglang.io/basic_usage/openai_api_completions.html) -- [Frontend Tutorial](https://docs.sglang.io/references/frontend/frontend_tutorial.html) -- [Contribution Guide](https://docs.sglang.io/developer_guide/contribution_guide.html) +## 入门指南 +- [安装 SGLang](https://docs.sglang.io/get_started/install.html) +- [快速开始](https://docs.sglang.io/basic_usage/send_request.html) +- [后端教程](https://docs.sglang.io/basic_usage/openai_api_completions.html) +- [前端教程](https://docs.sglang.io/references/frontend/frontend_tutorial.html) +- [贡献指南](https://docs.sglang.io/developer_guide/contribution_guide.html) -## Benchmark and Performance -Learn more in the release blogs: [v0.2 blog](https://lmsys.org/blog/2024-07-25-sglang-llama3/), [v0.3 blog](https://lmsys.org/blog/2024-09-04-sglang-v0-3/), [v0.4 blog](https://lmsys.org/blog/2024-12-04-sglang-v0-4/), [Large-scale expert parallelism](https://lmsys.org/blog/2025-05-05-large-scale-ep/), [GB200 rack-scale parallelism](https://lmsys.org/blog/2025-09-25-gb200-part-2/), [GB300 long context](https://lmsys.org/blog/2026-02-19-gb300-longctx/). +## 基准测试与性能 +详见发布博客：[v0.2 博客](https://lmsys.org/blog/2024-07-25-sglang-llama3/), [v0.3 博客](https://lmsys.org/blog/2024-09-04-sglang-v0-3/), [v0.4 博客](https://lmsys.org/blog/2024-12-04-sglang-v0-4/), [大规模专家并行（Large-scale expert parallelism）](https://lmsys.org/blog/2025-05-05-large-scale-ep/), [GB200 机架级并行（GB200 rack-scale parallelism）](https://lmsys.org/blog/2025-09-25-gb200-part-2/), [GB300 长上下文（GB300 long context）](https://lmsys.org/blog/2026-02-19-gb300-longctx/). -## Adoption and Sponsorship -SGLang has been deployed at large scale, generating trillions of tokens in production each day. It is trusted and adopted by a wide range of leading enterprises and institutions, including xAI, AMD, NVIDIA, Intel, LinkedIn, Cursor, Oracle Cloud, Google Cloud, Microsoft Azure, AWS, Atlas Cloud, Voltage Park, Nebius, DataCrunch, Novita, InnoMatrix, Modal, MIT, UCLA, the University of Washington, Stanford, UC Berkeley, Tsinghua University, Jam & Tea Studios, Baseten, and other major technology organizations. -As an open-source LLM inference engine, SGLang has become the de facto industry standard, with deployments running on over 400,000 GPUs worldwide. -SGLang is currently hosted under the non-profit open-source organization [LMSYS](https://lmsys.org/about/). +## 采用与赞助 +SGLang 已大规模部署，每天在生产环境中生成数万亿 token。它受到众多领先企业与机构的信赖与采用，包括 xAI、AMD、NVIDIA、Intel、LinkedIn、Cursor、Oracle Cloud、Google Cloud、Microsoft Azure、AWS、Atlas Cloud、Voltage Park、Nebius、DataCrunch、Novita、InnoMatrix、Modal、MIT、UCLA、华盛顿大学（University of Washington）、Stanford、UC Berkeley、清华大学（Tsinghua University）、Jam & Tea Studios、Baseten 及其他主要科技组织。 +作为一款开源 LLM 推理引擎，SGLang 已成为事实上的行业标准，全球部署规模超过 400,000 块 GPU。 +SGLang 目前由非营利开源组织 [LMSYS](https://lmsys.org/about/). 托管。 logo -## Contact Us -For enterprises interested in adopting or deploying SGLang at scale, including technical consulting, sponsorship opportunities, or partnership inquiries, please contact us at [sglang@lmsys.org](mailto:sglang@lmsys.org). +## 联系我们 +如有意大规模采用或部署 SGLang，包括技术咨询、赞助机会或合作洽谈，请通过 [sglang@lmsys.org](mailto:sglang@lmsys.org) 与我们联系。 -Long-term active SGLang contributors are eligible for coding agent sponsorship, such as Cursor, Claude Code, or OpenAI Codex. Email [sglang@lmsys.org](mailto:sglang@lmsys.org) with your most important commits or pull requests. +长期活跃的 SGLang 贡献者可能有资格获得编程智能体（coding agent）赞助，例如 Cursor、Claude Code 或 OpenAI Codex。请将您最重要的 commits 或 pull requests 发送至 [sglang@lmsys.org](mailto:sglang@lmsys.org)。 -## Acknowledgment -We learned the design and reused code from the following projects: [Guidance](https://github.com/guidance-ai/guidance), [vLLM](https://github.com/vllm-project/vllm), [LightLLM](https://github.com/ModelTC/lightllm), [FlashInfer](https://github.com/flashinfer-ai/flashinfer), [Outlines](https://github.com/outlines-dev/outlines), and [LMQL](https://github.com/eth-sri/lmql). +## 致谢 +我们从以下项目中学习设计思路并复用代码：[Guidance](https://github.com/guidance-ai/guidance), [vLLM](https://github.com/vllm-project/vllm), [LightLLM](https://github.com/ModelTC/lightllm), [FlashInfer](https://github.com/flashinfer-ai/flashinfer), [Outlines](https://github.com/outlines-dev/outlines), 以及 [LMQL](https://github.com/eth-sri/lmql).