chore: import upstream snapshot with attribution

docs_new/docs/get-started/install.mdx

@@ -0,0 +1,241 @@
+---
+title: Installation
+description: Install SGLang with pip/uv, source, Docker, Kubernetes, and cloud deployment options.
+keywords:
+  - installation
+  - sglang
+  - pip
+  - docker
+---
+You can install SGLang using one of the methods below.
+This page primarily applies to common NVIDIA GPU platforms.
+For other or newer platforms, please refer to the dedicated pages for [AMD GPUs](../hardware-platforms/amd_gpu), [Apple Metal](../hardware-platforms/apple_metal), [Intel Xeon CPUs](../hardware-platforms/cpu_server), [Google TPU](../hardware-platforms/tpu), [NVIDIA DGX Spark](https://lmsys.org/blog/2025-11-03-gpt-oss-on-nvidia-dgx-spark/), [NVIDIA Jetson](../hardware-platforms/nvidia_jetson), [Ascend NPUs](../hardware-platforms/ascend-npus/ascend_npu), and [Intel XPU](../hardware-platforms/xpu).
+
+<Note>
+Prerequisites: Python 3.10 or higher.
+</Note>
+
+## Method 1: With pip or uv
+
+It is recommended to use uv for faster installation:
+
+```bash Command
+pip install --upgrade pip
+pip install uv
+uv pip install --prerelease=allow sglang
+```
+
+The major version of Cuda is 13 by default. To install sglang under Cuda 12 with pip or uv, please try the following commands:
+```bash Command
+pip install --upgrade pip
+pip install uv
+uv pip install --prerelease=allow sglang
+uv pip install --force-reinstall  torch==2.11.0 torchaudio==2.11.0 torchvision --index-url https://download.pytorch.org/whl/cu129
+uv pip install --force-reinstall sglang-kernel --index-url https://docs.sglang.ai/whl/cu129/
+uv pip install --force-reinstall sgl-deep-gemm --index-url https://docs.sglang.ai/whl/cu129/ --no-deps
+```
+
+### Nightly builds
+
+To pick up the latest features and fixes before the next stable release, install a nightly build. Nightly wheels are built from the latest `main` and published to the SGLang wheel index. Add that index with `--extra-index-url`, and combine `--prerelease=allow` with `--index-strategy unsafe-best-match` so uv considers the nightly (pre-release) version alongside PyPI:
+
+```bash Command
+pip install --upgrade pip
+pip install uv
+uv pip install --prerelease=allow --index-strategy unsafe-best-match --extra-index-url https://docs.sglang.ai/whl/cu130/ sglang
+```
+
+To install a nightly build under Cuda 12, swap the index to `cu129`:
+```bash Command
+pip install --upgrade pip
+pip install uv
+uv pip install --prerelease=allow --index-strategy unsafe-best-match --extra-index-url https://docs.sglang.ai/whl/cu129/ sglang
+```
+
+### Quick fixes to common problems
+- If you encounter `OSError: CUDA_HOME environment variable is not set`. Please set it to your CUDA install root with either of the following solutions:
+  1. Use `export CUDA_HOME=/usr/local/cuda-<your-cuda-version>` to set the `CUDA_HOME` environment variable.
+  2. Install FlashInfer first following [FlashInfer installation doc](https://docs.flashinfer.ai/installation.html), then install SGLang as described above.
+
+## Method 2: From source
+
+```bash Command
+# Use the last release branch
+git clone -b v0.5.12 https://github.com/sgl-project/sglang.git
+cd sglang
+
+# Install the python packages
+pip install --upgrade pip
+pip install -e "python"
+```
+
+**Quick fixes to common problems**
+
+- If you want to develop SGLang, you can try the dev docker image. Please refer to [setup docker container](../developer_guide/development_guide_using_docker#setup-docker-container). The docker image is `lmsysorg/sglang:dev`.
+
+## Method 3: Using docker
+
+The docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags), built from [Dockerfile](https://github.com/sgl-project/sglang/tree/main/docker).
+Replace `<secret>` below with your huggingface hub [token](https://huggingface.co/docs/hub/en/security-tokens).
+
+<Note>
+`latest` and `dev` are **mutable** tags: `latest` always points at the newest stable release, while `dev` is rebuilt daily from the latest `main` and includes build/development tools. Because they are overwritten over time, pin an immutable version tag for reproducible deployments — e.g. `lmsysorg/sglang:v0.5.12`. Browse all released versions on [Docker Hub](https://hub.docker.com/r/lmsysorg/sglang/tags).
+</Note>
+
+```bash Command
+docker run --gpus all \
+    --shm-size 32g \
+    -p 30000:30000 \
+    -v ~/.cache/huggingface:/root/.cache/huggingface \
+    --env "HF_TOKEN=<secret>" \
+    --ipc=host \
+    lmsysorg/sglang:latest \
+    python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
+```
+
+For production deployments, use the `runtime` variant which is significantly smaller (~40% reduction) by excluding build tools and development dependencies:
+
+```bash Command
+docker run --gpus all \
+    --shm-size 32g \
+    -p 30000:30000 \
+    -v ~/.cache/huggingface:/root/.cache/huggingface \
+    --env "HF_TOKEN=<secret>" \
+    --ipc=host \
+    lmsysorg/sglang:latest-runtime \
+    python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
+```
+
+You can also find the nightly docker images [here](https://hub.docker.com/r/lmsysorg/sglang/tags?name=nightly).
+
+Notes:
+- SGLang is shipped with CUDA 13 environment by default. To run SGLang on CUDA 12 environment, please use images with `-cu12` or `-cu129` suffix, such as `lmsysorg/sglang:latest-cu129` or `lmsysorg/sglang:dev-cu12`.
+
+## Method 4: Using Kubernetes
+
+Please check out [OME](https://github.com/sgl-project/ome), a Kubernetes operator for enterprise-grade management and serving of large language models (LLMs).
+
+<Accordion title="More">
+
+1. Option 1: For single node serving (typically when the model size fits into GPUs on one node)
+
+   Execute command `kubectl apply -f docker/k8s-sglang-service.yaml`, to create k8s deployment and service, with llama-31-8b as example.
+
+2. Option 2: For multi-node serving (usually when a large model requires more than one GPU node, such as `DeepSeek-R1`)
+
+   Modify the LLM model path and arguments as necessary, then execute command `kubectl apply -f docker/k8s-sglang-distributed-sts.yaml`, to create two nodes k8s statefulset and serving service.
+
+</Accordion>
+
+## Method 5: Using docker compose
+
+<Accordion title="More">
+
+> This method is recommended if you plan to serve it as a service.
+> A better approach is to use the [k8s-sglang-service.yaml](https://github.com/sgl-project/sglang/blob/main/docker/k8s-sglang-service.yaml).
+
+1. Copy the [compose.yml](https://github.com/sgl-project/sglang/blob/main/docker/compose.yaml) to your local machine
+2. Execute the command `docker compose up -d` in your terminal.
+
+</Accordion>
+
+## Method 6: Run on Kubernetes or Clouds with SkyPilot
+
+<Accordion title="More">
+
+To deploy on Kubernetes or 12+ clouds, you can use [SkyPilot](https://github.com/skypilot-org/skypilot).
+
+1. Install SkyPilot and set up Kubernetes cluster or cloud access: see [SkyPilot's documentation](https://skypilot.readthedocs.io/en/latest/getting-started/installation.html).
+2. Deploy on your own infra with a single command and get the HTTP API endpoint:
+
+<Accordion title={<>SkyPilot YAML: <code>sglang.yaml</code></>}>
+
+```yaml Config
+# sglang.yaml
+envs:
+  HF_TOKEN: null
+
+resources:
+  image_id: docker:lmsysorg/sglang:latest
+  accelerators: A100
+  ports: 30000
+
+run: |
+  conda deactivate
+  python3 -m sglang.launch_server \
+    --model-path meta-llama/Llama-3.1-8B-Instruct \
+    --host 0.0.0.0 \
+    --port 30000
+```
+
+</Accordion>
+
+```bash Command
+# Deploy on any cloud or Kubernetes cluster. Use --cloud <cloud> to select a specific cloud provider.
+HF_TOKEN=<secret> sky launch -c sglang --env HF_TOKEN sglang.yaml
+
+# Get the HTTP API endpoint
+sky status --endpoint 30000 sglang
+```
+
+3. To further scale up your deployment with autoscaling and failure recovery, check out the [SkyServe + SGLang guide](https://github.com/skypilot-org/skypilot/tree/master/llm/sglang#serving-llama-2-with-sglang-for-more-traffic-using-skyserve).
+
+</Accordion>
+
+## Method 7: Run on AWS SageMaker
+
+<Accordion title="More">
+
+To deploy on SGLang on AWS SageMaker, check out [AWS SageMaker Inference](https://aws.amazon.com/sagemaker/ai/deploy)
+
+Amazon Web Services provide supports for SGLang containers along with routine security patching. For available SGLang containers, check out [AWS SGLang DLCs](https://aws.github.io/deep-learning-containers/reference/available_images/#sglang).
+
+To deploy a pre-built SGLang Deep Learning Container without building your own image, see [Amazon SageMaker AI](/docs/basic_usage/aws_sagemaker).
+
+To host a model with your own container, follow the following steps:
+
+1. Build a docker container with [sagemaker.Dockerfile](https://github.com/sgl-project/sglang/blob/main/docker/sagemaker.Dockerfile) alongside the [serve](https://github.com/sgl-project/sglang/blob/main/docker/serve) script.
+2. Push your container onto AWS ECR.
+
+<Accordion title={<>Dockerfile Build Script: <code>build-and-push.sh</code></>}>
+
+```bash Command
+#!/bin/bash
+AWS_ACCOUNT="<YOUR_AWS_ACCOUNT>"
+AWS_REGION="<YOUR_AWS_REGION>"
+REPOSITORY_NAME="<YOUR_REPOSITORY_NAME>"
+IMAGE_TAG="<YOUR_IMAGE_TAG>"
+
+ECR_REGISTRY="${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com"
+IMAGE_URI="${ECR_REGISTRY}/${REPOSITORY_NAME}:${IMAGE_TAG}"
+
+echo "Starting build and push process..."
+
+# Login to ECR
+echo "Logging into ECR..."
+aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${ECR_REGISTRY}
+
+# Build the image
+echo "Building Docker image..."
+docker build -t ${IMAGE_URI} -f sagemaker.Dockerfile .
+
+echo "Pushing ${IMAGE_URI}"
+docker push ${IMAGE_URI}
+
+echo "Build and push completed successfully!"
+```
+
+</Accordion>
+
+3. Deploy a model for serving on AWS Sagemaker, refer to [deploy_and_serve_endpoint.py](https://github.com/sgl-project/sglang/blob/main/examples/sagemaker/deploy_and_serve_endpoint.py). For more information, check out [sagemaker-python-sdk](https://github.com/aws/sagemaker-python-sdk).
+   1. By default, the model server on SageMaker will run with the following command: `python3 -m sglang.launch_server --model-path opt/ml/model --host 0.0.0.0 --port 8080`. This is optimal for hosting your own model with SageMaker.
+   2. To modify your model serving parameters, the [serve](https://github.com/sgl-project/sglang/blob/main/docker/serve) script allows for all available options within `python3 -m sglang.launch_server --help` cli by specifying environment variables with prefix `SM_SGLANG_`.
+   3. The serve script will automatically convert all environment variables with prefix `SM_SGLANG_` from `SM_SGLANG_INPUT_ARGUMENT` into `--input-argument` to be parsed into `python3 -m sglang.launch_server` cli.
+   4. For example, to run [Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) with reasoning parser, simply add additional environment variables `SM_SGLANG_MODEL_PATH=Qwen/Qwen3-0.6B` and `SM_SGLANG_REASONING_PARSER=qwen3`.
+
+</Accordion>
+
+## Common Notes
+
+- [FlashInfer](https://github.com/flashinfer-ai/flashinfer) is the default attention kernel backend. It only supports sm75 and above. If you encounter any FlashInfer-related issues on sm75+ devices (e.g., T4, A10, A100, L4, L40S, H100), please switch to other kernels by adding `--attention-backend triton --sampling-backend pytorch` and open an issue on GitHub.
+- To reinstall flashinfer locally, use the following command: `pip3 install --upgrade flashinfer-python --force-reinstall --no-deps` and then delete the cache with `rm -rf ~/.cache/flashinfer`.

docs_new/docs/get-started/quickstart.mdx

@@ -0,0 +1,332 @@
+---
+title: "Quickstart"
+description: "Get up and running with SGLang in minutes: install, launch a server, and send your first request."
+---
+
+## Overview
+
+This guide walks you through the entire flow of getting started with SGLang:
+
+1. **Install** SGLang
+2. **Launch** an inference server
+3. **Send requests** using cURL, OpenAI Python client, Python `requests`, or the native SGLang API
+
+By the end, you'll have a working SGLang server responding to your prompts.
+
+---
+
+## Prerequisites
+
+- **Python**: 3.10 or higher
+- **GPU**: NVIDIA GPU with CUDA support (sm80 and above, e.g., A10, A100, L4, L40S, H100)
+- **OS**: Linux (recommended)
+
+<Note>
+For other platforms, see the dedicated guides for [AMD GPUs](../hardware-platforms/amd_gpu), [Intel Xeon CPUs](../hardware-platforms/cpu_server), [Google TPUs](../hardware-platforms/tpu), [NVIDIA Jetson](../hardware-platforms/nvidia_jetson), [Ascend NPUs](../hardware-platforms/ascend-npus/ascend_npu), and [Intel XPU](../hardware-platforms/xpu).
+</Note>
+
+---
+
+## Installation
+
+<Tabs>
+  <Tab title="Pip / uv (Recommended)">
+    We recommend using **uv** for faster installation:
+
+```bash
+pip install --upgrade pip
+pip install uv
+uv pip install --prerelease=allow sglang
+```
+  </Tab>
+  <Tab title="From Source">
+```bash
+# Clone and install from source
+git clone https://github.com/sgl-project/sglang.git
+cd sglang
+pip install --upgrade pip
+pip install -e "python"
+```
+  </Tab>
+  <Tab title="Docker">
+    The Docker images are available on Docker Hub at [lmsysorg/sglang](https://hub.docker.com/r/lmsysorg/sglang/tags).
+
+    Replace `<secret>` with your [Hugging Face token](https://huggingface.co/docs/hub/en/security-tokens):
+
+    ```bash
+    docker run --gpus all \
+        --shm-size 32g \
+        -p 30000:30000 \
+        -v ~/.cache/huggingface:/root/.cache/huggingface \
+        --env "HF_TOKEN=<secret>" \
+        --ipc=host \
+        lmsysorg/sglang:latest \
+        python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
+    ```
+
+    For production deployments, use the smaller **runtime** variant (~40% size reduction):
+
+    ```bash
+    docker run --gpus all \
+        --shm-size 32g \
+        -p 30000:30000 \
+        -v ~/.cache/huggingface:/root/.cache/huggingface \
+        --env "HF_TOKEN=<secret>" \
+        --ipc=host \
+        lmsysorg/sglang:latest-runtime \
+        python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --host 0.0.0.0 --port 30000
+    ```
+  </Tab>
+</Tabs>
+
+<Tip>
+If you encounter `OSError: CUDA_HOME environment variable is not set`, set it with:
+```bash
+export CUDA_HOME=/usr/local/cuda-<your-cuda-version>
+```
+</Tip>
+
+---
+
+## Launch a Server
+
+Start the SGLang server with a model. Here we use `qwen/qwen2.5-0.5b-instruct` as a lightweight example:
+
+```bash
+python3 -m sglang.launch_server --model-path qwen/qwen2.5-0.5b-instruct --host 0.0.0.0 --port 30000
+```
+
+Wait until you see `The server is fired up and ready to roll!` in the terminal output.
+
+<Note>
+Once the server is running, API documentation is available at:
+- **Swagger UI**: `http://localhost:30000/docs`
+- **ReDoc**: `http://localhost:30000/redoc`
+- **OpenAPI Spec**: `http://localhost:30000/openapi.json`
+</Note>
+
+<Info>
+The server automatically applies the chat template from the Hugging Face tokenizer. You can override it with `--chat-template` when launching.
+</Info>
+
+---
+
+## Send Requests
+
+SGLang is fully **OpenAI API-compatible**, so you can use the same tools and libraries you already know.
+
+### Using cURL
+
+```bash
+curl http://localhost:30000/v1/chat/completions \
+  -H "Content-Type: application/json" \
+  -d '{
+    "model": "qwen/qwen2.5-0.5b-instruct",
+    "messages": [
+      {"role": "user", "content": "What is the capital of France?"}
+    ]
+  }'
+```
+
+### Using OpenAI Python Client
+
+Install the OpenAI Python library if you haven't:
+
+```bash
+pip install openai
+```
+
+Then send a request:
+
+```python Example
+import openai
+
+client = openai.Client(base_url="http://127.0.0.1:30000/v1", api_key="None")
+
+response = client.chat.completions.create(
+    model="qwen/qwen2.5-0.5b-instruct",
+    messages=[
+        {"role": "user", "content": "List 3 countries and their capitals."},
+    ],
+    temperature=0,
+    max_tokens=64,
+)
+
+print(response.choices[0].message.content)
+```
+
+#### Streaming
+
+```python Example
+import openai
+
+client = openai.Client(base_url="http://127.0.0.1:30000/v1", api_key="None")
+
+response = client.chat.completions.create(
+    model="qwen/qwen2.5-0.5b-instruct",
+    messages=[
+        {"role": "user", "content": "List 3 countries and their capitals."},
+    ],
+    temperature=0,
+    max_tokens=64,
+    stream=True,
+)
+
+for chunk in response:
+    if chunk.choices[0].delta.content:
+        print(chunk.choices[0].delta.content, end="", flush=True)
+```
+
+### Using Python Requests
+
+```python Example
+import requests
+
+url = "http://localhost:30000/v1/chat/completions"
+
+data = {
+    "model": "qwen/qwen2.5-0.5b-instruct",
+    "messages": [{"role": "user", "content": "What is the capital of France?"}],
+}
+
+response = requests.post(url, json=data)
+print(response.json())
+```
+
+### Using the Native `/generate` API
+
+SGLang also provides a native `/generate` endpoint for more flexibility.
+
+```python Example
+import requests
+
+response = requests.post(
+    "http://localhost:30000/generate",
+    json={
+        "text": "The capital of France is",
+        "sampling_params": {
+            "temperature": 0,
+            "max_new_tokens": 32,
+        },
+    },
+)
+
+print(response.json())
+```
+
+#### Streaming with `/generate`
+
+```python Example
+import requests
+import json
+
+response = requests.post(
+    "http://localhost:30000/generate",
+    json={
+        "text": "The capital of France is",
+        "sampling_params": {
+            "temperature": 0,
+            "max_new_tokens": 32,
+        },
+        "stream": True,
+    },
+    stream=True,
+)
+
+prev = 0
+for chunk in response.iter_lines(decode_unicode=False):
+    chunk = chunk.decode("utf-8")
+    if chunk and chunk.startswith("data:"):
+        if chunk == "data: [DONE]":
+            break
+        data = json.loads(chunk[5:].strip("\n"))
+        output = data["text"]
+        print(output[prev:], end="", flush=True)
+        prev = len(output)
+```
+
+---
+
+## Offline Batch Inference (No Server)
+
+SGLang also supports offline batch inference using the `Engine` class directly -- no HTTP server required.
+
+```python Example
+import sglang as sgl
+
+llm = sgl.Engine(model_path="qwen/qwen2.5-0.5b-instruct")
+
+prompts = [
+    "Hello, my name is",
+    "The president of the United States is",
+    "The capital of France is",
+    "The future of AI is",
+]
+
+sampling_params = {"temperature": 0.8, "top_p": 0.95}
+
+outputs = llm.generate(prompts, sampling_params)
+
+for prompt, output in zip(prompts, outputs):
+    print(f"Prompt: {prompt}\nGenerated text: {output['text']}\n")
+
+llm.shutdown()
+```
+
+---
+
+## Common Troubleshooting
+
+<AccordionGroup>
+  <Accordion title="CUDA_HOME not set">
+    Set the `CUDA_HOME` environment variable to your CUDA install root:
+    ```bash
+    export CUDA_HOME=/usr/local/cuda-<your-cuda-version>
+    ```
+  </Accordion>
+  <Accordion title="FlashInfer issues on sm75+ devices">
+    Switch to alternative backends by adding these flags when launching the server:
+    ```bash
+    --attention-backend triton --sampling-backend pytorch
+    ```
+  </Accordion>
+  <Accordion title="Reinstalling FlashInfer">
+    ```bash
+    pip3 install --upgrade flashinfer-python --force-reinstall --no-deps
+    rm -rf ~/.cache/flashinfer
+    ```
+  </Accordion>
+  <Accordion title="ptxas error on B300/GB300 (sm_103a)">
+    ```bash
+    export TRITON_PTXAS_PATH=/usr/local/cuda/bin/ptxas
+    ```
+  </Accordion>
+</AccordionGroup>
+
+---
+
+{/*
+WIP, TBD linked later
+## What's Next?
+
+<CardGroup cols={2}>
+  <Card title="OpenAI-Compatible APIs" href="../basic_usage/openai_api_completions">
+    Explore the full Chat Completions and Completions APIs, including multi-turn conversations.
+  </Card>
+  <Card title="Vision Language Models" href="../basic_usage/openai_api_vision">
+    Send image inputs alongside text using OpenAI-compatible vision APIs.
+  </Card>
+  <Card title="Sampling Parameters" href="../basic_usage/sampling_params">
+    Fine-tune generation with temperature, top-p, frequency penalty, and more.
+  </Card>
+  <Card title="Server Arguments" href="../advanced_features/server_arguments">
+    Customize server behavior with advanced launch arguments like tensor parallelism.
+  </Card>
+  <Card title="Structured Outputs" href="../advanced_features/structured_outputs">
+    Constrain model output to JSON, regex, or EBNF grammars.
+  </Card>
+  <Card title="Ollama-Compatible API" href="../basic_usage/ollama_api">
+    Use the familiar Ollama CLI and Python library with SGLang as the backend.
+  </Card>
+</CardGroup>
+*/}