wehub/ray-project--ray
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:17:40 +08:00
commit f1825c8ceb
10096 changed files with 2364182 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/ray/llm/examples/sglang/readme.md
+27
View File
@@ -0,0 +1,27 @@
# SGLang on Ray Serve LLM
This directory contains example scripts for using SGLang with Ray Serve LLM.
## Examples
| File | Description |
|------|-------------|
| `serve_sglang_example.py` | Single-node SGLang serving with autoscaling |
| `serve_sglang_multinode_example.py` | Multi-node serving with tensor and pipeline parallelism |
| `batch_sglang_example.py` | Batch inference using Ray Data |
| `query_example.py` | OpenAI client for querying a running deployment |
## Prerequisites
```bash
pip install ray[serve,llm] "sglang[all,ray]"
```
Set the environment variable before running:
- **CUDA:** `RAY_EXPERIMENTAL_NOSET_CUDA_VISIBLE_DEVICES=0`
- **ROCm:** `RAY_EXPERIMENTAL_NOSET_HIP_VISIBLE_DEVICES=0`
## Engine implementation
The `SGLangServer` class is located at `ray.llm._internal.serve.engines.sglang` and wraps SGLang's in-process engine with the Ray Serve LLM server protocol.