wehub-resource-sync
19 lines
1.8 KiB
Markdown
19 lines
1.8 KiB
Markdown
# Examples
|
||
|
||
vLLM's examples are organized into the following categories:
|
||
|
||
- **[`basic/`](../../examples/basic)** – Minimal examples for offline inference and online serving.
|
||
- **[`generate/`](../../examples/generate)** – Text generation examples, including multimodal models.
|
||
- **[`pooling/`](../../examples/pooling)** – Examples for embedding, classification, scoring, reward, etc.
|
||
- **[`speech_to_text/`](../../examples/speech_to_text)** – Speech transcription, translation and real-time audio examples.
|
||
- **[`features/`](../../examples/features)** – Demonstrations of individual vLLM features: automatic prefix caching, speculative decoding, LoRA, structured outputs, prompt embedding, pause/resume, batch invariance, KV events, data parallelism, and more.
|
||
- **[`reasoning/`](../../examples/reasoning)** – Examples for reasoning with vLLM.
|
||
- **[`tool_calling/`](../../examples/tool_calling)** – Examples for function/tool calling with vLLM.
|
||
- **[`applications/`](../../examples/applications)** – Application examples such as simpler api server, chatbots and RAG (Retrieval-Augmented Generation).
|
||
- **[`rl/`](../../examples/rl)** – Reinforcement learning examples.
|
||
- **[`deployment/`](../../examples/deployment)** – Examples for deploying vLLM in production.
|
||
- **[`ray_serving/`](../../examples/ray_serving)** – Scalable serving using Ray.
|
||
- **[`disaggregated/`](../../examples/disaggregated)** – Examples for Disaggregated P/D (Prefill/Decoding) inference, including various kv cache connectors (LMCache, Mooncake, FlexKV, P2P NCCL) and failure recovery.
|
||
- **[`scale_out/`](../../examples/scale_out)** – Examples for Token In <> Token Out API Server.
|
||
- **[`observability/`](../../examples/observability)** – Metrics, logging, tracing (OpenTelemetry), and dashboards (Grafana, Perses).
|