wehub/ray-project--ray
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:17:40 +08:00
commit f1825c8ceb
10096 changed files with 2364182 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/ray-overview/examples.rst
+7
View File
@@ -0,0 +1,7 @@
:orphan:
:html_theme.sidebar_secondary.remove:
.. _ref-ray-examples:
Ray Examples
============
doc/source/ray-overview/examples/index.rst
+19
View File
@@ -0,0 +1,19 @@
.. _ref-overview-examples:
Examples
========
.. toctree::
:maxdepth: 2
/_collections/ray-overview/examples/e2e-multimodal-ai-workloads/README
/_collections/ray-overview/examples/entity-recognition-with-llms/README
/_collections/ray-overview/examples/e2e-audio/README
/_collections/ray-overview/examples/e2e-xgboost/README
/_collections/ray-overview/examples/e2e-timeseries/README
/_collections/ray-overview/examples/object-detection/README
/_collections/ray-overview/examples/e2e-rag/README
/_collections/ray-overview/examples/mcp-ray-serve/README
/_collections/ray-overview/examples/langchain_agent_ray_serve/content/README
/_collections/ray-overview/examples/multi_agent_a2a/README
doc/source/ray-overview/getting-started.md
+819
View File
@@ -0,0 +1,819 @@
(gentle-intro)=
# Getting Started
Ray is an open source unified framework for scaling AI and Python applications. It provides a simple, universal API for building distributed applications that can scale from a laptop to a cluster.
## What's Ray?
Ray simplifies distributed computing by providing:
- **Scalable compute primitives**: Tasks and actors for painless parallel programming
- **Specialized AI libraries**: Tools for common ML workloads like data processing, model training, hyperparameter tuning, and model serving
- **Unified resource management**: Seamless scaling from laptop to cloud with automatic resource handling
## Choose Your Path
Select the guide that matches your needs:
* **Scale ML workloads**: [Ray Libraries Quickstart](#libraries-quickstart)
* **Scale general Python applications**: [Ray Core Quickstart](#ray-core-quickstart)
* **Deploy to the cloud**: [Ray Clusters Quickstart](#ray-cluster-quickstart)
* **Debug and monitor applications**: [Debugging and Monitoring Quickstart](#debugging-and-monitoring-quickstart)
```{image} ../images/map-of-ray.svg
:align: center
:alt: Ray Framework Architecture
```
(libraries-quickstart)=
## Ray AI Libraries Quickstart
Use individual libraries for ML workloads. Each library specializes in a specific part of the ML workflow, from data processing to model serving. Click on the dropdowns for your workload below.
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Data: Scalable Data Processing for AI Workloads
:animate: fade-in-slide-down
[Ray Data](data_quickstart) provides distributed data processing capabilities for AI workloads. It efficiently streams data through data pipelines.
Here's an example of how to scale offline inference and training ingest with Ray Data.
````{note}
To run this example, install Ray Data:
```bash
pip install -U "ray[data]"
```
````
```{testcode}
from typing import Dict
import numpy as np
import ray
# Create datasets from on-disk files, Python objects, and cloud storage like S3.
ds = ray.data.read_csv("s3://anonymous@ray-example-data/iris.csv")
# Apply functions to transform data. Ray Data executes transformations in parallel.
def compute_area(batch: Dict[str, np.ndarray]) -> Dict[str, np.ndarray]:
length = batch["petal length (cm)"]
width = batch["petal width (cm)"]
batch["petal area (cm^2)"] = length * width
return batch
transformed_ds = ds.map_batches(compute_area, batch_size="auto")
# Iterate over batches of data.
for batch in transformed_ds.iter_batches(batch_size=4):
print(batch)
# Save dataset contents to on-disk files or cloud storage.
transformed_ds.write_parquet("local:///tmp/iris/")
```
```{testoutput}
:hide:
...
```
```{button-ref} ../data/data
:color: primary
:outline:
:expand:
Learn more about Ray Data
```
`````
``````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Train: Distributed Model Training
:animate: fade-in-slide-down
**Ray Train** makes distributed model training simple. It abstracts away the complexity of setting up distributed training across popular frameworks like PyTorch and TensorFlow.
`````{tab-set}
````{tab-item} PyTorch
This example shows how you can use Ray Train with PyTorch.
To run this example install Ray Train and PyTorch packages:
:::{note}
```bash
pip install -U "ray[train]" torch torchvision
```
:::
Set up your dataset and model.
```{literalinclude} /../../python/ray/train/examples/pytorch/torch_quick_start.py
:language: python
:start-after: __torch_setup_begin__
:end-before: __torch_setup_end__
```
Now define your single-worker PyTorch training function.
```{literalinclude} /../../python/ray/train/examples/pytorch/torch_quick_start.py
:language: python
:start-after: __torch_single_begin__
:end-before: __torch_single_end__
```
This training function can be executed with:
```{literalinclude} /../../python/ray/train/examples/pytorch/torch_quick_start.py
:language: python
:start-after: __torch_single_run_begin__
:end-before: __torch_single_run_end__
:dedent: 4
```
Convert this to a distributed multi-worker training function.
Use the ``ray.train.torch.prepare_model`` and
``ray.train.torch.prepare_data_loader`` utility functions to
set up your model and data for distributed training.
This automatically wraps the model with ``DistributedDataParallel``
and places it on the right device, and adds ``DistributedSampler`` to the DataLoaders.
```{literalinclude} /../../python/ray/train/examples/pytorch/torch_quick_start.py
:language: python
:start-after: __torch_distributed_begin__
:end-before: __torch_distributed_end__
```
Instantiate a ``TorchTrainer``
with 4 workers, and use it to run the new training function.
```{literalinclude} /../../python/ray/train/examples/pytorch/torch_quick_start.py
:language: python
:start-after: __torch_trainer_begin__
:end-before: __torch_trainer_end__
:dedent: 4
```
To accelerate the training job using GPU, make sure you have GPU configured, then set `use_gpu` to `True`. If you don't have a GPU environment, Anyscale provides a development workspace integrated with an autoscaling GPU cluster for this purpose.
<div class="anyscale-cta">
<a href="https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=ray-doc-upsell&utm_content=get-started-train-torch">
<img src="../_static/img/try-ray-on-anyscale.svg" alt="Try Ray on Anyscale">
</a>
</div>
````
````{tab-item} TensorFlow
This example shows how you can use Ray Train to set up [Multi-worker training
with Keras](https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras).
To run this example install Ray Train and Tensorflow packages:
:::{note}
```bash
pip install -U "ray[train]" tensorflow
```
:::
Set up your dataset and model.
```{literalinclude} /../../python/ray/train/examples/tf/tensorflow_quick_start.py
:language: python
:start-after: __tf_setup_begin__
:end-before: __tf_setup_end__
```
Now define your single-worker TensorFlow training function.
```{literalinclude} /../../python/ray/train/examples/tf/tensorflow_quick_start.py
:language: python
:start-after: __tf_single_begin__
:end-before: __tf_single_end__
```
This training function can be executed with:
```{literalinclude} /../../python/ray/train/examples/tf/tensorflow_quick_start.py
:language: python
:start-after: __tf_single_run_begin__
:end-before: __tf_single_run_end__
:dedent: 0
```
Now convert this to a distributed multi-worker training function.
1. Set the *global* batch size - each worker processes the same size
batch as in the single-worker code.
2. Choose your TensorFlow distributed training strategy. This examples
uses the ``MultiWorkerMirroredStrategy``.
```{literalinclude} /../../python/ray/train/examples/tf/tensorflow_quick_start.py
:language: python
:start-after: __tf_distributed_begin__
:end-before: __tf_distributed_end__
```
Instantiate a ``TensorflowTrainer``
with 4 workers, and use it to run the new training function.
```{literalinclude} /../../python/ray/train/examples/tf/tensorflow_quick_start.py
:language: python
:start-after: __tf_trainer_begin__
:end-before: __tf_trainer_end__
:dedent: 0
```
To accelerate the training job using GPU, make sure you have GPU configured, then set `use_gpu` to `True`. If you don't have a GPU environment, Anyscale provides a development workspace integrated with an autoscaling GPU cluster for this purpose.
<div class="anyscale-cta">
<a href="https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=ray-doc-upsell&utm_content=get-started-train-tf">
<img src="../_static/img/try-ray-on-anyscale.svg" alt="Try Ray on Anyscale">
</a>
</div>
```{button-ref} ../train/train
:color: primary
:outline:
:expand:
Learn more about Ray Train
```
````
`````
``````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Tune: Hyperparameter Tuning at Scale
:animate: fade-in-slide-down
[Ray Tune](../tune/index.rst) is a library for hyperparameter tuning at any scale.
It automatically finds the best hyperparameters for your models with efficient distributed search algorithms.
With Tune, you can launch a multi-node distributed hyperparameter sweep in less than 10 lines of code, supporting any deep learning framework including PyTorch, TensorFlow, and Keras.
````{note}
To run this example, install Ray Tune:
```bash
pip install -U "ray[tune]"
```
````
This example runs a small grid search with an iterative training function.
```{literalinclude} ../../../python/ray/tune/tests/example.py
:end-before: __quick_start_end__
:language: python
:start-after: __quick_start_begin__
```
If TensorBoard is installed (`pip install tensorboard`), you can automatically visualize all trial results:
```bash
tensorboard --logdir ~/ray_results
```
```{button-ref} ../tune/index
:color: primary
:outline:
:expand:
Learn more about Ray Tune
```
`````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Serve: Scalable Model Serving
:animate: fade-in-slide-down
[Ray Serve](../serve/index) provides scalable and programmable serving for ML models and business logic. Deploy models from any framework with production-ready performance.
````{note}
To run this example, install Ray Serve and scikit-learn:
```{code-block} bash
pip install -U "ray[serve]" scikit-learn
```
````
This example runs serves a scikit-learn gradient boosting classifier.
```{literalinclude} ../serve/doc_code/sklearn_quickstart.py
:language: python
:start-after: __serve_example_begin__
:end-before: __serve_example_end__
```
The response shows `{"result": "versicolor"}`.
```{button-ref} ../serve/index
:color: primary
:outline:
:expand:
Learn more about Ray Serve
```
`````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> RLlib: Industry-Grade Reinforcement Learning
:animate: fade-in-slide-down
[RLlib](../rllib/index.rst) is a reinforcement learning (RL) library that offers high performance implementations of popular RL algorithms and supports various training environments. RLlib offers high scalability and unified APIs for a variety of industry- and research applications.
````{note}
To run this example, install `rllib` and either `tensorflow` or `pytorch`:
```bash
pip install -U "ray[rllib]" tensorflow # or torch
```
You may also need CMake installed on your system.
````
```{literalinclude} ../rllib/doc_code/rllib_on_ray_readme.py
:end-before: __quick_start_end__
:language: python
:start-after: __quick_start_begin__
```
```{button-ref} ../rllib/index
:color: primary
:outline:
:expand:
Learn more about Ray RLlib
```
`````
## Ray Core Quickstart
<a href="https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=ray-core-quickstart&redirectTo=/v2/template-preview/workspace-intro">
<img src="../_static/img/run-on-anyscale.svg" alt="try-anyscale-quickstart-ray-quickstart">
</a>
<br></br>
Ray Core provides simple primitives for building and running distributed applications. It enables you to turn regular Python or Java functions and classes into distributed stateless tasks and stateful actors with just a few lines of code.
The examples below show you how to:
1. Convert Python functions to Ray tasks for parallel execution
2. Convert Python classes to Ray actors for distributed stateful computation
``````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Core: Parallelizing Functions with Ray Tasks
:animate: fade-in-slide-down
`````{tab-set}
````{tab-item} Python
:::{note}
To run this example install Ray Core:
```bash
pip install -U "ray"
```
:::
Import Ray and and initialize it with `ray.init()`.
Then decorate the function with ``@ray.remote`` to declare that you want to run this function remotely.
Lastly, call the function with ``.remote()`` instead of calling it normally.
This remote call yields a future, a Ray _object reference_, that you can then fetch with ``ray.get``.
```{code-block} python
import ray
ray.init()
@ray.remote
def f(x):
return x * x
futures = [f.remote(i) for i in range(4)]
print(ray.get(futures)) # [0, 1, 4, 9]
```
````
````{tab-item} Java
```{note}
To run this example, add the [ray-api](https://mvnrepository.com/artifact/io.ray/ray-api) and [ray-runtime](https://mvnrepository.com/artifact/io.ray/ray-runtime) dependencies in your project.
```
Use `Ray.init` to initialize Ray runtime.
Then use `Ray.task(...).remote()` to convert any Java static method into a Ray task.
The task runs asynchronously in a remote worker process. The `remote` method returns an ``ObjectRef``,
and you can fetch the actual result with ``get``.
```{code-block} java
import io.ray.api.ObjectRef;
import io.ray.api.Ray;
import java.util.ArrayList;
import java.util.List;
public class RayDemo {
public static int square(int x) {
return x * x;
}
public static void main(String[] args) {
// Initialize Ray runtime.
Ray.init();
List<ObjectRef<Integer>> objectRefList = new ArrayList<>();
// Invoke the `square` method 4 times remotely as Ray tasks.
// The tasks run in parallel in the background.
for (int i = 0; i < 4; i++) {
objectRefList.add(Ray.task(RayDemo::square, i).remote());
}
// Get the actual results of the tasks.
System.out.println(Ray.get(objectRefList)); // [0, 1, 4, 9]
}
}
```
In the above code block we defined some Ray Tasks. While these are great for stateless operations, sometimes you
must maintain the state of your application. You can do that with Ray Actors.
```{button-ref} ../ray-core/walkthrough
:color: primary
:outline:
:expand:
Learn more about Ray Core
```
````
`````
``````
``````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Core: Parallelizing Classes with Ray Actors
:animate: fade-in-slide-down
Ray provides actors to allow you to parallelize an instance of a class in Python or Java.
When you instantiate a class that is a Ray actor, Ray starts a remote instance
of that class in the cluster. This actor can then execute remote method calls and
maintain its own internal state.
`````{tab-set}
````{tab-item} Python
:::{note}
To run this example install Ray Core:
```bash
pip install -U "ray"
```
:::
```{code-block} python
import ray
ray.init() # Only call this once.
@ray.remote
class Counter(object):
def __init__(self):
self.n = 0
def increment(self):
self.n += 1
def read(self):
return self.n
counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
print(ray.get(futures)) # [1, 1, 1, 1]
```
````
````{tab-item} Java
```{note}
To run this example, add the [ray-api](https://mvnrepository.com/artifact/io.ray/ray-api) and [ray-runtime](https://mvnrepository.com/artifact/io.ray/ray-runtime) dependencies in your project.
```
```{code-block} java
import io.ray.api.ActorHandle;
import io.ray.api.ObjectRef;
import io.ray.api.Ray;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Collectors;
public class RayDemo {
public static class Counter {
private int value = 0;
public void increment() {
this.value += 1;
}
public int read() {
return this.value;
}
}
public static void main(String[] args) {
// Initialize Ray runtime.
Ray.init();
List<ActorHandle<Counter>> counters = new ArrayList<>();
// Create 4 actors from the `Counter` class.
// These run in remote worker processes.
for (int i = 0; i < 4; i++) {
counters.add(Ray.actor(Counter::new).remote());
}
// Invoke the `increment` method on each actor.
// This sends an actor task to each remote actor.
for (ActorHandle<Counter> counter : counters) {
counter.task(Counter::increment).remote();
}
// Invoke the `read` method on each actor, and print the results.
List<ObjectRef<Integer>> objectRefList = counters.stream()
.map(counter -> counter.task(Counter::read).remote())
.collect(Collectors.toList());
System.out.println(Ray.get(objectRefList)); // [1, 1, 1, 1]
}
}
```
```{button-ref} ../ray-core/walkthrough
:color: primary
:outline:
:expand:
Learn more about Ray Core
```
````
`````
``````
## Ray Cluster Quickstart
Deploy your applications on Ray clusters on AWS, GCP, Azure, and more, often with minimal code changes to your existing code.
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Clusters: Launching a Ray Cluster on AWS
:animate: fade-in-slide-down
Ray programs can run on a single machine, or seamlessly scale to large clusters.
:::{note}
To run this example install the following:
```bash
pip install -U "ray[default]" boto3
```
If you haven't already, configure your credentials as described in the [documentation for boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#guide-credentials).
:::
Take this simple example that waits for individual nodes to join the cluster.
````{dropdown} example.py
:animate: fade-in-slide-down
```{literalinclude} ../../yarn/example.py
:language: python
```
````
You can also download this example from the [GitHub repository](https://github.com/ray-project/ray/blob/master/doc/yarn/example.py).
Store it locally in a file called `example.py`.
To execute this script in the cloud, download [this configuration file](https://github.com/ray-project/ray/blob/master/python/ray/autoscaler/aws/example-minimal.yaml),
or copy it here:
````{dropdown} cluster.yaml
:animate: fade-in-slide-down
```{literalinclude} ../../../python/ray/autoscaler/aws/example-minimal.yaml
:language: yaml
```
````
Assuming you have stored this configuration in a file called `cluster.yaml`, you can now launch an AWS cluster as follows:
```bash
ray submit cluster.yaml example.py --start
```
```{button-ref} cluster-index
:color: primary
:outline:
:expand:
Learn more about launching Ray Clusters on AWS, GCP, Azure, and more
```
`````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Clusters: Launching a Ray Cluster on Kubernetes
:animate: fade-in-slide-down
Ray programs can run on a single node Kubernetes cluster, or seamlessly scale to larger clusters.
```{button-ref} kuberay-index
:color: primary
:outline:
:expand:
Learn more about launching Ray Clusters on Kubernetes
```
`````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Clusters: Launching a Ray Cluster on Anyscale
:animate: fade-in-slide-down
Anyscale is the company behind Ray. The Anyscale platform provides an enterprise-grade Ray deployment on top of your AWS, GCP, Azure, or on-prem Kubernetes clusters.
```{button-link} https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=ray-doc-upsell&utm_content=get-started-launch-ray-cluster
:color: primary
:outline:
:expand:
Try Ray on Anyscale
```
`````
## Debugging and Monitoring Quickstart
Use built-in observability tools to monitor and debug Ray applications and clusters. These tools help you understand your application's performance and identify bottlenecks.
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Ray Dashboard: Web GUI to monitor and debug Ray
:animate: fade-in-slide-down
Ray dashboard provides a visual interface that displays real-time system metrics, node-level resource monitoring, job profiling, and task visualizations. The dashboard is designed to help users understand the performance of their Ray applications and identify potential issues.
```{image} https://raw.githubusercontent.com/ray-project/Images/master/docs/new-dashboard/Dashboard-overview.png
:align: center
```
````{note}
To get started with the dashboard, install the default installation as follows:
```bash
pip install -U "ray[default]"
```
````
The dashboard automatically becomes available when running Ray scripts. Access the dashboard through the default URL, http://localhost:8265.
```{button-ref} observability-getting-started
:color: primary
:outline:
:expand:
Learn more about Ray Dashboard
```
`````
`````{dropdown} <img src="images/ray_svg_logo.svg" alt="ray" width="50px"> Ray State APIs: CLI to access cluster states
:animate: fade-in-slide-down
Ray state APIs allow users to conveniently access the current state (snapshot) of Ray through CLI or Python SDK.
````{note}
To get started with the state API, install the default installation as follows:
```bash
pip install -U "ray[default]"
```
````
Run the following code.
```{code-block} python
import ray
import time
ray.init(num_cpus=4)
@ray.remote
def task_running_300_seconds():
print("Start!")
time.sleep(300)
@ray.remote
class Actor:
def __init__(self):
print("Actor created")
# Create 2 tasks
tasks = [task_running_300_seconds.remote() for _ in range(2)]
# Create 2 actors
actors = [Actor.remote() for _ in range(2)]
ray.get(tasks)
```
See the summarized statistics of Ray tasks using ``ray summary tasks`` in a terminal.
```{code-block} bash
ray summary tasks
```
```{code-block} text
======== Tasks Summary: 2022-07-22 08:54:38.332537 ========
Stats:
------------------------------------
total_actor_scheduled: 2
total_actor_tasks: 0
total_tasks: 2
Table (group by func_name):
------------------------------------
FUNC_OR_CLASS_NAME STATE_COUNTS TYPE
0 task_running_300_seconds RUNNING: 2 NORMAL_TASK
1 Actor.__init__ FINISHED: 2 ACTOR_CREATION_TASK
```
```{button-ref} observability-programmatic
:color: primary
:outline:
:expand:
Learn more about Ray State APIs
```
`````
## Learn More
Ray has a rich ecosystem of resources to help you learn more about distributed computing and AI scaling.
### Blog and Press
- [Modern Parallel and Distributed Python: A Quick Tutorial on Ray](https://medium.com/data-science/modern-parallel-and-distributed-python-a-quick-tutorial-on-ray-99f8d70369b8)
- [Why Every Python Developer Will Love Ray](https://www.datanami.com/2019/11/05/why-every-python-developer-will-love-ray/)
- [Ray: A Distributed System for AI (Berkeley Artificial Intelligence Research, BAIR)](http://bair.berkeley.edu/blog/2018/01/09/ray/)
- [10x Faster Parallel Python Without Python Multiprocessing](https://medium.com/data-science/10x-faster-parallel-python-without-python-multiprocessing-e5017c93cce1)
- [Implementing A Parameter Server in 15 Lines of Python with Ray](https://ray-project.github.io/2018/07/15/parameter-server-in-fifteen-lines.html)
- [Ray Distributed AI Framework Curriculum](https://rise.cs.berkeley.edu/blog/ray-intel-curriculum/)
- [RayOnSpark: Running Emerging AI Applications on Big Data Clusters with Ray and Analytics Zoo](https://medium.com/riselab/rayonspark-running-emerging-ai-applications-on-big-data-clusters-with-ray-and-analytics-zoo-923e0136ed6a)
- [First user tips for Ray](https://rise.cs.berkeley.edu/blog/ray-tips-for-first-time-users/)
- [Tune: a Python library for fast hyperparameter tuning at any scale](https://medium.com/data-science/fast-hyperparameter-tuning-at-scale-d428223b081c)
- [Cutting edge hyperparameter tuning with Ray Tune](https://medium.com/riselab/cutting-edge-hyperparameter-tuning-with-ray-tune-be6c0447afdf)
- [New Library Targets High Speed Reinforcement Learning](https://www.datanami.com/2018/02/01/rays-new-library-targets-high-speed-reinforcement-learning/)
- [Scaling Multi Agent Reinforcement Learning](http://bair.berkeley.edu/blog/2018/12/12/rllib/)
- [Functional RL with Keras and Tensorflow Eager](https://bair.berkeley.edu/blog/2019/10/14/functional-rl/)
- [How to Speed up Pandas by 4x with one line of code](https://www.kdnuggets.com/2019/11/speed-up-pandas-4x.html)
- [Quick Tip—Speed up Pandas using Modin](https://ericbrown.com/quick-tip-speed-up-pandas-using-modin.htm)
- [Ray Blog](https://medium.com/distributed-computing-with-ray)
### Videos
- [Unifying Large Scale Data Preprocessing and Machine Learning Pipelines with Ray Data \| PyData 2021](https://www.youtube.com/watch?v=wl4tvru9_Cg) [(slides)](https://docs.google.com/presentation/d/19F_wxkpo1JAROPxULmJHYZd3sKryapkbMd0ib3ndMiU/edit?usp=sharing)
- [Programming at any Scale with Ray \| SF Python Meetup Sept 2019](https://www.youtube.com/watch?v=LfpHyIXBhlE)
- [Ray for Reinforcement Learning \| Data Council 2019](https://www.youtube.com/watch?v=Ayc0ca150HI)
- [Scaling Interactive Pandas Workflows with Modin](https://www.youtube.com/watch?v=-HjLd_3ahCw)
- [Ray: A Distributed Execution Framework for AI \| SciPy 2018](https://www.youtube.com/watch?v=D_oz7E4v-U0)
- [Ray: A Cluster Computing Engine for Reinforcement Learning Applications \| Spark Summit](https://www.youtube.com/watch?v=xadZRRB_TeI)
- [RLlib: Ray Reinforcement Learning Library \| RISECamp 2018](https://www.youtube.com/watch?v=eeRGORQthaQ)
- [Enabling Composition in Distributed Reinforcement Learning \| Spark Summit 2018](https://www.youtube.com/watch?v=jAEPqjkjth4)
- [Tune: Distributed Hyperparameter Search \| RISECamp 2018](https://www.youtube.com/watch?v=38Yd_dXW51Q)
### Slides
- [Talk given at UC Berkeley DS100](https://docs.google.com/presentation/d/1sF5T_ePR9R6fAi2R6uxehHzXuieme63O2n_5i9m7mVE/edit?usp=sharing)
- [Talk given in October 2019](https://docs.google.com/presentation/d/13K0JsogYQX3gUCGhmQ1PQ8HILwEDFysnq0cI2b88XbU/edit?usp=sharing)
- [Talk given at RISECamp 2019](https://docs.google.com/presentation/d/1v3IldXWrFNMK-vuONlSdEuM82fuGTrNUDuwtfx4axsQ/edit?usp=sharing)
### Papers
- [Ray 2.0 Architecture white paper](https://docs.google.com/document/d/1tBw9A4j62ruI5omIJbMxly-la5w4q_TjyJgJL_jN2fI/preview)
- [Ray 1.0 Architecture white paper (old)](https://docs.google.com/document/d/1lAy0Owi-vPz2jEqBSaHNQcy2IBSDEHyXNOQZlGuj93c/preview)
- [Exoshuffle: large-scale data shuffle in Ray](https://arxiv.org/abs/2203.05072)
- [RLlib paper](https://arxiv.org/abs/1712.09381)
- [RLlib flow paper](https://arxiv.org/abs/2011.12719)
- [Tune paper](https://arxiv.org/abs/1807.05118)
- [Ray paper (old)](https://arxiv.org/abs/1712.05889)
- [Ray HotOS paper (old)](https://arxiv.org/abs/1703.03924)
If you encounter technical issues, post on the [Ray discussion forum](https://discuss.ray.io/). For general questions, announcements, and community discussions, join the [Ray community on Slack](https://www.ray.io/join-slack).
doc/source/ray-overview/images/ray_ecosystem_integration_v2.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 100 KiB

doc/source/ray-overview/images/ray_framework_v2.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 26 KiB

doc/source/ray-overview/images/ray_svg_logo.svg
+13
View File
@@ -0,0 +1,13 @@
<svg width="400" height="201" viewBox="0 0 400 201" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M325.948 134.356V109.785L302.441 66.6406H314.243L330.495 97.3062H331.946L348.198 66.6406H360L336.493 109.785V134.356H325.948Z" fill="black"/>
<path d="M253.043 134.364L272.39 66.6484H290.77L310.021 134.364H299.283L294.833 118.403H268.327L263.877 134.364H253.043ZM270.939 108.729H292.221L282.354 73.1298H280.806L270.939 108.729Z" fill="black"/>
<path d="M198.887 134.364V66.6484H227.327C231.519 66.6484 235.195 67.3901 238.355 68.8734C241.58 70.2922 244.063 72.3559 245.804 75.0645C247.61 77.7732 248.513 80.9977 248.513 84.7382V85.8023C248.513 90.0587 247.481 93.4767 245.417 96.0564C243.418 98.5715 240.967 100.345 238.065 101.377V102.925C240.516 103.054 242.483 103.892 243.966 105.44C245.449 106.923 246.191 109.084 246.191 111.921V134.364H235.647V113.372C235.647 111.631 235.195 110.244 234.292 109.213C233.39 108.181 231.938 107.665 229.939 107.665H209.334V134.364H198.887ZM209.334 98.1846H226.166C229.907 98.1846 232.809 97.2495 234.873 95.3792C236.937 93.4445 237.968 90.8326 237.968 87.5436V86.7697C237.968 83.4806 236.937 80.901 234.873 79.0307C232.874 77.096 229.971 76.1286 226.166 76.1286H209.334V98.1846Z" fill="black"/>
<path fill-rule="evenodd" clip-rule="evenodd" d="M143.629 101.311L98.3087 146.631L94.9902 143.313L140.311 97.9922L143.629 101.311Z" fill="#02A0CF"/>
<path fill-rule="evenodd" clip-rule="evenodd" d="M141.953 102.334L51.4453 102.334L51.4453 97.6406L141.953 97.6406L141.953 102.334Z" fill="#02A0CF"/>
<path fill-rule="evenodd" clip-rule="evenodd" d="M94.9916 55.9864L140.312 101.307L143.631 97.9887L98.3101 52.668L94.9916 55.9864Z" fill="#02A0CF"/>
<rect x="40" y="88.3164" width="22.6604" height="22.6604" fill="#02A0CF"/>
<rect x="85.3203" y="88.3164" width="22.6604" height="22.6604" fill="#02A0CF"/>
<rect x="85.3203" y="43" width="22.6604" height="22.6604" fill="#02A0CF"/>
<rect x="85.3203" y="133.645" width="22.6604" height="22.6604" fill="#02A0CF"/>
<rect x="130.641" y="88.3164" width="22.6604" height="22.6604" fill="#02A0CF"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 2.0 KiB

doc/source/ray-overview/index.md
+117
View File
@@ -0,0 +1,117 @@
(overview-overview)=
# Overview
Ray is an open-source unified framework for scaling AI and Python applications like machine learning. It provides the compute layer for parallel processing so that you don’t need to be a distributed systems expert. Ray minimizes the complexity of running your distributed individual workflows and end-to-end machine learning workflows with these components:
* Scalable libraries for common machine learning tasks such as data preprocessing, distributed training, hyperparameter tuning, reinforcement learning, and model serving.
* Pythonic distributed computing primitives for parallelizing and scaling Python applications.
* Integrations and utilities for integrating and deploying a Ray cluster with existing tools and infrastructure such as Kubernetes, AWS, GCP, and Azure.
For data scientists and machine learning practitioners, Ray lets you scale jobs without needing infrastructure expertise:
* Easily parallelize and distribute ML workloads across multiple nodes and GPUs.
* Leverage the ML ecosystem with native and extensible integrations.
For ML platform builders and ML engineers, Ray:
* Provides compute abstractions for creating a scalable and robust ML platform.
* Provides a unified ML API that simplifies onboarding and integration with the broader ML ecosystem.
* Reduces friction between development and production by enabling the same Python code to scale seamlessly from a laptop to a large cluster.
For distributed systems engineers, Ray automatically handles key processes:
* Orchestration: Managing the various components of a distributed system.
* Scheduling: Coordinating when and where tasks are executed.
* Fault tolerance: Ensuring tasks complete regardless of inevitable points of failure.
* Auto-scaling: Adjusting the number of resources allocated to dynamic demand.
## What you can do with Ray
These are some common ML workloads that individuals, organizations, and companies leverage Ray to build their AI applications:
* [Batch inference on CPUs and GPUs](project:#ref-use-cases-batch-infer)
* [Model serving](project:#ref-use-cases-model-serving)
* [Distributed training of large models](project:#ref-use-cases-distributed-training)
* [Parallel hyperparameter tuning experiments](project:#ref-use-cases-hyperparameter-tuning)
* [Reinforcement learning](project:#ref-use-cases-reinforcement-learning)
* [ML platform](project:#ref-use-cases-ml-platform)
## Ray framework
|<img src="../images/map-of-ray.svg" width="70%" loading="lazy">|
|:--:|
|Stack of Ray libraries - unified toolkit for ML workloads.|
Ray's unified compute framework consists of three layers:
1. **Ray AI Libraries**--An open-source, Python, domain-specific set of libraries that equip ML engineers, data scientists, and researchers with a scalable and unified toolkit for ML applications.
2. **Ray Core**--An open-source, Python, general purpose, distributed computing library that enables ML engineers and Python developers to scale Python applications and accelerate machine learning workloads.
3. **Ray Clusters**--A set of worker nodes connected to a common Ray head node. Ray clusters can be fixed-size, or they can autoscale up and down according to the resources requested by applications running on the cluster.
```{eval-rst}
.. grid:: 1 2 3 3
:gutter: 1
:class-container: container pb-3
.. grid-item-card::
**Scale machine learning workloads**
^^^
Build ML applications with a toolkit of libraries for distributed
:doc:`data processing <../data/data>`,
:doc:`model training <../train/train>`,
:doc:`tuning <../tune/index>`,
:doc:`reinforcement learning <../rllib/index>`,
:doc:`model serving <../serve/index>`,
and :doc:`more <../ray-more-libs/index>`.
+++
.. button-ref:: libraries-quickstart
:color: primary
:outline:
:expand:
Ray AI Libraries
.. grid-item-card::
**Build distributed applications**
^^^
Build and run distributed applications with a
:doc:`simple and flexible API <../ray-core/walkthrough>`.
:doc:`Parallelize <../ray-core/walkthrough>` single machine code with
little to zero code changes.
+++
.. button-ref:: ../ray-core/walkthrough
:color: primary
:outline:
:expand:
Ray Core
.. grid-item-card::
**Deploy large-scale workloads**
^^^
Deploy workloads on :doc:`AWS, GCP, Azure <../cluster/getting-started>` or
:doc:`on premise <../cluster/vms/user-guides/launching-clusters/on-premises>`.
Use Ray cluster managers to run Ray on existing
:doc:`Kubernetes <../cluster/kubernetes/index>`,
:doc:`YARN <../cluster/vms/user-guides/community/yarn>`,
or :doc:`Slurm <../cluster/vms/user-guides/community/slurm>` clusters.
+++
.. button-ref:: ../cluster/getting-started
:color: primary
:outline:
:expand:
Ray Clusters
```
Each of [Ray's](../ray-air/getting-started) five native libraries distributes a specific ML task:
- [Data](../data/data): Scalable, framework-agnostic data loading and transformation across training, tuning, and prediction.
- [Train](../train/train): Distributed multi-node and multi-core model training with fault tolerance that integrates with popular training libraries.
- [Tune](../tune/index): Scalable hyperparameter tuning to optimize model performance.
- [Serve](../serve/index): Scalable and programmable serving to deploy models for online inference, with optional microbatching to improve performance.
- [RLlib](../rllib/index): Scalable distributed reinforcement learning workloads.
Ray's libraries are for both data scientists and ML engineers. For data scientists, these libraries can be used to scale individual workloads and end-to-end ML applications. For ML engineers, these libraries provide scalable platform abstractions that can be used to easily onboard and integrate tooling from the broader ML ecosystem.
For custom applications, the [Ray Core](../ray-core/walkthrough) library enables Python developers to easily build scalable, distributed systems that can run on a laptop, cluster, cloud, or Kubernetes. It's the foundation that Ray AI libraries and third-party integrations (Ray ecosystem) are built on.
Ray runs on any machine, cluster, cloud provider, and Kubernetes, and features a growing [ecosystem of community integrations](ray-libraries).
doc/source/ray-overview/installation.rst
+519
View File
@@ -0,0 +1,519 @@
.. _installation:
Installing Ray
==============
.. raw:: html
<a id="try-anyscale-quickstart-install-ray" target="_blank" href="https://console.anyscale.com/register/ha?render_flow=ray&utm_source=ray_docs&utm_medium=docs&utm_campaign=installing_ray&redirectTo=/v2/template-preview/workspace-intro">
<img src="../_static/img/run-on-anyscale.svg" alt="Run Quickstart on Anyscale" />
<br/><br/>
</a>
Ray currently officially supports x86_64, aarch64 (ARM) for Linux, and Apple silicon (M1) hardware.
Ray on Windows is currently in beta.
.. warning::
**Pydantic v1 Deprecation Notice:** Pydantic v1 is deprecated and Ray will drop support for it in
version 2.56. If you're using Pydantic v1, upgrade to Pydantic v2 by running ``pip install -U pydantic``.
See `GitHub issue #58876 <https://github.com/ray-project/ray/issues/58876>`_ for more details.
Official Releases
-----------------
From Wheels
~~~~~~~~~~~
You can install the latest official version of Ray from PyPI on Linux, Windows,
and macOS by choosing the option that best matches your use case.
.. tab-set::
.. tab-item:: Recommended
**For machine learning applications**
.. code-block:: shell
pip install -U "ray[data,train,tune,serve]"
# For reinforcement learning support, install RLlib instead.
# pip install -U "ray[rllib]"
**For general Python applications**
.. code-block:: shell
pip install -U "ray[default]"
# If you don't want Ray Dashboard or Cluster Launcher, install Ray with minimal dependencies instead.
# pip install -U "ray"
.. tab-item:: Advanced
.. list-table::
:widths: 2 3
:header-rows: 1
* - Command
- Installed components
* - `pip install -U "ray"`
- Core
* - `pip install -U "ray[default]"`
- Core, Dashboard, Cluster Launcher
* - `pip install -U "ray[data]"`
- Core, Data
* - `pip install -U "ray[train]"`
- Core, Train
* - `pip install -U "ray[tune]"`
- Core, Tune
* - `pip install -U "ray[serve]"`
- Core, Dashboard, Cluster Launcher, Serve
* - `pip install -U "ray[serve-grpc]"`
- Core, Dashboard, Cluster Launcher, Serve with gRPC support
* - `pip install -U "ray[rllib]"`
- Core, Tune, RLlib
* - `pip install -U "ray[all]"`
- Core, Dashboard, Cluster Launcher, Data, Train, Tune, Serve, RLlib. This option isn't recommended. Specify the extras you need as shown below instead.
.. tip::
You can combine installation extras.
For example, to install Ray with Dashboard, Cluster Launcher, and Train support, you can run:
.. code-block:: shell
pip install -U "ray[default,train]"
.. _install-nightlies:
Daily Releases (Nightlies)
--------------------------
You can install the nightly Ray wheels via the following links. These daily releases are tested via automated tests but do not go through the full release process. To install these wheels, use the following ``pip`` command and wheels:
.. code-block:: bash
# Clean removal of previous install
pip uninstall -y ray
# Install Ray with support for the dashboard + cluster launcher
pip install -U "ray[default] @ LINK_TO_WHEEL.whl"
# Install Ray with minimal dependencies
# pip install -U LINK_TO_WHEEL.whl
.. tab-set::
.. tab-item:: Linux
=============================================== ================================================
Linux (x86_64) Linux (arm64/aarch64)
=============================================== ================================================
`Linux Python 3.10 (x86_64)`_ `Linux Python 3.10 (aarch64)`_
`Linux Python 3.11 (x86_64)`_ `Linux Python 3.11 (aarch64)`_
`Linux Python 3.12 (x86_64)`_ `Linux Python 3.12 (aarch64)`_
`Linux Python 3.13 (x86_64)`_ (beta) `Linux Python 3.13 (aarch64)`_ (beta)
=============================================== ================================================
.. tab-item:: MacOS
.. list-table::
:header-rows: 1
* - MacOS (arm64)
* - `MacOS Python 3.10 (arm64)`_
* - `MacOS Python 3.11 (arm64)`_
* - `MacOS Python 3.12 (arm64)`_
* - `MacOS Python 3.13 (arm64)`_ (beta)
.. tab-item:: Windows (beta)
.. list-table::
:header-rows: 1
* - Windows (beta)
* - `Windows Python 3.10 (amd64)`_
* - `Windows Python 3.11 (amd64)`_
* - `Windows Python 3.12 (amd64)`_
.. note::
On Windows, support for multi-node Ray clusters is currently experimental and untested.
If you run into issues please file a report at https://github.com/ray-project/ray/issues.
.. note::
:ref:`Usage stats <ref-usage-stats>` collection is enabled by default (can be :ref:`disabled <usage-disable>`) for nightly wheels including both local clusters started via ``ray.init()`` and remote clusters via cli.
.. If you change the list of wheel links below, remember to update `get_wheel_filename()` in `https://github.com/ray-project/ray/blob/master/python/ray/_private/utils.py`.
.. _`Linux Python 3.10 (x86_64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp310-cp310-manylinux2014_x86_64.whl
.. _`Linux Python 3.11 (x86_64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp311-cp311-manylinux2014_x86_64.whl
.. _`Linux Python 3.12 (x86_64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp312-cp312-manylinux2014_x86_64.whl
.. _`Linux Python 3.13 (x86_64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp313-cp313-manylinux2014_x86_64.whl
.. _`Linux Python 3.10 (aarch64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp310-cp310-manylinux2014_aarch64.whl
.. _`Linux Python 3.11 (aarch64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp311-cp311-manylinux2014_aarch64.whl
.. _`Linux Python 3.12 (aarch64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp312-cp312-manylinux2014_aarch64.whl
.. _`Linux Python 3.13 (aarch64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp313-cp313-manylinux2014_aarch64.whl
.. _`MacOS Python 3.10 (arm64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp310-cp310-macosx_12_0_arm64.whl
.. _`MacOS Python 3.11 (arm64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp311-cp311-macosx_12_0_arm64.whl
.. _`MacOS Python 3.12 (arm64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp312-cp312-macosx_12_0_arm64.whl
.. _`MacOS Python 3.13 (arm64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp313-cp313-macosx_12_0_arm64.whl
.. _`Windows Python 3.10 (amd64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp310-cp310-win_amd64.whl
.. _`Windows Python 3.11 (amd64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp311-cp311-win_amd64.whl
.. _`Windows Python 3.12 (amd64)`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp312-cp312-win_amd64.whl
Installing from a specific commit
---------------------------------
You can install the Ray wheels of any particular commit on ``master`` with the following template. You need to specify the commit hash, Ray version, Operating System, and Python version:
.. code-block:: bash
pip install https://s3-us-west-2.amazonaws.com/ray-wheels/master/{COMMIT_HASH}/ray-{RAY_VERSION}-{PYTHON_VERSION}-{PYTHON_VERSION}-{OS_VERSION}.whl
For example, here are the Ray 3.0.0.dev0 wheels for Python 3.10, MacOS for commit ``4f2ec46c3adb6ba9f412f09a9732f436c4a5d0c9``:
.. code-block:: bash
pip install https://s3-us-west-2.amazonaws.com/ray-wheels/master/4f2ec46c3adb6ba9f412f09a9732f436c4a5d0c9/ray-3.0.0.dev0-cp310-cp310-macosx_12_0_arm64.whl
There are minor variations to the format of the wheel filename; it's best to match against the format in the URLs listed in the :ref:`Nightlies section <install-nightlies>`.
Here's a summary of the variations:
* For MacOS x86_64, commits predating August 7, 2021 will have ``macosx_10_13`` in the filename instead of ``macosx_10_15``.
* For MacOS x86_64, commits predating June 1, 2025 will have ``macosx_10_15`` in the filename instead of ``macosx_12_0``.
.. _apple-silicon-support:
M1 Mac (Apple Silicon) Support
------------------------------
Ray supports machines running Apple Silicon (such as M1 macs).
Multi-node clusters are untested. To get started with local Ray development:
#. Install `miniforge <https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh>`_.
* ``wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh``
* ``bash Miniforge3-MacOSX-arm64.sh``
* ``rm Miniforge3-MacOSX-arm64.sh # Cleanup.``
#. Ensure you're using the miniforge environment (you should see (base) in your terminal).
* ``source ~/.bash_profile``
* ``conda activate``
#. Install Ray as you normally would.
* ``pip install ray``
.. _windows-support:
Windows Support
---------------
Windows support is in Beta. Ray supports running on Windows with the following caveats (only the first is
Ray-specific, the rest are true anywhere Windows is used):
* Multi-node Ray clusters are untested.
* Filenames are tricky on Windows and there still may be a few places where Ray
assumes UNIX filenames rather than Windows ones. This can be true in downstream
packages as well.
* Performance on Windows is known to be slower since opening files on Windows
is considerably slower than on other operating systems. This can affect logging.
* Windows does not have a copy-on-write forking model, so spinning up new
processes can require more memory.
Submit any issues you encounter to
`GitHub <https://github.com/ray-project/ray/issues/>`_.
Installing Ray on Arch Linux
----------------------------
Note: Installing Ray on Arch Linux is not tested by the Project Ray developers.
Ray is available on Arch Linux via the Arch User Repository (`AUR`_) as
``python-ray``.
You can manually install the package by following the instructions on the
`Arch Wiki`_ or use an `AUR helper`_ like `yay`_ (recommended for ease of install)
as follows:
.. code-block:: bash
yay -S python-ray
To discuss any issues related to this package refer to the comments section
on the AUR page of ``python-ray`` `here`_.
.. _`AUR`: https://wiki.archlinux.org/index.php/Arch_User_Repository
.. _`Arch Wiki`: https://wiki.archlinux.org/index.php/Arch_User_Repository#Installing_packages
.. _`AUR helper`: https://wiki.archlinux.org/index.php/Arch_User_Repository#Installing_packages
.. _`yay`: https://aur.archlinux.org/packages/yay
.. _`here`: https://aur.archlinux.org/packages/python-ray
.. _ray_anaconda:
Installing From conda-forge
---------------------------
Ray can also be installed as a conda package on Linux and Windows.
.. code-block:: bash
# also works with mamba
conda create -c conda-forge python=3.10 -n ray
conda activate ray
# Install Ray with support for the dashboard + cluster launcher
conda install -c conda-forge "ray-default"
# Install Ray with minimal dependencies
# conda install -c conda-forge ray
To install Ray libraries, use ``pip`` as above or ``conda``/``mamba``.
.. code-block:: bash
conda install -c conda-forge "ray-data" # installs Ray + dependencies for Ray Data
conda install -c conda-forge "ray-train" # installs Ray + dependencies for Ray Train
conda install -c conda-forge "ray-tune" # installs Ray + dependencies for Ray Tune
conda install -c conda-forge "ray-serve" # installs Ray + dependencies for Ray Serve
conda install -c conda-forge "ray-rllib" # installs Ray + dependencies for Ray RLlib
For a complete list of available ``ray`` libraries on Conda-forge, have a look
at https://anaconda.org/conda-forge/ray-default
.. note::
Ray conda packages are maintained by the community, not the Ray team. While
using a conda environment, it is recommended to install Ray from PyPi using
`pip install ray` in the newly created environment.
Building Ray from Source
------------------------
Installing from ``pip`` should be sufficient for most Ray users.
However, should you need to build from source, follow :ref:`these instructions for building <building-ray>` Ray.
.. _docker-images:
Docker Source Images
--------------------
Users can pull a Docker image from the ``rayproject/ray`` `Docker Hub repository <https://hub.docker.com/r/rayproject/ray>`__.
The images include Ray and all required dependencies. It comes with anaconda and various versions of Python.
Images are `tagged` with the format ``{Ray version}[-{Python version}][-{Platform}]``. ``Ray version`` tag can be one of the following:
.. list-table::
:widths: 25 50
:header-rows: 1
* - Ray version tag
- Description
* - latest
- The most recent Ray release.
* - x.y.z
- A specific Ray release, e.g. 2.31.0
* - nightly
- The most recent Ray development build (a recent commit from Github ``master``)
The optional ``Python version`` tag specifies the Python version in the image. All Python versions supported by Ray are available, e.g. ``py310``, ``py311`` and ``py312``. If unspecified, the tag points to an image of the lowest Python version that the Ray version supports.
The optional ``Platform`` tag specifies the platform where the image is intended for:
.. list-table::
:widths: 16 40
:header-rows: 1
* - Platform tag
- Description
* - -cpu
- These are based off of an Ubuntu image.
* - -cuXX
- These are based off of an NVIDIA CUDA image with the specified CUDA version. They require the NVIDIA Docker Runtime.
* - -gpu
- Aliases to a specific ``-cuXX`` tagged image.
* - <no tag>
- Aliases to ``-cpu`` tagged images.
Example: for the nightly image based on ``Python 3.10`` and without GPU support, the tag is ``nightly-py310-cpu``.
If you want to tweak some aspects of these images and build them locally, refer to the following script:
.. code-block:: bash
cd ray
./build-docker.sh
Review images by listing them:
.. code-block:: bash
docker images
Output should look something like the following:
.. code-block:: bash
REPOSITORY TAG IMAGE ID CREATED SIZE
rayproject/ray dev 7243a11ac068 2 days ago 1.11 GB
rayproject/base-deps latest 5606591eeab9 8 days ago 512 MB
ubuntu 22.04 1e4467b07108 3 weeks ago 73.9 MB
Launch Ray in Docker
~~~~~~~~~~~~~~~~~~~~
Start out by launching the deployment container.
.. code-block:: bash
docker run --shm-size=<shm-size> -t -i rayproject/ray
Replace ``<shm-size>`` with a limit appropriate for your system, for example
``512M`` or ``2G``. A good estimate for this is to use roughly 30% of your available memory (this is
what Ray uses internally for its Object Store). The ``-t`` and ``-i`` options here are required to support
interactive use of the container.
If you use a GPU version Docker image, remember to add ``--gpus all`` option. Replace ``<ray-version>`` with your target ray version in the following command:
.. code-block:: bash
docker run --shm-size=<shm-size> -t -i --gpus all rayproject/ray:<ray-version>-gpu
**Note:** Ray requires a **large** amount of shared memory because each object
store keeps all of its objects in shared memory, so the amount of shared memory
will limit the size of the object store.
You should now see a prompt that looks something like:
.. code-block:: bash
root@ebc78f68d100:/ray#
Test if the installation succeeded
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To test if the installation was successful, try running some tests. This assumes
that you've cloned the git repository.
.. code-block:: bash
python -m pytest -v python/ray/tests/test_mini.py
Installed Python dependencies
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Our docker images are shipped with pre-installed Python dependencies
required for Ray and its libraries.
We publish the dependencies that are installed in our ``ray`` Docker images for Python 3.10.
.. tab-set::
.. tab-item:: ray (Python 3.10)
:sync: ray (Python 3.10)
Ray version: nightly (`cf3939d <https://github.com/ray-project/ray/commit/cf3939d61ace058f973c1bd3e7166c2bbc8a69a3>`_)
.. literalinclude:: ./pip_freeze_ray-py310-cpu.txt
.. _ray-install-java:
Install Ray Java with Maven
---------------------------
.. note::
All Ray Java APIs are experimental and only supported by the community.
Before installing Ray Java with Maven, you should install Ray Python with `pip install -U ray` . Note that the versions of Ray Java and Ray Python must match.
Note that nightly Ray python wheels are also required if you want to install Ray Java snapshot version.
Find the latest Ray Java release in the `central repository <https://mvnrepository.com/artifact/io.ray>`__. To use the latest Ray Java release in your application, add the following entries in your ``pom.xml``:
.. code-block:: xml
<dependency>
<groupId>io.ray</groupId>
<artifactId>ray-api</artifactId>
<version>${ray.version}</version>
</dependency>
<dependency>
<groupId>io.ray</groupId>
<artifactId>ray-runtime</artifactId>
<version>${ray.version}</version>
</dependency>
The latest Ray Java snapshot can be found in `sonatype repository <https://oss.sonatype.org/#nexus-search;quick~io.ray>`__. To use the latest Ray Java snapshot in your application, add the following entries in your ``pom.xml``:
.. code-block:: xml
<!-- only needed for snapshot version of ray -->
<repositories>
<repository>
<id>sonatype</id>
<url>https://oss.sonatype.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>io.ray</groupId>
<artifactId>ray-api</artifactId>
<version>${ray.version}</version>
</dependency>
<dependency>
<groupId>io.ray</groupId>
<artifactId>ray-runtime</artifactId>
<version>${ray.version}</version>
</dependency>
</dependencies>
.. note::
When you run ``pip install`` to install Ray, Java jars are installed as well. The above dependencies are only used to build your Java code and to run your code in local mode.
If you want to run your Java code in a multi-node Ray cluster, it's better to exclude Ray jars when packaging your code to avoid jar conflicts if the versions (installed Ray with ``pip install`` and maven dependencies) don't match.
.. _ray-install-cpp:
Install Ray C++
---------------
.. note::
All Ray C++ APIs are experimental and only supported by the community.
You can install and use Ray C++ API as follows.
.. code-block:: bash
pip install -U ray[cpp]
# Create a Ray C++ project template to start with.
ray cpp --generate-bazel-project-template-to ray-template
.. note::
If you build Ray from source, remove the build option ``build --cxxopt="-D_GLIBCXX_USE_CXX11_ABI=0"`` from the file ``cpp/example/.bazelrc`` before running your application. The related issue is `this <https://github.com/ray-project/ray/issues/26031>`_.
doc/source/ray-overview/pip_freeze_ray-py310-cpu.txt
+168
View File
@@ -0,0 +1,168 @@
adlfs==2023.8.0
aiohappyeyeballs==2.6.1
aiohttp==3.13.3
aiohttp-cors==0.7.0
aiosignal==1.4.0
amqp==5.3.1
annotated-doc==0.0.4
annotated-types==0.6.0
anyio==4.12.0
archspec @ file:///home/conda/feedstock_root/build_artifacts/archspec_1737352602016/work
async-timeout==5.0.1
attrs==25.1.0
azure-common==1.1.28
azure-core==1.29.5
azure-datalake-store==0.0.53
azure-identity==1.17.1
azure-storage-blob==12.22.0
billiard==4.2.1
boltons @ file:///home/conda/feedstock_root/build_artifacts/boltons_1733827268945/work
boto3==1.29.7
botocore==1.32.7
Brotli @ file:///home/conda/feedstock_root/build_artifacts/brotli-split_1764016952863/work
cachetools==5.5.2
celery==5.5.3
certifi==2025.1.31
cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1725560520483/work
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.1
click-plugins==1.1.1.2
click-repl==0.3.0
cloudpickle==2.2.0
colorama @ file:///home/conda/feedstock_root/build_artifacts/colorama_1733218098505/work
colorful==0.5.5
conda @ file:///home/conda/feedstock_root/build_artifacts/conda_1775855674855/work/conda-src
conda-libmamba-solver @ file:///home/conda/feedstock_root/build_artifacts/conda-libmamba-solver_1775557888859/work/src
conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1736345463896/work
conda_package_streaming @ file:///home/conda/feedstock_root/build_artifacts/conda-package-streaming_1729004031731/work
cryptography==44.0.3
cupy-cuda12x==13.4.0
Cython==0.29.37
distlib==0.3.7
distro @ file:///home/conda/feedstock_root/build_artifacts/distro_1734729835256/work
dm-tree==0.1.8
exceptiongroup==1.3.1
Farama-Notifications==0.0.4
fastapi==0.121.0
fastrlock==0.8.3
filelock==3.17.0
flatbuffers==23.5.26
frozendict @ file:///home/conda/feedstock_root/build_artifacts/frozendict_1763082794572/work
frozenlist==1.4.1
fsspec==2023.12.1
google-api-core==2.24.2
google-api-python-client==2.111.0
google-auth==2.23.4
google-auth-httplib2==0.1.1
google-cloud-core==2.4.1
google-cloud-storage==2.14.0
google-crc32c==1.5.0
google-oauth==1.0.1
google-resumable-media==2.6.0
googleapis-common-protos==1.61.0
grpcio==1.74.0
gymnasium==1.2.2
h11==0.16.0
h2 @ file:///home/conda/feedstock_root/build_artifacts/h2_1733298745555/work
hpack @ file:///home/conda/feedstock_root/build_artifacts/hpack_1733299205993/work
httplib2==0.20.4
httptools==0.7.1
hyperframe @ file:///home/conda/feedstock_root/build_artifacts/hyperframe_1733298771451/work
idna==3.7
importlib-metadata==6.11.0
isodate==0.6.1
izulu==0.50.0
Jinja2==3.1.6
jmespath==1.0.1
jsonpatch @ file:///home/conda/feedstock_root/build_artifacts/jsonpatch_1733814567314/work
jsonpointer @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_jsonpointer_1774311356/work
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
kombu==5.5.4
libmambapy @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_libmambapy_1767884157/work/libmambapy
linkify-it-py==2.0.3
lz4==4.4.5
markdown-it-py==2.2.0
MarkupSafe==2.1.3
mdit-py-plugins==0.3.5
mdurl==0.1.2
memray==1.19.1
menuinst @ file:///home/conda/feedstock_root/build_artifacts/menuinst_1765733081264/work
msal==1.28.1
msal-extensions==1.2.0b1
msgpack @ file:///home/conda/feedstock_root/build_artifacts/msgpack-python_1762503960095/work
multidict==6.0.5
numpy==1.26.4
opencensus==0.11.4
opencensus-context==0.1.3
opentelemetry-api==1.34.1
opentelemetry-exporter-prometheus==0.55b1
opentelemetry-proto==1.27.0
opentelemetry-sdk==1.34.1
opentelemetry-semantic-conventions==0.55b1
ormsgpack==1.7.0
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1733203243479/work
pandas==2.3.3
platformdirs==3.11.0
pluggy @ file:///home/conda/feedstock_root/build_artifacts/pluggy_1733222765875/work
portalocker==2.8.2
prometheus-client==0.19.0
prompt-toolkit==3.0.41
propcache==0.3.0
proto-plus==1.22.3
protobuf==4.25.8
psutil==5.9.6
py-spy==0.4.1
pyarrow==19.0.1
pyasn1==0.5.1
pyasn1-modules==0.3.0
pycosat @ file:///home/conda/feedstock_root/build_artifacts/pycosat_1757744612102/work
pycparser==2.21
pycron==3.2.0
pydantic==2.12.4
pydantic_core==2.41.5
Pygments==2.18.0
PyJWT==2.8.0
pyOpenSSL==25.0.0
pyparsing==3.1.1
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1733217236728/work
python-dateutil==2.8.2
python-dotenv==1.2.1
pytz==2022.7.1
PyYAML==6.0.3
ray @ file:///home/ray/ray-3.0.0.dev0-cp310-cp310-manylinux2014_x86_64.whl#sha256=914c129fd383540b13859722e91b9b23866d6d5bbc92d6cc1be2682c3782ebc7
referencing==0.36.2
requests==2.32.5
rich==13.7.1
rpds-py==0.22.3
rsa==4.7.2
ruamel.yaml @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_ruamel.yaml_1766175793/work
ruamel.yaml.clib @ file:///home/conda/feedstock_root/build_artifacts/bld/rattler-build_ruamel.yaml.clib_1766159514/work
s3transfer==0.8.0
scipy==1.11.4
six==1.16.0
smart-open==6.2.0
starlette==0.49.1
taskiq==0.12.1
taskiq-dependencies==1.5.7
tensorboardX==2.6.2.2
textual==4.0.0
tqdm @ file:///home/conda/feedstock_root/build_artifacts/tqdm_1735661334605/work
truststore @ file:///home/conda/feedstock_root/build_artifacts/truststore_1729762363021/work
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.2
uc-micro-py==1.0.3
uritemplate==4.1.1
urllib3==1.26.19
uvicorn==0.22.0
uvloop==0.22.1
vine==5.1.0
virtualenv==20.29.1
watchfiles==0.19.0
wcwidth==0.2.13
websockets==11.0.3
yarl==1.18.3
zipp==3.19.2
zstandard==0.25.0
doc/source/ray-overview/ray-libraries.rst
+375
View File
@@ -0,0 +1,375 @@
.. _ray-oss-list:
The Ray Ecosystem
=================
This page lists libraries that have integrations with Ray for distributed execution
in alphabetical order.
It's easy to add your own integration to this list.
Simply open a pull request with a few lines of text, see the dropdown below for
more information.
.. dropdown:: Adding Your Integration
To add an integration add an entry to this file, using the same
``grid-item-card`` directive that the other examples use.
.. grid:: 1 2 2 3
:gutter: 1
:class-container: container pb-3
.. grid-item-card::
.. figure:: ../images/aibrix.jpeg
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/vllm-project/aibrix?style=social
:target: https://github.com/vllm-project/aibrix
AIBrix is a cloud-native LLM inference infrastructure platform that provides building blocks for deploying, scaling, and optimizing large language model serving with Ray-based hybrid orchestration.
+++
.. button-link:: https://github.com/vllm-project/aibrix
:color: primary
:outline:
:expand:
AIBrix Integration
.. grid-item-card::
.. figure:: ../images/areal.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/areal-project/AReaL?style=social
:target: https://github.com/areal-project/AReaL
AReaL is an asynchronous reinforcement learning system for LLM agents developed by Ant Group. It decouples generation from training for efficient distributed post-training on Ray clusters.
+++
.. button-link:: https://github.com/areal-project/AReaL
:color: primary
:outline:
:expand:
AReaL Integration
.. grid-item-card::
.. figure:: ../images/cosmos_curate.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/NVIDIA/cosmos-curator?style=social
:target: https://github.com/NVIDIA/cosmos-curator
Cosmos Curate is a GPU-accelerated video and image data curation toolkit from NVIDIA. It provides scalable pipelines for filtering, deduplication, and quality scoring using Ray for multi-node, multi-GPU distributed processing.
+++
.. button-link:: https://github.com/NVIDIA/cosmos-curator
:color: primary
:outline:
:expand:
Cosmos Curate Integration
.. grid-item-card::
.. figure:: ../images/daft.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/Eventual-Inc/Daft?style=social
:target: https://github.com/Eventual-Inc/Daft
Daft is a high-performance multimodal data engine that provides simple and reliable data processing for any modality - from structured tables to images, audio, video, and embeddings. Built with Python and Rust for modern AI workflows, Daft offers seamless scaling from local to distributed clusters, enabling efficient batch inference, document processing, and multimodal ETL pipelines at scale.
+++
.. button-link:: https://docs.daft.ai/en/stable/distributed/ray/
:color: primary
:outline:
:expand:
Daft Integration
.. grid-item-card::
.. figure:: ../images/data_juicer.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/modelscope/data-juicer?style=social
:target: https://github.com/modelscope/data-juicer
Data-Juicer is a one-stop multimodal data processing system to make data higher-quality, juicier, and more digestible for foundation models. It integrates with Ray for distributed data processing on large-scale datasets with over 100 multimodal operators and supports TB-size dataset deduplication.
+++
.. button-link:: https://github.com/modelscope/data-juicer?tab=readme-ov-file#distributed-data-processing
:color: primary
:outline:
:expand:
Data-Juicer Integration
.. grid-item-card::
.. figure:: ../images/deltacat.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/ray-project/deltacat?style=social
:target: https://github.com/ray-project/deltacat
DeltaCAT is a portable multimodal lakehouse powered by Ray for petabyte-scale data compaction, deduplication, and incremental table processing with ACID compliance.
+++
.. button-link:: https://github.com/ray-project/deltacat
:color: primary
:outline:
:expand:
DeltaCAT Integration
.. grid-item-card::
.. figure:: ../images/modin.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/modin-project/modin?style=social
:target: https://github.com/modin-project/modin
Scale your pandas workflows by changing one line of code. Modin transparently distributes the data and computation so that all you need to do is continue using the pandas API as you were before installing Modin.
+++
.. button-link:: https://github.com/modin-project/modin
:color: primary
:outline:
:expand:
Modin Integration
.. grid-item-card::
.. figure:: ../images/nemo.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/NVIDIA-NeMo/Curator?style=social
:target: https://github.com/NVIDIA-NeMo/Curator
NeMo Curator is a scalable data curation toolkit from NVIDIA for preparing high-quality datasets for large language model training. It uses Ray for distributed data processing including deduplication, filtering, and quality classification at scale.
+++
.. button-link:: https://github.com/NVIDIA-NeMo/Curator
:color: primary
:outline:
:expand:
NeMo Curator Integration
.. grid-item-card::
.. figure:: ../images/nemo.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/NVIDIA-NeMo/RL?style=social
:target: https://github.com/NVIDIA-NeMo/RL
NeMo-RL is NVIDIA's scalable post-training toolkit for large language models. It provides RLHF and alignment training built on Ray for distributed orchestration of training and inference workloads.
+++
.. button-link:: https://github.com/NVIDIA-NeMo/RL
:color: primary
:outline:
:expand:
NeMo-RL Integration
.. grid-item-card::
.. figure:: ../images/openrlhf.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/OpenRLHF/OpenRLHF?style=social
:target: https://github.com/OpenRLHF/OpenRLHF
OpenRLHF is an easy-to-use, scalable RLHF training framework. It supports distributed PPO, DPO, rejection sampling, and other alignment methods using Ray for orchestrating training and generation across multiple GPUs and nodes.
+++
.. button-link:: https://github.com/OpenRLHF/OpenRLHF
:color: primary
:outline:
:expand:
OpenRLHF Integration
.. grid-item-card::
.. figure:: ../images/intel.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/Intel-bigdata/oap-raydp?style=social
:target: https://github.com/Intel-bigdata/oap-raydp
RayDP ("Spark on Ray") enables you to easily use Spark inside a Ray program. You can use Spark to read the input data, process the data using SQL, Spark DataFrame, or Pandas (via Koalas) API, extract and transform features using Spark MLLib, and use RayDP Estimator API for distributed training on the preprocessed dataset.
+++
.. button-link:: https://github.com/Intel-bigdata/oap-raydp
:color: primary
:outline:
:expand:
RayDP Integration
.. grid-item-card::
.. figure:: ../images/roll.jpg
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/alibaba/ROLL?style=social
:target: https://github.com/alibaba/ROLL
ROLL is Alibaba's reinforcement learning scaling library for large language models. It provides efficient distributed RL training with flexible resource scheduling and heterogeneous task management built on Ray.
+++
.. button-link:: https://github.com/alibaba/ROLL
:color: primary
:outline:
:expand:
ROLL Integration
.. grid-item-card::
.. figure:: ../images/skyrl.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/NovaSky-AI/SkyRL?style=social
:target: https://github.com/NovaSky-AI/SkyRL
SkyRL is a modular reinforcement learning library for LLM agents from UC Berkeley. It enables training through multi-turn environment interactions using Ray for distributed rollout and training.
+++
.. button-link:: https://github.com/NovaSky-AI/SkyRL
:color: primary
:outline:
:expand:
SkyRL Integration
.. grid-item-card::
.. figure:: ../images/slime.jpg
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/THUDM/slime?style=social
:target: https://github.com/THUDM/slime
SLIME is a post-training framework for large language models from Tsinghua University. It provides RL scaling with a service-oriented architecture built on Ray and Megatron-LM for distributed training orchestration.
+++
.. button-link:: https://github.com/THUDM/slime
:color: primary
:outline:
:expand:
SLIME Integration
.. grid-item-card::
.. figure:: ../images/syftr.jpg
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/datarobot/syftr?style=social
:target: https://github.com/datarobot/syftr
Syftr is an open-source agent workflow optimizer from DataRobot. It uses Ray and Ray Tune for scalable multi-objective optimization of agentic AI workflows across prompts, models, and tool selections.
+++
.. button-link:: https://github.com/datarobot/syftr
:color: primary
:outline:
:expand:
Syftr Integration
.. grid-item-card::
.. figure:: ../images/verl.jpg
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/verl-project/verl?style=social
:target: https://github.com/verl-project/verl
verl is a flexible and efficient reinforcement learning training library for large language models from ByteDance. It provides a Ray-native hybrid controller for scalable RLHF training with distributed orchestration of rollout, training, and reward computation.
+++
.. button-link:: https://github.com/verl-project/verl
:color: primary
:outline:
:expand:
verl Integration
.. grid-item-card::
.. figure:: ../images/vllm.png
:class: card-figure
.. div::
.. image:: https://img.shields.io/github/stars/vllm-project/vllm?style=social
:target: https://github.com/vllm-project/vllm
vLLM is a high-throughput and memory-efficient inference and serving engine for large language models. It uses Ray for distributed tensor parallelism and pipeline parallelism across multiple GPUs and nodes.
+++
.. button-link:: https://docs.vllm.ai/
:color: primary
:outline:
:expand:
vLLM Integration
doc/source/ray-overview/use-cases.rst
+184
View File
@@ -0,0 +1,184 @@
.. _ref-use-cases:
Ray Use Cases
=============
.. toctree::
:hidden:
../ray-air/getting-started
This page indexes common Ray use cases for scaling ML.
It contains highlighted references to blogs, examples, and tutorials also located
elsewhere in the Ray documentation.
.. _ref-use-cases-llm:
LLMs and Gen AI
---------------
Large language models (LLMs) and generative AI are rapidly changing industries, and demand compute at an astonishing pace. Ray provides a distributed compute framework for scaling these models, allowing developers to train and deploy models faster and more efficiently. With specialized libraries for data streaming, training, fine-tuning, hyperparameter tuning, and serving, Ray simplifies the process of developing and deploying large-scale AI models.
.. figure:: /images/llm-stack.png
.. query-param-ref:: ray-overview/examples
:parameters: ?tags=llm
:ref-type: doc
:classes: example-gallery-link
.. raw:: html
<svg width='24' height='24' viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg' class='star-icon'>
<g>
<path class='star-icon-path' d='M15.199 9.945C14.7653 9.53412 14.4863 8.98641 14.409 8.394L14.006 5.311L11.276 6.797C10.7511 7.08302 10.1436 7.17943 9.55597 7.07L6.49997 6.5L7.06997 9.556C7.1794 10.1437 7.08299 10.7511 6.79697 11.276L5.31097 14.006L8.39397 14.409C8.98603 14.4865 9.53335 14.7655 9.94397 15.199L12.082 17.456L13.418 14.649C13.6744 14.1096 14.1087 13.6749 14.648 13.418L17.456 12.082L15.199 9.945ZM15.224 15.508L13.011 20.158C12.9691 20.2459 12.9065 20.3223 12.8285 20.3806C12.7505 20.4389 12.6594 20.4774 12.5633 20.4926C12.4671 20.5079 12.3686 20.4995 12.2764 20.4682C12.1842 20.4369 12.101 20.3836 12.034 20.313L8.49197 16.574C8.39735 16.4742 8.27131 16.41 8.13497 16.392L3.02797 15.724C2.93149 15.7113 2.83954 15.6753 2.76006 15.6191C2.68058 15.563 2.61596 15.4883 2.57177 15.4016C2.52758 15.3149 2.50514 15.2187 2.5064 15.1214C2.50765 15.0241 2.53256 14.9285 2.57897 14.843L5.04097 10.319C5.10642 10.198 5.12831 10.0582 5.10297 9.923L4.15997 4.86C4.14207 4.76417 4.14778 4.66541 4.17662 4.57229C4.20546 4.47916 4.25656 4.39446 4.3255 4.32553C4.39444 4.25659 4.47913 4.20549 4.57226 4.17665C4.66539 4.14781 4.76414 4.14209 4.85997 4.16L9.92297 5.103C10.0582 5.12834 10.198 5.10645 10.319 5.041L14.843 2.579C14.9286 2.53257 15.0242 2.50769 15.1216 2.50648C15.219 2.50528 15.3152 2.52781 15.4019 2.57211C15.4887 2.61641 15.5633 2.68116 15.6194 2.76076C15.6755 2.84036 15.7114 2.93242 15.724 3.029L16.392 8.135C16.4099 8.27134 16.4742 8.39737 16.574 8.492L20.313 12.034C20.3836 12.101 20.4369 12.1842 20.4682 12.2765C20.4995 12.3687 20.5079 12.4671 20.4926 12.5633C20.4774 12.6595 20.4389 12.7505 20.3806 12.8285C20.3223 12.9065 20.2459 12.9691 20.158 13.011L15.508 15.224C15.3835 15.2832 15.2832 15.3835 15.224 15.508ZM16.021 17.435L17.435 16.021L21.678 20.263L20.263 21.678L16.021 17.435Z'> </path>
</g>
</svg>Explore LLMs and Gen AI examples
.. _ref-use-cases-batch-infer:
Batch Inference
---------------
Batch inference is the process of generating model predictions on a large "batch" of input data.
Ray for batch inference works with any cloud provider and ML framework,
and is fast and cheap for modern deep learning applications.
It scales from single machines to large clusters with minimal code changes.
As a Python-first framework, you can easily express and interactively develop your inference workloads in Ray.
To learn more about running batch inference with Ray, see the :ref:`batch inference guide<batch_inference_home>`.
.. figure:: ../data/images/batch_inference.png
.. query-param-ref:: ray-overview/examples
:parameters: ?tags=inference
:ref-type: doc
:classes: example-gallery-link
.. raw:: html
<svg width='24' height='24' viewBox='0 0 24 24' xmlns='http://www.w3.org/2000/svg' class='star-icon'>
<g>
<path class='star-icon-path' d='M15.199 9.945C14.7653 9.53412 14.4863 8.98641 14.409 8.394L14.006 5.311L11.276 6.797C10.7511 7.08302 10.1436 7.17943 9.55597 7.07L6.49997 6.5L7.06997 9.556C7.1794 10.1437 7.08299 10.7511 6.79697 11.276L5.31097 14.006L8.39397 14.409C8.98603 14.4865 9.53335 14.7655 9.94397 15.199L12.082 17.456L13.418 14.649C13.6744 14.1096 14.1087 13.6749 14.648 13.418L17.456 12.082L15.199 9.945ZM15.224 15.508L13.011 20.158C12.9691 20.2459 12.9065 20.3223 12.8285 20.3806C12.7505 20.4389 12.6594 20.4774 12.5633 20.4926C12.4671 20.5079 12.3686 20.4995 12.2764 20.4682C12.1842 20.4369 12.101 20.3836 12.034 20.313L8.49197 16.574C8.39735 16.4742 8.27131 16.41 8.13497 16.392L3.02797 15.724C2.93149 15.7113 2.83954 15.6753 2.76006 15.6191C2.68058 15.563 2.61596 15.4883 2.57177 15.4016C2.52758 15.3149 2.50514 15.2187 2.5064 15.1214C2.50765 15.0241 2.53256 14.9285 2.57897 14.843L5.04097 10.319C5.10642 10.198 5.12831 10.0582 5.10297 9.923L4.15997 4.86C4.14207 4.76417 4.14778 4.66541 4.17662 4.57229C4.20546 4.47916 4.25656 4.39446 4.3255 4.32553C4.39444 4.25659 4.47913 4.20549 4.57226 4.17665C4.66539 4.14781 4.76414 4.14209 4.85997 4.16L9.92297 5.103C10.0582 5.12834 10.198 5.10645 10.319 5.041L14.843 2.579C14.9286 2.53257 15.0242 2.50769 15.1216 2.50648C15.219 2.50528 15.3152 2.52781 15.4019 2.57211C15.4887 2.61641 15.5633 2.68116 15.6194 2.76076C15.6755 2.84036 15.7114 2.93242 15.724 3.029L16.392 8.135C16.4099 8.27134 16.4742 8.39737 16.574 8.492L20.313 12.034C20.3836 12.101 20.4369 12.1842 20.4682 12.2765C20.4995 12.3687 20.5079 12.4671 20.4926 12.5633C20.4774 12.6595 20.4389 12.7505 20.3806 12.8285C20.3223 12.9065 20.2459 12.9691 20.158 13.011L15.508 15.224C15.3835 15.2832 15.2832 15.3835 15.224 15.508ZM16.021 17.435L17.435 16.021L21.678 20.263L20.263 21.678L16.021 17.435Z'> </path>
</g>
</svg>Explore batch inference examples
.. _ref-use-cases-model-serving:
Model Serving
-------------
:ref:`Ray Serve <rayserve>` is well suited for model composition, enabling you to build a complex inference service consisting of multiple ML models and business logic all in Python code.
It supports complex `model deployment patterns <https://www.youtube.com/watch?v=mM4hJLelzSw>`_ requiring the orchestration of multiple Ray actors, where different actors provide inference for different models. Serve handles both batch and online inference and can scale to thousands of models in production.
.. figure:: /images/multi_model_serve.png
Deployment patterns with Ray Serve. (Click image to enlarge.)
Learn more about model serving with the following resources.
- `[Talk] Productionizing ML at Scale with Ray Serve <https://www.youtube.com/watch?v=UtH-CMpmxvI>`_
- `[Blog] Simplify your MLOps with Ray & Ray Serve <https://www.anyscale.com/blog/simplify-your-mlops-with-ray-and-ray-serve>`_
- :doc:`[Guide] Getting Started with Ray Serve </serve/getting_started>`
- :doc:`[Guide] Model Composition in Serve </serve/model_composition>`
- :doc:`[Gallery] Serve Examples Gallery </serve/examples>`
- `[Gallery] More Serve Use Cases on the Blog <https://www.anyscale.com/blog?tag=ray_serve>`_
.. _ref-use-cases-hyperparameter-tuning:
Hyperparameter Tuning
---------------------
The :ref:`Ray Tune <tune-main>` library enables any parallel Ray workload to be run under a hyperparameter tuning algorithm.
Running multiple hyperparameter tuning experiments is a pattern apt for distributed computing because each experiment is independent of one another. Ray Tune handles the hard bit of distributing hyperparameter optimization and makes available key features such as checkpointing the best result, optimizing scheduling, and specifying search patterns.
.. figure:: /images/tuning_use_case.png
Distributed tuning with distributed training per trial.
Learn more about the Tune library with the following talks and user guides.
- :doc:`[Guide] Getting Started with Ray Tune </tune/getting-started>`
- `[Blog] How to distribute hyperparameter tuning with Ray Tune <https://www.anyscale.com/blog/how-to-distribute-hyperparameter-tuning-using-ray-tune>`_
- `[Talk] Simple Distributed Hyperparameter Optimization <https://www.youtube.com/watch?v=KgYZtlbFYXE>`_
- `[Blog] Hyperparameter Search with 🤗 Transformers <https://www.anyscale.com/blog/hyperparameter-search-hugging-face-transformers-ray-tune>`_
- :doc:`[Gallery] Ray Tune Examples Gallery </tune/examples/index>`
- `More Tune use cases on the Blog <https://www.anyscale.com/blog?tag=ray-tune>`_
.. _ref-use-cases-distributed-training:
Distributed Training
--------------------
The :ref:`Ray Train <train-docs>` library integrates many distributed training frameworks under a simple Trainer API,
providing distributed orchestration and management capabilities out of the box.
In contrast to training many models, model parallelism partitions a large model across many machines for training. Ray Train has built-in abstractions for distributing shards of models and running training in parallel.
.. figure:: /images/model_parallelism.png
Model parallelism pattern for distributed large model training.
Learn more about the Train library with the following talks and user guides.
- `[Talk] Ray Train, PyTorch, TorchX, and distributed deep learning <https://www.youtube.com/watch?v=e-A93QftCfc>`_
- `[Blog] Elastic Distributed Training with XGBoost on Ray <https://www.uber.com/blog/elastic-xgboost-ray/>`_
- :doc:`[Guide] Getting Started with Ray Train </train/train>`
- :doc:`[Example] Fine-tune a 🤗 Transformers model </train/examples/transformers/huggingface_text_classification>`
- :doc:`[Gallery] Ray Train Examples Gallery </train/examples>`
- `[Gallery] More Train Use Cases on the Blog <https://www.anyscale.com/blog?tag=ray_train>`_
.. _ref-use-cases-reinforcement-learning:
Reinforcement Learning
----------------------
RLlib is an open-source library for reinforcement learning (RL), offering support for production-level, highly distributed RL workloads while maintaining unified and simple APIs for a large variety of industry applications. RLlib is used by industry leaders in many different verticals, such as climate control, industrial control, manufacturing and logistics, finance, gaming, automobile, robotics, boat design, and many others.
.. figure:: /images/rllib_use_case.png
Decentralized distributed proximal policy optimization (DD-PPO) architecture.
Learn more about reinforcement learning with the following resources.
- `[Course] Applied Reinforcement Learning with RLlib <https://applied-rl-course.netlify.app/>`_
- `[Blog] Intro to RLlib: Example Environments <https://medium.com/distributed-computing-with-ray/intro-to-rllib-example-environments-3a113f532c70>`_
- :doc:`[Guide] Getting Started with RLlib </rllib/getting-started>`
- `[Talk] Deep reinforcement learning at Riot Games <https://www.anyscale.com/events/2022/03/29/deep-reinforcement-learning-at-riot-games>`_
- :doc:`[Gallery] RLlib Examples Gallery </rllib/rllib-examples>`
- `[Gallery] More RL Use Cases on the Blog <https://www.anyscale.com/blog?tag=rllib>`_
.. _ref-use-cases-ml-platform:
ML Platform
-----------
Ray and its AI libraries provide unified compute runtime for teams looking to simplify their ML platform.
Ray's libraries such as Ray Train, Ray Data, and Ray Serve can be used to compose end-to-end ML workflows, providing features and APIs for
data preprocessing as part of training, and transitioning from training to serving.
Read more about building ML platforms with Ray in :ref:`this section <ray-for-ml-infra>`.
..
https://docs.google.com/drawings/d/1PFA0uJTq7SDKxzd7RHzjb5Sz3o1WvP13abEJbD0HXTE/edit
.. image:: /images/ray-air.svg
End-to-End ML Workflows
-----------------------
The following highlights examples utilizing Ray AI libraries to implement end-to-end ML workflows.
- :doc:`[Example] Text classification with Ray </train/examples/transformers/huggingface_text_classification>`
- :doc:`[Example] Object detection with Ray </train/examples/pytorch/torch_detection>`
- :doc:`[Example] Machine learning on tabular data </_collections/ray-overview/examples/e2e-xgboost/README>`
Large Scale Workload Orchestration
----------------------------------
The following highlights feature projects leveraging Ray Core's distributed APIs to simplify the orchestration of large scale workloads.
- `[Blog] Highly Available and Scalable Online Applications on Ray at Ant Group <https://www.anyscale.com/blog/building-highly-available-and-scalable-online-applications-on-ray-at-ant>`_
- `[Blog] Ray Forward 2022 Conference: Hyper-scale Ray Application Use Cases <https://www.anyscale.com/blog/ray-forward-2022>`_
- `[Blog] A new world record on the CloudSort benchmark using Ray <https://www.anyscale.com/blog/ray-breaks-the-usd1-tb-barrier-as-the-worlds-most-cost-efficient-sorting>`_
- :doc:`[Example] Speed up your web crawler by parallelizing it with Ray </ray-core/examples/web_crawler>`