mirrors/ray-project--ray
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:17:40 +08:00
commit f1825c8ceb
10096 changed files with 2364182 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/ray-core/actors/actor-utils.rst
+34
View File
@@ -0,0 +1,34 @@
Utility Classes
===============
Actor Pool
~~~~~~~~~~
.. tab-set::
.. tab-item:: Python
The ``ray.util`` module contains a utility class, ``ActorPool``.
This class is similar to multiprocessing.Pool and lets you schedule Ray tasks over a fixed pool of actors.
.. literalinclude:: ../doc_code/actor-pool.py
See the :class:`package reference <ray.util.ActorPool>` for more information.
.. tab-item:: Java
Actor pool hasn't been implemented in Java yet.
.. tab-item:: C++
Actor pool hasn't been implemented in C++ yet.
Message passing using Ray Queue
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sometimes just using one signal to synchronize is not enough. If you need to send data among many tasks or
actors, you can use :class:`ray.util.queue.Queue <ray.util.queue.Queue>`.
.. literalinclude:: ../doc_code/actor-queue.py
Ray's Queue API has a similar API to Python's ``asyncio.Queue`` and ``queue.Queue``.
doc/source/ray-core/actors/async_api.rst
+335
View File
@@ -0,0 +1,335 @@
AsyncIO / Concurrency for Actors
================================
Within a single actor process, it is possible to execute concurrent threads.
Ray offers two types of concurrency within an actor:
* :ref:`async execution <async-actors>`
* :ref:`threading <threaded-actors>`
Keep in mind that the Python's `Global Interpreter Lock (GIL) <https://wiki.python.org/moin/GlobalInterpreterLock>`_ will only allow one thread of Python code running at once.
This means if you are just parallelizing Python code, you won't get true parallelism. If you call Numpy, Cython, Tensorflow, or PyTorch code, these libraries will release the GIL when calling into C/C++ functions.
**Neither the** :ref:`threaded-actors` nor :ref:`async-actors` **model will allow you to bypass the GIL.**
.. _async-actors:
AsyncIO for Actors
------------------
Since Python 3.5, it is possible to write concurrent code using the
``async/await`` `syntax <https://docs.python.org/3/library/asyncio.html>`__.
Ray natively integrates with asyncio. You can use Ray alongside popular
async frameworks like aiohttp, aioredis, etc.
.. testcode::
import ray
import asyncio
@ray.remote
class AsyncActor:
def __init__(self, expected_num_tasks: int):
self._event = asyncio.Event()
self._curr_num_tasks = 0
self._expected_num_tasks = expected_num_tasks
# Multiple invocations of this method can run concurrently on the same event loop.
async def run_concurrent(self):
self._curr_num_tasks += 1
if self._curr_num_tasks == self._expected_num_tasks:
print("All coroutines are executing concurrently, unblocking.")
self._event.set()
else:
print("Waiting for other coroutines to start.")
await self._event.wait()
print("All coroutines ran concurrently.")
actor = AsyncActor.remote(4)
refs = [actor.run_concurrent.remote() for _ in range(4)]
# Fetch results using regular `ray.get`.
ray.get(refs)
# Fetch results using `asyncio` APIs.
async def get_async():
return await asyncio.gather(*refs)
asyncio.run(get_async())
.. testoutput::
:options: +MOCK
(AsyncActor pid=9064) Waiting for other coroutines to start.
(AsyncActor pid=9064) Waiting for other coroutines to start.
(AsyncActor pid=9064) Waiting for other coroutines to start.
(AsyncActor pid=9064) All coroutines are executing concurrently, unblocking.
(AsyncActor pid=9064) All coroutines ran concurrently.
(AsyncActor pid=9064) All coroutines ran concurrently.
(AsyncActor pid=9064) All coroutines ran concurrently.
(AsyncActor pid=9064) All coroutines ran concurrently.
.. testcode::
:hide:
# NOTE: The outputs from the previous code block can show up in subsequent tests.
# To prevent flakiness, we wait for a grace period.
import time
print("Sleeping...")
time.sleep(1)
.. testoutput::
...
ObjectRefs as asyncio.Futures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ObjectRefs can be translated to asyncio.Futures. This feature
make it possible to ``await`` on ray futures in existing concurrent
applications.
Instead of:
.. testcode::
import ray
@ray.remote
def some_task():
return 1
ray.get(some_task.remote())
ray.wait([some_task.remote()])
you can wait on the ref with Python 3.9 and Python 3.10:
.. testcode::
import ray
import asyncio
@ray.remote
def some_task():
return 1
async def await_obj_ref():
await some_task.remote()
await asyncio.wait([some_task.remote()])
asyncio.run(await_obj_ref())
or the Future object directly with Python 3.11+:
.. testcode::
import asyncio
async def convert_to_asyncio_future():
ref = some_task.remote()
fut: asyncio.Future = asyncio.wrap_future(ref.future())
print(await fut)
asyncio.run(convert_to_asyncio_future())
.. testoutput::
1
See the `asyncio doc <https://docs.python.org/3/library/asyncio-task.html>`__
for more `asyncio` patterns including timeouts and ``asyncio.gather``.
.. _async-ref-to-futures:
ObjectRefs as concurrent.futures.Futures
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ObjectRefs can also be wrapped into ``concurrent.futures.Future`` objects. This
is useful for interfacing with existing ``concurrent.futures`` APIs:
.. testcode::
import concurrent
refs = [some_task.remote() for _ in range(4)]
futs = [ref.future() for ref in refs]
for fut in concurrent.futures.as_completed(futs):
assert fut.done()
print(fut.result())
.. testoutput::
1
1
1
1
Defining an Async Actor
~~~~~~~~~~~~~~~~~~~~~~~
By using `async` method definitions, Ray will automatically detect whether an actor support `async` calls or not.
.. testcode::
import ray
import asyncio
@ray.remote
class AsyncActor:
def __init__(self, expected_num_tasks: int):
self._event = asyncio.Event()
self._curr_num_tasks = 0
self._expected_num_tasks = expected_num_tasks
async def run_task(self):
print("Started task")
self._curr_num_tasks += 1
if self._curr_num_tasks == self._expected_num_tasks:
self._event.set()
else:
# Yield the event loop for multiple coroutines to run concurrently.
await self._event.wait()
print("Finished task")
actor = AsyncActor.remote(5)
# All 5 tasks will start at once and run concurrently.
ray.get([actor.run_task.remote() for _ in range(5)])
.. testoutput::
:options: +MOCK
(AsyncActor pid=3456) Started task
(AsyncActor pid=3456) Started task
(AsyncActor pid=3456) Started task
(AsyncActor pid=3456) Started task
(AsyncActor pid=3456) Started task
(AsyncActor pid=3456) Finished task
(AsyncActor pid=3456) Finished task
(AsyncActor pid=3456) Finished task
(AsyncActor pid=3456) Finished task
(AsyncActor pid=3456) Finished task
Under the hood, Ray runs all of the methods inside a single python event loop.
Please note that running blocking ``ray.get`` or ``ray.wait`` inside async
actor method is not allowed, because ``ray.get`` will block the execution
of the event loop.
In async actors, only one task can be running at any point in time (though tasks can be multiplexed). There will be only one thread in AsyncActor! See :ref:`threaded-actors` if you want a threadpool.
Setting concurrency in Async Actors
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can set the number of "concurrent" task running at once using the
``max_concurrency`` flag. By default, 1000 tasks can be running concurrently.
.. testcode::
import asyncio
import ray
@ray.remote
class AsyncActor:
def __init__(self, batch_size: int):
self._event = asyncio.Event()
self._curr_tasks = 0
self._batch_size = batch_size
async def run_task(self):
print("Started task")
self._curr_tasks += 1
if self._curr_tasks == self._batch_size:
self._event.set()
else:
await self._event.wait()
self._event.clear()
self._curr_tasks = 0
print("Finished task")
actor = AsyncActor.options(max_concurrency=2).remote(2)
# Only 2 tasks will run concurrently.
# Once 2 finish, the next 2 should run.
ray.get([actor.run_task.remote() for _ in range(8)])
.. testoutput::
:options: +MOCK
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Started task
(AsyncActor pid=5859) Finished task
(AsyncActor pid=5859) Finished task
.. _threaded-actors:
Threaded Actors
---------------
Sometimes, asyncio is not an ideal solution for your actor. For example, you may
have one method that performs some computation heavy task while blocking the event loop, not giving up control via ``await``. This would hurt the performance of an Async Actor because Async Actors can only execute 1 task at a time and rely on ``await`` to context switch.
Instead, you can use the ``max_concurrency`` Actor options without any async methods, allowing you to achieve threaded concurrency (like a thread pool).
.. warning::
When there is at least one ``async def`` method in actor definition, Ray
will recognize the actor as AsyncActor instead of ThreadedActor.
.. testcode::
@ray.remote
class ThreadedActor:
def task_1(self): print("I'm running in a thread!")
def task_2(self): print("I'm running in another thread!")
a = ThreadedActor.options(max_concurrency=2).remote()
ray.get([a.task_1.remote(), a.task_2.remote()])
.. testoutput::
:options: +MOCK
(ThreadedActor pid=4822) I'm running in a thread!
(ThreadedActor pid=4822) I'm running in another thread!
Each invocation of the threaded actor will be running in a thread pool. The size of the threadpool is limited by the ``max_concurrency`` value.
AsyncIO for Remote Tasks
------------------------
We don't support asyncio for remote tasks. The following snippet will fail:
.. testcode::
:skipif: True
@ray.remote
async def f():
pass
Instead, you can wrap the ``async`` function with a wrapper to run the task synchronously:
.. testcode::
async def f():
pass
@ray.remote
def wrapper():
import asyncio
asyncio.run(f())
doc/source/ray-core/actors/concurrency_group_api.rst
+199
View File
@@ -0,0 +1,199 @@
Limiting Concurrency Per-Method with Concurrency Groups
=======================================================
Besides setting the max concurrency overall for an actor, Ray allows methods to be separated into *concurrency groups*, each with its own thread(s). This allows you to limit the concurrency per-method, e.g., allow a health-check method to be given its own concurrency quota separate from request serving methods.
.. tip:: Concurrency groups work with both asyncio and threaded actors. The syntax is the same.
.. _defining-concurrency-groups:
Defining Concurrency Groups
---------------------------
This defines two concurrency groups, "io" with max concurrency = 2 and
"compute" with max concurrency = 4. The methods ``f1`` and ``f2`` are
placed in the "io" group, and the methods ``f3`` and ``f4`` are placed
into the "compute" group. Note that there is always a default
concurrency group for actors, which has a default concurrency of 1000 for
AsyncIO actors and 1 otherwise.
.. tab-set::
.. tab-item:: Python
You can define concurrency groups for actors using the ``concurrency_group`` decorator argument:
.. testcode::
import ray
@ray.remote(concurrency_groups={"io": 2, "compute": 4})
class AsyncIOActor:
def __init__(self):
pass
@ray.method(concurrency_group="io")
async def f1(self):
pass
@ray.method(concurrency_group="io")
async def f2(self):
pass
@ray.method(concurrency_group="compute")
async def f3(self):
pass
@ray.method(concurrency_group="compute")
async def f4(self):
pass
async def f5(self):
pass
a = AsyncIOActor.remote()
a.f1.remote() # executed in the "io" group.
a.f2.remote() # executed in the "io" group.
a.f3.remote() # executed in the "compute" group.
a.f4.remote() # executed in the "compute" group.
a.f5.remote() # executed in the default group.
.. tab-item:: Java
You can define concurrency groups for concurrent actors using the API ``setConcurrencyGroups()`` argument:
.. code-block:: java
class ConcurrentActor {
public long f1() {
return Thread.currentThread().getId();
}
public long f2() {
return Thread.currentThread().getId();
}
public long f3(int a, int b) {
return Thread.currentThread().getId();
}
public long f4() {
return Thread.currentThread().getId();
}
public long f5() {
return Thread.currentThread().getId();
}
}
ConcurrencyGroup group1 =
new ConcurrencyGroupBuilder<ConcurrentActor>()
.setName("io")
.setMaxConcurrency(1)
.addMethod(ConcurrentActor::f1)
.addMethod(ConcurrentActor::f2)
.build();
ConcurrencyGroup group2 =
new ConcurrencyGroupBuilder<ConcurrentActor>()
.setName("compute")
.setMaxConcurrency(1)
.addMethod(ConcurrentActor::f3)
.addMethod(ConcurrentActor::f4)
.build();
ActorHandle<ConcurrentActor> myActor = Ray.actor(ConcurrentActor::new)
.setConcurrencyGroups(group1, group2)
.remote();
myActor.task(ConcurrentActor::f1).remote(); // executed in the "io" group.
myActor.task(ConcurrentActor::f2).remote(); // executed in the "io" group.
myActor.task(ConcurrentActor::f3, 3, 5).remote(); // executed in the "compute" group.
myActor.task(ConcurrentActor::f4).remote(); // executed in the "compute" group.
myActor.task(ConcurrentActor::f5).remote(); // executed in the "default" group.
.. _default-concurrency-group:
Default Concurrency Group
-------------------------
By default, methods are placed in a default concurrency group which has a concurrency limit of 1000 for AsyncIO actors and 1 otherwise.
The concurrency of the default group can be changed by setting the ``max_concurrency`` actor option.
.. tab-set::
.. tab-item:: Python
The following actor has 2 concurrency groups: "io" and "default".
The max concurrency of "io" is 2, and the max concurrency of "default" is 10.
.. testcode::
@ray.remote(concurrency_groups={"io": 2})
class AsyncIOActor:
async def f1(self):
pass
actor = AsyncIOActor.options(max_concurrency=10).remote()
.. tab-item:: Java
The following concurrent actor has 2 concurrency groups: "io" and "default".
The max concurrency of "io" is 2, and the max concurrency of "default" is 10.
.. code-block:: java
class ConcurrentActor {
public long f1() {
return Thread.currentThread().getId();
}
}
ConcurrencyGroup group =
new ConcurrencyGroupBuilder<ConcurrentActor>()
.setName("io")
.setMaxConcurrency(2)
.addMethod(ConcurrentActor::f1)
.build();
ActorHandle<ConcurrentActor> myActor = Ray.actor(ConcurrentActor::new)
.setConcurrencyGroups(group)
.setMaxConcurrency(10)
.remote();
.. _setting-the-concurrency-group-at-runtime:
Setting the Concurrency Group at Runtime
----------------------------------------
You can also dispatch actor methods into a specific concurrency group at runtime.
The following snippet demonstrates setting the concurrency group of the
``f2`` method dynamically at runtime.
.. tab-set::
.. tab-item:: Python
You can use the ``.options`` method.
.. testcode::
# Executed in the "io" group (as defined in the actor class).
a.f2.options().remote()
# Executed in the "compute" group.
a.f2.options(concurrency_group="compute").remote()
.. tab-item:: Java
You can use ``setConcurrencyGroup`` method.
.. code-block:: java
// Executed in the "io" group (as defined in the actor creation).
myActor.task(ConcurrentActor::f2).remote();
// Executed in the "compute" group.
myActor.task(ConcurrentActor::f2).setConcurrencyGroup("compute").remote();
doc/source/ray-core/actors/named-actors.rst
+218
View File
@@ -0,0 +1,218 @@
Named Actors
============
An actor can be given a unique name within their :ref:`namespace <namespaces-guide>`.
This allows you to retrieve the actor from any job in the Ray cluster.
This can be useful if you cannot directly
pass the actor handle to the task that needs it, or if you are trying to
access an actor launched by another driver.
Note that the actor will still be garbage-collected if no handles to it
exist. See :ref:`actor-lifetimes` for more details.
.. tab-set::
.. tab-item:: Python
.. testcode::
import ray
@ray.remote
class Counter:
pass
# Create an actor with a name
counter = Counter.options(name="some_name").remote()
# Retrieve the actor later somewhere
counter = ray.get_actor("some_name")
.. tab-item:: Java
.. code-block:: java
// Create an actor with a name.
ActorHandle<Counter> counter = Ray.actor(Counter::new).setName("some_name").remote();
...
// Retrieve the actor later somewhere
Optional<ActorHandle<Counter>> counter = Ray.getActor("some_name");
Assert.assertTrue(counter.isPresent());
.. tab-item:: C++
.. code-block:: c++
// Create an actor with a globally unique name
ActorHandle<Counter> counter = ray::Actor(CreateCounter).SetGlobalName("some_name").Remote();
...
// Retrieve the actor later somewhere
boost::optional<ray::ActorHandle<Counter>> counter = ray::GetGlobalActor("some_name");
We also support non-global named actors in C++, which means that the actor name is only valid within the job and the actor cannot be accessed from another job.
.. code-block:: c++
// Create an actor with a job-scope-unique name
ActorHandle<Counter> counter = ray::Actor(CreateCounter).SetName("some_name").Remote();
...
// Retrieve the actor later somewhere in the same job
boost::optional<ray::ActorHandle<Counter>> counter = ray::GetActor("some_name");
.. note::
Named actors are scoped by namespace. If no namespace is assigned, they will
be placed in an anonymous namespace by default.
.. tab-set::
.. tab-item:: Python
.. testcode::
:skipif: True
import ray
@ray.remote
class Actor:
pass
# driver_1.py
# Job 1 creates an actor, "orange" in the "colors" namespace.
ray.init(address="auto", namespace="colors")
Actor.options(name="orange", lifetime="detached").remote()
# driver_2.py
# Job 2 is now connecting to a different namespace.
ray.init(address="auto", namespace="fruit")
# This fails because "orange" was defined in the "colors" namespace.
ray.get_actor("orange")
# You can also specify the namespace explicitly.
ray.get_actor("orange", namespace="colors")
# driver_3.py
# Job 3 connects to the original "colors" namespace
ray.init(address="auto", namespace="colors")
# This returns the "orange" actor we created in the first job.
ray.get_actor("orange")
.. tab-item:: Java
.. code-block:: java
import ray
class Actor {
}
// Driver1.java
// Job 1 creates an actor, "orange" in the "colors" namespace.
System.setProperty("ray.job.namespace", "colors");
Ray.init();
Ray.actor(Actor::new).setName("orange").remote();
// Driver2.java
// Job 2 is now connecting to a different namespace.
System.setProperty("ray.job.namespace", "fruits");
Ray.init();
// This fails because "orange" was defined in the "colors" namespace.
Optional<ActorHandle<Actor>> actor = Ray.getActor("orange");
Assert.assertFalse(actor.isPresent()); // actor.isPresent() is false.
// Driver3.java
System.setProperty("ray.job.namespace", "colors");
Ray.init();
// This returns the "orange" actor we created in the first job.
Optional<ActorHandle<Actor>> actor = Ray.getActor("orange");
Assert.assertTrue(actor.isPresent()); // actor.isPresent() is true.
Get-Or-Create a Named Actor
---------------------------
A common use case is to create an actor only if it doesn't exist.
Ray provides a ``get_if_exists`` option for actor creation that does this out of the box.
This method is available after you set a name for the actor via ``.options()``.
If the actor already exists, a handle to the actor will be returned
and the arguments will be ignored. Otherwise, a new actor will be
created with the specified arguments.
.. tab-set::
.. tab-item:: Python
.. literalinclude:: ../doc_code/get_or_create.py
.. tab-item:: Java
.. code-block:: java
// This feature is not yet available in Java.
.. tab-item:: C++
.. code-block:: c++
// This feature is not yet available in C++.
.. _actor-lifetimes:
Actor Lifetimes
---------------
Separately, actor lifetimes can be decoupled from the job, allowing an actor to persist even after the driver process of the job exits. We call these actors *detached*.
.. tab-set::
.. tab-item:: Python
.. testcode::
counter = Counter.options(name="CounterActor", lifetime="detached").remote()
The ``CounterActor`` will be kept alive even after the driver running above script
exits. Therefore it is possible to run the following script in a different
driver:
.. testcode::
counter = ray.get_actor("CounterActor")
Note that an actor can be named but not detached. If we only specified the
name without specifying ``lifetime="detached"``, then the CounterActor can
only be retrieved as long as the original driver is still running.
.. tab-item:: Java
.. code-block:: java
System.setProperty("ray.job.namespace", "lifetime");
Ray.init();
ActorHandle<Counter> counter = Ray.actor(Counter::new).setName("some_name").setLifetime(ActorLifetime.DETACHED).remote();
The CounterActor will be kept alive even after the driver running above process
exits. Therefore it is possible to run the following code in a different
driver:
.. code-block:: java
System.setProperty("ray.job.namespace", "lifetime");
Ray.init();
Optional<ActorHandle<Counter>> counter = Ray.getActor("some_name");
Assert.assertTrue(counter.isPresent());
.. tab-item:: C++
Customizing lifetime of an actor hasn't been implemented in C++ yet.
Unlike normal actors, detached actors are not automatically garbage-collected by Ray.
Detached actors must be manually destroyed once you are sure that they are no
longer needed. To do this, use ``ray.kill`` to :ref:`manually terminate <ray-kill-actors>` the actor.
After this call, the actor's name may be reused.
doc/source/ray-core/actors/out-of-band-communication.rst
+36
View File
@@ -0,0 +1,36 @@
Out-of-band Communication
=========================
Typically, Ray actor communication is done through actor method calls and data is shared through the distributed object store.
However, in some use cases out-of-band communication can be useful.
Wrapping Library Processes
--------------------------
Many libraries already have mature, high-performance internal communication stacks and
they leverage Ray as a language-integrated actor scheduler.
The actual communication between actors is mostly done out-of-band using existing communication stacks.
For example, Horovod-on-Ray uses NCCL or MPI-based collective communications, and RayDP uses Spark's internal RPC and object manager.
See `Ray Distributed Library Patterns <https://www.anyscale.com/blog/ray-distributed-library-patterns>`_ for more details.
Ray Collective
--------------
Ray's collective communication library (\ ``ray.util.collective``\ ) allows efficient out-of-band collective and point-to-point communication between distributed CPUs or GPUs.
See :ref:`Ray Collective <ray-collective>` for more details.
HTTP Server
-----------
You can start an HTTP server inside the actor and expose HTTP endpoints to clients
so users outside of the Ray cluster can communicate with the actor.
.. tab-set::
.. tab-item:: Python
.. literalinclude:: ../doc_code/actor-http-server.py
Similarly, you can expose other types of servers as well (e.g., gRPC servers).
Limitations
-----------
When using out-of-band communication with Ray actors, keep in mind that Ray does not manage the calls between actors. This means that functionality like distributed reference counting will not work with out-of-band communication, so you should take care not to pass object references in this way.
doc/source/ray-core/actors/task-orders.rst
+149
View File
@@ -0,0 +1,149 @@
.. _actor-task-order:
Actor Task Execution Order
==========================
Synchronous, Single-Threaded Actor
----------------------------------
In Ray, an actor receives tasks from multiple submitters (including driver and workers).
For tasks received from the same submitter, a synchronous, single-threaded actor executes
them in the order they were submitted, unless you set ``allow_out_of_order_execution``,
or Ray retries tasks. In other words, a given task will not be executed until previously
submitted tasks from the same submitter have finished execution.
For actors where `max_task_retries` is set to a non-zero number, the task
execution order is not guaranteed when task retries occur.
.. tab-set::
.. tab-item:: Python
.. testcode::
import ray
@ray.remote
class Counter:
def __init__(self):
self.value = 0
def add(self, addition):
self.value += addition
return self.value
counter = Counter.remote()
# For tasks from the same submitter,
# they are executed according to submission order.
value0 = counter.add.remote(1)
value1 = counter.add.remote(2)
# Output: 1. The first submitted task is executed first.
print(ray.get(value0))
# Output: 3. The later submitted task is executed later.
print(ray.get(value1))
.. testoutput::
1
3
However, the actor does not guarantee the execution order of the tasks from different
submitters. For example, suppose an unfulfilled argument blocks a previously submitted
task. In this case, the actor can still execute tasks submitted by a different worker.
.. tab-set::
.. tab-item:: Python
.. testcode::
import time
import ray
@ray.remote
class Counter:
def __init__(self):
self.value = 0
def add(self, addition):
self.value += addition
return self.value
counter = Counter.remote()
# Submit task from a worker
@ray.remote
def submitter(value):
return ray.get(counter.add.remote(value))
# Simulate delayed result resolution.
@ray.remote
def delayed_resolution(value):
time.sleep(1)
return value
# Submit tasks from different workers, with
# the first submitted task waiting for
# dependency resolution.
value0 = submitter.remote(delayed_resolution.remote(1))
value1 = submitter.remote(2)
# Output: 3. The first submitted task is executed later.
print(ray.get(value0))
# Output: 2. The later submitted task is executed first.
print(ray.get(value1))
.. testoutput::
3
2
Asynchronous or Threaded Actor
------------------------------
:ref:`Asynchronous or threaded actors <async-actors>` do not guarantee the
task execution order. This means the system might execute a task
even though previously submitted tasks are pending execution.
.. tab-set::
.. tab-item:: Python
.. testcode::
import time
import ray
@ray.remote
class AsyncCounter:
def __init__(self):
self.value = 0
async def add(self, addition):
self.value += addition
return self.value
counter = AsyncCounter.remote()
# Simulate delayed result resolution.
@ray.remote
def delayed_resolution(value):
time.sleep(1)
return value
# Submit tasks from the driver, with
# the first submitted task waiting for
# dependency resolution.
value0 = counter.add.remote(delayed_resolution.remote(1))
value1 = counter.add.remote(2)
# Output: 3. The first submitted task is executed later.
print(ray.get(value0))
# Output: 2. The later submitted task is executed first.
print(ray.get(value1))
.. testoutput::
3
2
doc/source/ray-core/actors/terminating-actors.rst
+250
View File
@@ -0,0 +1,250 @@
Terminating Actors
==================
Actor processes will be terminated automatically when all copies of the
actor handle have gone out of scope in Python, or if the original creator
process dies. When actors terminate gracefully, Ray calls the actor's
``__ray_shutdown__()`` method if defined, allowing for cleanup of resources
(see :ref:`actor-cleanup` for details).
Note that automatic termination of actors is not yet supported in Java or C++.
.. _ray-kill-actors:
Manual termination via an actor handle
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In most cases, Ray will automatically terminate actors that have gone out of
scope, but you may sometimes need to terminate an actor forcefully. This should
be reserved for cases where an actor is unexpectedly hanging or leaking
resources, and for :ref:`detached actors <actor-lifetimes>`, which must be
manually destroyed.
.. tab-set::
.. tab-item:: Python
.. testcode::
import ray
@ray.remote
class Actor:
pass
actor_handle = Actor.remote()
ray.kill(actor_handle)
# Force kill: the actor exits immediately without cleanup.
# This will NOT call __ray_shutdown__() or atexit handlers.
.. tab-item:: Java
.. code-block:: java
actorHandle.kill();
// This will not go through the normal Java System.exit teardown logic, so any
// shutdown hooks installed in the actor using ``Runtime.addShutdownHook(...)`` will
// not be called.
.. tab-item:: C++
.. code-block:: c++
actor_handle.Kill();
// This will not go through the normal C++ std::exit
// teardown logic, so any exit handlers installed in
// the actor using ``std::atexit`` will not be called.
This will cause the actor to immediately exit its process, causing any current,
pending, and future tasks to fail with a ``RayActorError``. If you would like
Ray to :ref:`automatically restart <fault-tolerance-actors>` the actor, make sure to set a nonzero
``max_restarts`` in the ``@ray.remote`` options for the actor, then pass the
flag ``no_restart=False`` to ``ray.kill``.
For :ref:`named and detached actors <actor-lifetimes>`, calling ``ray.kill`` on
an actor handle destroys the actor and allows the name to be reused.
Use `ray list actors --detail` from :ref:`State API <state-api-overview-ref>` to see the death cause of dead actors:
.. code-block:: bash
# This API is only available when you download Ray via `pip install "ray[default]"`
ray list actors --detail
.. code-block:: bash
---
- actor_id: e8702085880657b355bf7ef001000000
class_name: Actor
state: DEAD
job_id: '01000000'
name: ''
node_id: null
pid: 0
ray_namespace: dbab546b-7ce5-4cbb-96f1-d0f64588ae60
serialized_runtime_env: '{}'
required_resources: {}
death_cause:
actor_died_error_context: # <---- You could see the error message w.r.t why the actor exits.
error_message: The actor is dead because `ray.kill` killed it.
owner_id: 01000000ffffffffffffffffffffffffffffffffffffffffffffffff
owner_ip_address: 127.0.0.1
ray_namespace: dbab546b-7ce5-4cbb-96f1-d0f64588ae60
class_name: Actor
actor_id: e8702085880657b355bf7ef001000000
never_started: true
node_ip_address: ''
pid: 0
name: ''
is_detached: false
placement_group_id: null
repr_name: ''
Manual termination within the actor
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If necessary, you can manually terminate an actor from within one of the actor methods.
This will kill the actor process and release resources associated/assigned to the actor.
.. tab-set::
.. tab-item:: Python
.. testcode::
@ray.remote
class Actor:
def exit(self):
ray.actor.exit_actor()
actor = Actor.remote()
actor.exit.remote()
This approach should generally not be necessary as actors are automatically garbage
collected. The ``ObjectRef`` resulting from the task can be waited on to wait
for the actor to exit (calling ``ray.get()`` on it will raise a ``RayActorError``).
.. tab-item:: Java
.. code-block:: java
Ray.exitActor();
Garbage collection for actors hasn't been implemented yet, so this is currently the
only way to terminate an actor gracefully. The ``ObjectRef`` resulting from the task
can be waited on to wait for the actor to exit (calling ``ObjectRef::get`` on it will
throw a ``RayActorException``).
.. tab-item:: C++
.. code-block:: c++
ray::ExitActor();
Garbage collection for actors hasn't been implemented yet, so this is currently the
only way to terminate an actor gracefully. The ``ObjectRef`` resulting from the task
can be waited on to wait for the actor to exit (calling ``ObjectRef::Get`` on it will
throw a ``RayActorException``).
Note that this method of termination waits until any previously submitted
tasks finish executing and then exits the process gracefully with sys.exit.
You could see the actor is dead as a result of the user's `exit_actor()` call:
.. code-block:: bash
# This API is only available when you download Ray via `pip install "ray[default]"`
ray list actors --detail
.. code-block:: bash
---
- actor_id: 070eb5f0c9194b851bb1cf1602000000
class_name: Actor
state: DEAD
job_id: '02000000'
name: ''
node_id: 47ccba54e3ea71bac244c015d680e202f187fbbd2f60066174a11ced
pid: 47978
ray_namespace: 18898403-dda0-485a-9c11-e9f94dffcbed
serialized_runtime_env: '{}'
required_resources: {}
death_cause:
actor_died_error_context:
error_message: 'The actor is dead because its worker process has died.
Worker exit type: INTENDED_USER_EXIT Worker exit detail: Worker exits
by a user request. exit_actor() is called.'
owner_id: 02000000ffffffffffffffffffffffffffffffffffffffffffffffff
owner_ip_address: 127.0.0.1
node_ip_address: 127.0.0.1
pid: 47978
ray_namespace: 18898403-dda0-485a-9c11-e9f94dffcbed
class_name: Actor
actor_id: 070eb5f0c9194b851bb1cf1602000000
name: ''
never_started: false
is_detached: false
placement_group_id: null
repr_name: ''
.. _actor-cleanup:
Actor cleanup with `__ray_shutdown__`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When an actor terminates gracefully, Ray calls the ``__ray_shutdown__()`` method
if it exists, allowing cleanup of resources like database connections or file handles.
.. tab-set::
.. tab-item:: Python
.. testcode::
import ray
import tempfile
import os
@ray.remote
class FileProcessorActor:
def __init__(self):
self.temp_file = tempfile.NamedTemporaryFile(delete=False)
self.temp_file.write(b"processing data")
self.temp_file.flush()
def __ray_shutdown__(self):
# Clean up temporary file
if hasattr(self, 'temp_file'):
self.temp_file.close()
os.unlink(self.temp_file.name)
def process(self):
return "done"
actor = FileProcessorActor.remote()
ray.get(actor.process.remote())
del actor # __ray_shutdown__() is called automatically
When ``__ray_shutdown__()`` is called:
- **Automatic termination**: When all actor handles go out of scope (``del actor`` or natural scope exit)
- **Manual graceful termination**: When you call ``actor.__ray_terminate__.remote()``
When ``__ray_shutdown__()`` is **NOT** called:
- **Force kill**: When you use ``ray.kill(actor)`` - the actor is killed immediately without cleanup.
- **Unexpected termination**: When the actor process crashes or exits unexpectedly (such as a segfault or being killed by the OOM killer).
**Important notes:**
- ``__ray_shutdown__()`` runs after all actor tasks complete.
- By default, Ray waits 30 seconds for the graceful shutdown procedure (including ``__ray_shutdown__()``) to complete. If the actor doesn't exit within this timeout, it's force killed. Configure this with ``ray.init(_system_config={"actor_graceful_shutdown_timeout_ms": 60000})``.
- Exceptions in ``__ray_shutdown__()`` are caught and logged but don't prevent actor termination.
- ``__ray_shutdown__()`` must be a synchronous method, including for async actors.