chore: import upstream snapshot with attribution

doc/source/ray-core/head-node-memory-management.rst

@@ -0,0 +1,111 @@
+.. _head-node-memory-management:  
+  
+Head Node Memory Management  
+============================  
+  
+When running Ray clusters for extended periods, the head node's memory usage can steadily increase over time, potentially leading to out-of-memory (OOM) errors that can make the entire cluster unusable. This guide explains the causes of head node memory growth and provides mitigation strategies.  
+  
+.. contents::  
+    :local:  
+  
+Why Head Node Memory Grows  
+---------------------------  
+  
+- The Ray Dashboard provides a web interface for cluster monitoring and debugging. For more details, see :ref:`observability-getting-started`.
+- The Ray Dashboard caches cluster events in memory for display and debugging purposes. The ``RAY_DASHBOARD_MAX_EVENTS_TO_CACHE`` environment variable controls the cache size. For implementation details, see the `event caching code <https://github.com/ray-project/ray/blob/814768317813afca2f0af740f58d024b059ae7d7/python/ray/dashboard/modules/event/event_head.py#L35>`_.  
+- The dashboard processes and stores logs and metadata from jobs and workers, which accumulate over time in long-running clusters.  
+  
+Mitigation Strategies  
+---------------------  
+
+Avoid Scheduling on the Head Node
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Running tasks or actors on the head node isn't recommended because it hosts critical system components. Preventing scheduling on the head node helps reduce contention and memory pressure.
+
+See :ref:`vms-large-cluster-configure-head-node` for head-node best practices.
+
+Disable the Dashboard  
+~~~~~~~~~~~~~~~~~~~~~  
+  
+If you don't need the dashboard, disabling it removes event caching and related memory overhead. This reduces observability into the system so it's not recommended for production clusters.
+  
+**Python API:**  
+  
+.. code-block:: python  
+  
+    import ray  
+    ray.init(include_dashboard=False)  
+  
+**CLI:**  
+  
+.. code-block:: bash  
+  
+    ray start --head --include-dashboard=False  
+  
+**Kubernetes:**  
+  
+Set ``spec.headGroupSpec.rayStartParams.include-dashboard`` to ``"false"`` in your RayCluster configuration.  
+  
+.. warning::  
+    Disabling the dashboard prevents KubeRay's ``RayJob`` and ``RayService`` features from working properly.
+
+
+Kubernetes Configuration
+------------------------
+
+Head Pod Memory Settings  
+~~~~~~~~~~~~~~~~~~~~~~~~  
+  
+When deploying on Kubernetes, configure appropriate memory requests and limits for the head pod.  
+  
+**Important:** Set memory and CPU resource requests equal to their limits. KubeRay uses the container's resource **limits** to configure Ray's logical resource capacities and ignores memory and CPU **requests**.  
+  
+Example configuration:  
+  
+.. code-block:: yaml  
+  
+    headGroupSpec:  
+      template:  
+        spec:  
+          containers:  
+          - name: ray-head  
+            resources:  
+              requests:  
+                memory: "8Gi"  
+                cpu: "4"  
+              limits:  
+                memory: "8Gi"  
+                cpu: "4"  
+  
+Recommended Head Node Specifications  
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
+  
+For large clusters, a good starting specification for the head node is:
+  
+- **CPU:** 16 cores  
+- **Memory:** 64 GB  
+  
+The actual requirements depend on your workload and cluster size.  
+  
+Additionally, consider preventing Ray from scheduling tasks on the head node by setting ``num-cpus: "0"`` in ``rayStartParams``.  
+  
+Best Practices  
+--------------  
+
+1. **Avoid scheduling on the head node** to reduce contention and memory pressure.
+2. **Scale vertically and use a larger head node** before adjusting internal settings.
+3. **Set appropriate Kubernetes resource limits** (match requests for memory and GPU).
+
+.. note::
+    You *can* disable the dashboard, but doing so severely limits observability and isn't **recommended for production**. If you choose to disable it, see the `Disable the Dashboard` section in the preceding text.
+
+Troubleshooting
+---------------  
+  
+If your head node experiences OOM issues:  
+  
+1. Check current memory usage: ``ray memory``. See :ref:`debug-with-ray-memory`
+2. Consider increasing head node memory allocation
+
+For more information on OOM prevention, see :ref:`ray-oom-prevention`.