ray-project--ray/doc/source/ray-contribute/writing-code-snippets.md

myst

myst
html_meta description Guide to writing runnable, CI-tested code examples for Ray docs using doctest-style, code-output-style, and literalinclude formats. Read this when adding code snippets to docstrings or user guides so they keep working for users.

(writing-code-snippets_ref)=

How to write code snippets

Users learn from example. So, whether you're writing a docstring or a user guide, include examples that illustrate the relevant APIs. Your examples should run out-of-the-box so that users can copy them and adapt them to their own needs.

This page describes how to write code snippets so that they're tested in CI.

:::{note} The examples in this guide use reStructuredText. If you're writing Markdown, use MyST syntax. To learn more, read the MyST documentation. :::

Types of examples

There are three types of examples: doctest-style, code-output-style, and literalinclude.

doctest-style examples

doctest-style examples mimic interactive Python sessions.

.. doctest::

    >>> def is_even(x):
    ...     return (x % 2) == 0
    >>> is_even(0)
    True
    >>> is_even(1)
    False

They're rendered like this:

>>> def is_even(x):
...     return (x % 2) == 0
>>> is_even(0)
True
>>> is_even(1)
False

:::{tip} If you're writing docstrings, exclude .. doctest:: to simplify your code:

Example:
    >>> def is_even(x):
    ...     return (x % 2) == 0
    >>> is_even(0)
    True
    >>> is_even(1)
    False

:::

code-output-style examples

code-output-style examples contain ordinary Python code.

.. testcode::

    def is_even(x):
        return (x % 2) == 0

    print(is_even(0))
    print(is_even(1))

.. testoutput::

    True
    False

They're rendered like this:

def is_even(x):
    return (x % 2) == 0

print(is_even(0))
print(is_even(1))

True
False

literalinclude examples

literalinclude examples display Python modules.

.. literalinclude:: ./doc_code/example_module.py
    :language: python
    :start-after: __is_even_begin__
    :end-before: __is_even_end__

:language: python

They're rendered like this:

:language: python
:start-after: __is_even_begin__
:end-before: __is_even_end__

Which type of example should you write?

There's no hard rule about which style you should use. Choose the style that best illustrates your API.

:::{tip} If you're not sure which style to use, use code-output-style. :::

When to use doctest-style

If you're writing a small example that emphasizes object representations, or if you want to print intermediate objects, use doctest-style.

.. doctest::

    >>> import ray
    >>> ds = ray.data.range(100)
    >>> ds.schema()
    Column  Type
    ------  ----
    id      int64
    >>> ds.take(5)
    [{'id': 0}, {'id': 1}, {'id': 2}, {'id': 3}, {'id': 4}]

When to use code-output-style

If you're writing a longer example, or if object representations aren't relevant to your example, use code-output-style.

.. testcode::

    from typing import Dict
    import numpy as np
    import ray

    ds = ray.data.read_csv("s3://anonymous@air-example-data/iris.csv")

    # Compute a "petal area" attribute.
    def transform_batch(batch: Dict[str, np.ndarray]) -> Dict[str, np.ndarray]:
        vec_a = batch["petal length (cm)"]
        vec_b = batch["petal width (cm)"]
        batch["petal area (cm^2)"] = np.round(vec_a * vec_b, 2)
        return batch

    transformed_ds = ds.map_batches(transform_batch)
    print(transformed_ds.materialize())

.. testoutput::

    shape: (150, 6)
    ╭───────────────────┬──────────────────┬───────────────────┬──────────────────┬────────┬───────────────────╮
    │ sepal length (cm) ┆ sepal width (cm) ┆ petal length (cm) ┆ petal width (cm) ┆ target ┆ petal area (cm^2) │
    │ ---               ┆ ---              ┆ ---               ┆ ---              ┆ ---    ┆ ---               │
    │ double            ┆ double           ┆ double            ┆ double           ┆ int64  ┆ double            │
    ╞═══════════════════╪══════════════════╪═══════════════════╪══════════════════╪════════╪═══════════════════╡
    │ 5.1               ┆ 3.5              ┆ 1.4               ┆ 0.2              ┆ 0      ┆ 0.28              │
    │ 4.9               ┆ 3.0              ┆ 1.4               ┆ 0.2              ┆ 0      ┆ 0.28              │
    │ 4.7               ┆ 3.2              ┆ 1.3               ┆ 0.2              ┆ 0      ┆ 0.26              │
    │ 4.6               ┆ 3.1              ┆ 1.5               ┆ 0.2              ┆ 0      ┆ 0.3               │
    │ 5.0               ┆ 3.6              ┆ 1.4               ┆ 0.2              ┆ 0      ┆ 0.28              │
    │ …                 ┆ …                ┆ …                 ┆ …                ┆ …      ┆ …                 │
    │ 6.7               ┆ 3.0              ┆ 5.2               ┆ 2.3              ┆ 2      ┆ 11.96             │
    │ 6.3               ┆ 2.5              ┆ 5.0               ┆ 1.9              ┆ 2      ┆ 9.5               │
    │ 6.5               ┆ 3.0              ┆ 5.2               ┆ 2.0              ┆ 2      ┆ 10.4              │
    │ 6.2               ┆ 3.4              ┆ 5.4               ┆ 2.3              ┆ 2      ┆ 12.42             │
    │ 5.9               ┆ 3.0              ┆ 5.1               ┆ 1.8              ┆ 2      ┆ 9.18              │
    ╰───────────────────┴──────────────────┴───────────────────┴──────────────────┴────────┴───────────────────╯
    (Showing 10 of 150 rows)

When to use literalinclude

If you're writing an end-to-end example and your example doesn't contain outputs, use literalinclude.

How to handle hard-to-test examples

When is it okay to not test an example?

You don't need to test examples that depend on external systems such as Weights and Biases.

Skipping doctest-style examples

To skip a doctest-style example, append # doctest: +SKIP to your Python code.

.. doctest::

    >>> import ray
    >>> ray.data.read_images("s3://private-bucket")  # doctest: +SKIP

Skipping code-output-style examples

To skip a code-output-style example, add :skipif: True to the testcode block.

.. testcode::
    :skipif: True

    from ray.air.integrations.wandb import WandbLoggerCallback
    callback = WandbLoggerCallback(
        project="Optimization_Project",
        api_key_file=...,
        log_config=True
    )

How to handle long or non-deterministic outputs

If your Python code is non-deterministic, or if your output is excessively long, you can skip all or part of the output.

Ignoring doctest-style outputs

To ignore parts of a doctest-style output, replace problematic sections with ellipses.

>>> import ray
>>> ray.data.read_images("s3://anonymous@ray-example-data/image-datasets/simple")
Dataset(num_rows=..., schema=...)

To ignore an output altogether, write a code-output-style snippet. Don't use # doctest: +SKIP.

Ignoring code-output-style outputs

If parts of your output are long or non-deterministic, replace problematic sections with ellipses.

.. testcode::

    import ray
    ds = ray.data.read_images("s3://anonymous@ray-example-data/image-datasets/simple")
    print(ds)

.. testoutput::

    Dataset(num_rows=..., schema=...)

If your output is non-deterministic and you want to display a sample output, add :options: +MOCK.

.. testcode::

    import random
    print(random.random())

.. testoutput::
    :options: +MOCK

    0.969461416250246

If your output is hard to test and you don't want to display a sample output, exclude the testoutput.

.. testcode::

    print("This output is hidden and untested")

How to test examples with GPUs

To configure Bazel to run an example with GPUs, complete the following steps:

1. Open the corresponding BUILD file. If your example is in the doc/ folder, open doc/BUILD. If your example is in the python/ folder, open a file such as python/ray/train/BUILD.

2. Locate the doctest rule. It looks like this:

   doctest(
       files = glob(
           include=["source/**/*.rst"],
       ),
       size = "large",
       tags = ["team:none"]
   )
   

3. Add the file that contains your example to the list of excluded files.

   doctest(
       files = glob(
           include=["source/**/*.rst"],
           exclude=["source/data/requires-gpus.rst"]
       ),
       tags = ["team:none"]
   )
   

4. If it doesn't already exist, create a doctest rule with gpu set to True.

   doctest(
       files = [],
       tags = ["team:none"],
       gpu = True
   )
   

5. Add the file that contains your example to the GPU rule.

   doctest(
       files = ["source/data/requires-gpus.rst"]
       size = "large",
       tags = ["team:none"],
       gpu = True
   )
   

For a practical example, see doc/BUILD or python/ray/train/BUILD.

How to locally test examples

To locally test examples, install the Ray fork of pytest-sphinx.

pip install git+https://github.com/ray-project/pytest-sphinx

Then, run pytest on a module, docstring, or user guide.

pytest --doctest-modules python/ray/data/read_api.py
pytest --doctest-modules python/ray/data/read_api.py::ray.data.read_api.range
pytest --doctest-modules doc/source/data/getting-started.rst