chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,58 @@
|
||||
<!-- Loaded on-demand when Claude works on Ray documentation files. -->
|
||||
<!-- Keep under 50 lines. Multi-step procedures → skills. Code style → rules/. -->
|
||||
|
||||
# Ray Documentation
|
||||
|
||||
Sphinx documentation built by Read the Docs and served at `docs.ray.io`. Build pipeline: `.buildkite/doc.rayci.yml`.
|
||||
|
||||
## File format for new pages
|
||||
|
||||
Author new files under `doc/source/` as MyST Markdown (`.md`). A lint check rejects newly added `.rst` files. Edits to existing `.rst` files are not flagged. `git mv` is not flagged for any rename, which covers file renames and directory reorganization. This is deliberate, to keep reorganization and agent-driven moves unblocked.
|
||||
|
||||
## How CI is configured
|
||||
|
||||
`.buildkite/test.rules.txt` maps file-change patterns to tag sets, and tags drive which CI suites run. The `doc` tag covers content under `doc/`, `doc/requirements-doc.txt`, `.vale.ini`/`.vale/`, and `.buildkite/doc.rayci.yml`. Docs-only deplock changes (`python/deplocks/docs/*.lock`, `ci/raydepsets/configs/docs.depsets.yaml`) tag to `doc python_dependencies` to skip the full python-deps test set.
|
||||
|
||||
## Scope discipline for PRs
|
||||
|
||||
For docs-only fixes, take the lightest path:
|
||||
|
||||
- Touch only files mapped to the `doc` tag.
|
||||
- Don't bundle in a non-doc change "while you're at it" — that change pulls its own (often expensive) test set into the PR.
|
||||
- If a docs change requires a non-doc change to land cleanly (e.g., autodoc references a renamed symbol), land them in the larger non-doc PR, not a docs-led PR.
|
||||
|
||||
For generated API docs that depend on Python source under `python/ray/...`, expect broader test runs. That's correct, not a misconfiguration.
|
||||
|
||||
## DCO sign-off
|
||||
|
||||
Every commit on a `ray-project/ray` PR needs a `Signed-off-by:` trailer (Developer Certificate of Origin). Pass `--signoff` to `git commit`, or `git commit --amend --signoff` to fix an unsigned commit. The DCO GitHub check fails the PR otherwise.
|
||||
|
||||
## When to revise the test rules
|
||||
|
||||
If a docs-only PR hits unnecessarily broad tests, file a `.buildkite/test.rules.txt` PR (precedent: #63132) instead of working around it. Quick check: does the file change affect any non-`doc` build artifact, runtime behavior, or API surface? If no, tag it `doc`.
|
||||
|
||||
## Cross-references between .md and .rst sources
|
||||
|
||||
When linking from .md to a doc whose source is .rst, use the {doc} role, not a bare `[text](path.md)` link: `` {doc}`Ray documentation home page </index>` ``. MyST-Parser 2.0 has a cross-extension resolution bug — a bare `[text](path.md)` link to an .rst source emits `myst.xref_missing`, which `fail_on_warning: true` in `.readthedocs.yaml` turns into a build failure.
|
||||
|
||||
## Comparing PR previews against `/en/master` vs `/en/latest`
|
||||
|
||||
For regression checks, compare a PR preview against `/en/master` — it's the docs build of the same master state your PR branches from, so byte-level content diffs against master are attributable to the stack. `/en/latest` is the last tagged release and lags master by every unreleased commit, so diffs there mix in upstream feature work and don't isolate regressions. Use `/en/latest` only to inventory net-new content since the last release.
|
||||
|
||||
## Editing the docs lockfile
|
||||
|
||||
`doc/requirements-doc.txt` is source of truth; `python/deplocks/docs/docbuild_depset_py3.10.lock` is regenerated by raydepsets, and CI's `raydepsets --check` enforces consistency.
|
||||
|
||||
- Additive packages or version bumps with unchanged dep trees: hand-edit the package's entry block plus its via comments. Precedent: #63130.
|
||||
- Version bumps that change a package's direct deps (gaining or losing requirements): regenerate via `bazelisk run //ci/raydepsets:raydepsets -- build ci/raydepsets/configs/docs.depsets.yaml`. Hand-edits miss transitive churn — precedent: #63344 caught a stale `# via pydata-sphinx-theme` line under `packaging` after the theme dropped that dep.
|
||||
|
||||
## Anonymizing paths in notebook output cells
|
||||
|
||||
`/home/ray` is Ray's runtime home directory in containers and clusters, not a personal-path leak — never anonymize it. Anonymize only real user identifiers: personal home prefixes (`/Users/<name>`, `/home/<person>`) become `~`, and experiment or output dirs that encode a person or the deprecated AIR runtime (e.g. `christy-air`) become a neutral equivalent such as `ray_results`. Precedent: #63464.
|
||||
|
||||
## Skills
|
||||
|
||||
- `/lint` — run linters on modified files.
|
||||
- `/fetch-buildkite-logs` — pull build logs for CI failures.
|
||||
|
||||
See top-level `.claude/CLAUDE.md` for the full skill index.
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
paths:
|
||||
- "doc/source/**/*.md"
|
||||
- "doc/source/**/*.rst"
|
||||
---
|
||||
<!-- Prose style for Ray documentation. -->
|
||||
<!-- Source of truth: doc/source/ray-contribute/writing-style.md — read it before writing or editing docs prose. -->
|
||||
<!-- These bullets are the highest-frequency corrections; the full guide is broader. -->
|
||||
|
||||
- Read `doc/source/ray-contribute/writing-style.md` before writing or editing docs prose. It's the source of truth and supersedes prior style guidance.
|
||||
- Active voice with a named actor. Present tense. Second person ("you"). Imperative for instructions.
|
||||
- No first-person plural. Rewrite to remove "we" and "our"; use "The Ray project" only when a subject is unavoidable.
|
||||
- Use contractions (don't, isn't, can't), except in warnings and error messages.
|
||||
- "such as," not "like," for examples. "ID," not "id," in prose. Plain words: "use" (not utilize/leverage), "through" (not via), "before" (not prior to).
|
||||
- Cut filler: simply, just, basically, actually, really, very. Cut time-relative words: currently, now, recently, new.
|
||||
- Sentence-case headings; imperative for tasks, questions for concepts.
|
||||
- Short sentences. No dashes, parentheticals, or semicolons in prose. A colon only introduces a list or code block.
|
||||
- List items that are sentences or contain verbs get periods. Lead-ins are complete sentences ending in a colon.
|
||||
- MyST admonitions (`:::{note}`, `:::{warning}`), not bold inline labels. Tag code blocks with a language; no `$` prompt.
|
||||
- Cross-references: `{doc}` for whole pages (required when the target source is `.rst`), `{ref}` for labeled targets, `[text](page.md)` between `.md` pages.
|
||||
- Never fabricate technical details. Every technical claim needs a verifiable source: the docs, the source code, or a config file. When you can't verify a detail, leave it out.
|
||||
@@ -0,0 +1,211 @@
|
||||
---
|
||||
name: rst-to-myst
|
||||
description: Convert Ray documentation pages from reStructuredText (.rst) to MyST Markdown (.md). Use when migrating existing files under doc/source/ to MyST, finishing a partial MyST migration of a directory, or when asked to convert/migrate a doc page to markdown. Covers the RST-to-MyST directive mapping, label and cross-reference preservation, sphinx-design tabs/dropdowns, doctest/testcode handling, the doc/BUILD.bazel doctest exclusions, and the build and doctest verification needed to land a clean docs PR.
|
||||
user-invocable: true
|
||||
argument-hint: <file(s) or directory under doc/source to convert>
|
||||
---
|
||||
|
||||
# Convert RST to MyST Markdown
|
||||
|
||||
MyST Markdown is the standard for new Ray doc pages — `doc/.claude/CLAUDE.md` declares it, and a lint check rejects newly-added `.rst`. This skill converts an **existing** `.rst` page (or a batch) to MyST `.md` **faithfully**: format only, preserving the rendered HTML and any test coverage.
|
||||
|
||||
The Ray docs build with `fail_on_warning: true` (`.readthedocs.yaml`), so a sloppy conversion doesn't render wrong — it **fails the build**. Most of this skill is about the handful of constructs that break the build or silently drop test coverage if mishandled.
|
||||
|
||||
---
|
||||
|
||||
## When to use this skill
|
||||
|
||||
**Use when:**
|
||||
|
||||
- Migrating one or more existing `doc/source/**/*.rst` files to MyST `.md`.
|
||||
- Finishing a partial MyST migration of a directory.
|
||||
|
||||
**Not for:**
|
||||
|
||||
- Authoring a brand-new page — just write `.md` directly (no conversion needed).
|
||||
- Editing `.rst` content you're not converting (edits to existing `.rst` aren't lint-flagged).
|
||||
- Notebooks (`.ipynb`) — different workflow.
|
||||
- Bundling unrelated content rewrites — keep the diff a pure format conversion (see Golden rule).
|
||||
|
||||
---
|
||||
|
||||
## Golden rule: faithful conversion
|
||||
|
||||
Convert the **format**, not the content. The rendered HTML should be byte-equivalent to the pre-conversion page, **except** for deliberate, called-out light cleanup (a dead link, a stale version ref). No restructuring, no rewording of sound content, no heading-level "fixes."
|
||||
|
||||
Why: the decisive regression check compares the PR's Read the Docs preview against `/en/master` (per `doc/.claude/CLAUDE.md`). A faithful conversion makes that diff empty and the PR trivially reviewable. Capitalization nits ("github"→"GitHub"), heading-case changes, and rewraps all add noise and invite scope debates — leave them unless explicitly asked.
|
||||
|
||||
**Faithful does not mean byte-copying links.** A few RST link forms render fine in RST but are *wrong* in MyST and fail `fail_on_warning` (see Hard rule 2). Translate them; don't transcribe them.
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Read the source(s) and the two style models
|
||||
|
||||
Read every `.rst` you're converting **in full**. Also read the canonical MyST examples in the same tree for house style: `doc/source/ray-contribute/docs.md` and `doc/source/ray-contribute/agent-development.md` (frontmatter, `(label)=`, `{contents}`, admonition and image conventions).
|
||||
|
||||
### 2. Pre-flight — verify every reference resolves *before* converting
|
||||
|
||||
A stale `literalinclude` path, autodoc symbol, or `{ref}` target turns into a build failure under `fail_on_warning`. Confirm each up front:
|
||||
|
||||
- **Labels this file defines** — `grep -nE '^\.\. _.*:' file.rst`. You must preserve **every** one (Hard rule 1). Note them.
|
||||
- **External callers of those labels** — `grep -rn '<label-name>' doc/source python rllib`. Confirms they're load-bearing (and that you must not rename them).
|
||||
- **`literalinclude` targets** — the file exists; `:lines:`/`:start-after:`/`:end-before:` markers still resolve.
|
||||
- **autodoc targets** — every `.. autofunction::`/`.. autoclass::` symbol imports.
|
||||
- **Who references THIS file** — grep the **bare filename** across all of `doc/`, e.g. `grep -rn 'getting-involved' doc/source`. Do **not** grep only the `dir/stem.rst` path: siblings link relatively (`[text](./getting-involved.rst)`, `(getting-involved.rst)`), and those break silently when you rename the file. Classify each hit (see "Reference updates"); most are no-ops, but `doc/BUILD.bazel`, `{include}`, and any relative `.rst` link from another page are not.
|
||||
|
||||
### 3. Convert using the mapping
|
||||
|
||||
Apply the table below construct-by-construct. Preserve prose line-wrapping verbatim (keeps the diff line-aligned). Then apply the Hard rules and Construct notes.
|
||||
|
||||
### 4. Update references that actually need it
|
||||
|
||||
Most don't (see checklist). The ones that do go **in the same PR** as the file they track.
|
||||
|
||||
### 5. Verify
|
||||
|
||||
Static checks → build (RtD) → doctest (if the file is doctest-tested) → regression vs `/en/master`. See "Verification".
|
||||
|
||||
### 6. Ship
|
||||
|
||||
`git rm` the `.rst`, add the `.md`. Commit, push, PR. For the OSS PR conventions (branch base, DCO sign-off, no internal ticket keys, etc.) follow the project's docs-PR workflow.
|
||||
|
||||
---
|
||||
|
||||
## The mapping
|
||||
|
||||
| RST | MyST Markdown |
|
||||
|---|---|
|
||||
| `.. meta::` / `:description:` | YAML frontmatter `myst:\n html_meta:\n description: "…"` |
|
||||
| `.. _label:` above a heading | `(label)=` on its own line, blank line, then the heading |
|
||||
| `====` / `----` underline | `#` / `##` … — **level by order of appearance, see Hard rule 3** |
|
||||
| `` ``literal`` `` (double backtick) | `` `code` `` (single backtick) |
|
||||
| `` `text` `` (single backtick) | `` `code` `` — faithful, since `default_role = "code"` |
|
||||
| `` `text <url>`_ `` / `` `text <url>`__ `` | `[text](url)` |
|
||||
| bare URL `https://…` | `<https://…>` (angle-bracket autolink — **`linkify` is off**) |
|
||||
| same-page section link `` `text <page.html#sec>`_ `` | `[text](#sec)` (fragment) — **never keep the `.html#` URL; see Hard rule 2** |
|
||||
| `` :ref:`text <label>` `` | `` {ref}`text <label>` `` |
|
||||
| `` :doc:`text <path>` `` | `` {doc}`text <path>` `` |
|
||||
| `.. note::` / `.. tip::` / `.. warning::` | `:::{note}` / `:::{tip}` / `:::{warning}` (colon fence) |
|
||||
| `.. code-block:: LANG` / `.. code:: LANG` | fenced ` ```LANG ` |
|
||||
| `.. tab-set::` / `.. tab-item:: T` | `::::{tab-set}` / `:::{tab-item} T` (colon fences — see Construct notes) |
|
||||
| `.. dropdown:: T` (`:open:`) | `:::{dropdown} T` with `:open:` on the next line |
|
||||
| `.. testcode::` / `.. testoutput::` / `.. doctest::` | ` ```{testcode} ` / `{testoutput}` / `{doctest}` — **only for real, executed blocks; see Hard rule 4** |
|
||||
| `.. literalinclude:: P` (+opts) | ` ```{literalinclude} P ` with each option as a `:key: val` line |
|
||||
| `.. autofunction::` / `.. autoclass::` | wrap in ` ```{eval-rst} ` … ` ``` ` (keep any adjacent `.. _label:` inside the same block) |
|
||||
| `.. list-table::` (+opts) | ` ```{list-table} ` (keep the `* -` / ` -` body; don't reflow to a Markdown table) |
|
||||
| `.. contents::` `:local:` | ` ```{contents} ` with `:local:` |
|
||||
| `.. toctree::` | ` ```{toctree} ` — entries stay **extensionless** |
|
||||
| `.. include:: f.rst` (you're converting `f`) | ` ```{include} f.md ` (convert the included file in the same PR) |
|
||||
| `.. include:: _shared.rst` (shared partial, stays `.rst`) | ` ```{include} _shared.rst ` with `:parser: rst` (don't convert a shared `_includes/` partial) |
|
||||
| `.. image:: URL` | `` (match the `` style in `docs.md`) |
|
||||
| `::` literal block | a plain ` ``` ` fence (no language) — see Construct notes |
|
||||
| auto-lettered list `a.` / `b.` / `c.` | numbered `1.` / `2.` / `3.` — **MyST/CommonMark has no alpha lists** |
|
||||
|
||||
---
|
||||
|
||||
## Hard rules (get these wrong → broken build or lost test coverage)
|
||||
|
||||
1. **Preserve every label name exactly.** `.. _name:` → `(name)=` (own line, blank line, then the heading it labeled). External `{ref}`/`:ref:` callers resolve by **name** and are format-agnostic, so an unchanged label keeps working from `.rst` and `.md` callers alike. A renamed or dropped label breaks every caller. Labels sitting directly above an autodoc directive stay **inside** the `{eval-rst}` block as RST (`.. _name:` next to `.. autofunction::`); targets created inside `eval-rst` still register globally. A label directly above a non-heading directive (e.g. a `.. warning::`) becomes `(name)=` immediately before the converted `:::{warning}` — it still anchors.
|
||||
|
||||
2. **Links — translate, don't transcribe.** Three RST link forms need real translation; left as-is they emit a `myst.xref_*` warning (→ build failure):
|
||||
- **Whole-doc links** should use the `` {doc}`text <doc>` `` role — it resolves to the document and is never ambiguous. A bare `[text](sibling.rst)` (or `[text](sibling.md)` pointing at an `.rst` source) emits `myst.xref_missing`. An **extensionless** `[text](sibling)` works *only if* the target doc has no same-named label; if it does (e.g. a page carrying both the doc name `getting-involved` and a `(getting-involved)=` label), the bare link is ambiguous and emits `myst.xref_ambiguous`. So just use `{doc}`. This bites in *both* directions: a converted file linking to a still-`.rst` sibling, **and** an already-`.md` sibling whose link to the file you renamed now points at a dead `.rst`. (Re-check the bare-stem grep from pre-flight.)
|
||||
- **Same-page section links** written as a raw `page.html#section` URL must become a `#section` fragment (`[text](#section)`), resolved via `myst_heading_anchors`. The `.html#` URL renders in RST but MyST treats it as a cross-reference target and can't find it.
|
||||
- **`{ref}`** links are exempt (resolve by label, not path). **Extensionless** links and toctree entries are exempt (Sphinx resolves to whichever source exists).
|
||||
- These `myst.xref_*` classes and their fixes are also encoded as machine-readable rules in [`sphinx-fix/rules.yaml`](../sphinx-fix/rules.yaml) — the canonical category→fix table the `sphinx-fix` skill uses to diagnose a failing build. It's one shared source; keep the two in sync.
|
||||
|
||||
3. **Heading levels are assigned by ORDER OF FIRST APPEARANCE of each underline style — not by the character.** The same `-` underline can be `##` in one file and `###` in another, depending on what appeared before it. Overline+underline is a distinct style from underline-only. Walk the file top to bottom, assign level 1 to the first style seen, level 2 to the next new style, and so on; reproduce that exactly. Do **not** "fix" surprising nesting (e.g. a section that lands one level too deep) — that's restructuring and changes anchors. When unsure, check the live `/en/master` render of the page and match it.
|
||||
|
||||
4. **doctest/testcode: literal-vs-executed.** A meta-doc that *demonstrates* testcode often contains two kinds of blocks:
|
||||
- **Illustrative** — shown as syntax to copy. In RST they follow a `::` and are indented (a `literal_block`). Convert to a **plain ` ``` ` fence** (no language). These render but are **never executed**. Leaving the RST directive text (`.. testcode::`) as literal content inside the fence is correct and faithful.
|
||||
- **Real** — actually run and rendered. In RST they're column-0 `.. testcode::` / `.. doctest::` directives. Convert to `{testcode}` / `{doctest}` / `{testoutput}` fences.
|
||||
Decide **per block**. An illustrative block converted to a directive will execute and fail; a real block left as a plain fence silently loses CI coverage. After converting, count the executed directives and confirm the number matches the original's real blocks. (Note: a `{testcode}` in a doctest-*excluded* file still renders but doesn't run — see Hard rule 5.)
|
||||
|
||||
5. **`doc/BUILD.bazel` doctest exclusions.** The main `doctest(` rule globs `source/**/*.md` **and** `source/**/*.rst` with a per-file `exclude` list. If a file you convert is named in that exclude list, **rewrite its entry from `.rst` to `.md` in the same PR.** Otherwise the `*.md` glob pulls the newly-converted file **into** doctest, and blocks that were excluded for a reason (e.g. `ray.init(...)` with no `import ray`) execute and fail. Conversely, a file that is *included* (not excluded) stays tested as `.md` — that's when Hard rule 4 matters most.
|
||||
|
||||
---
|
||||
|
||||
## Construct notes
|
||||
|
||||
- **`default_role = "code"`** (`doc/source/conf.py`): an RST single-backtick already renders as inline code, so single-backtick → single-backtick is byte-faithful, not a rendering change.
|
||||
- **Admonitions**: prefer colon fences `:::{note}` … `:::` (the `colon_fence` MyST extension is on). They nest a ` ``` ` code fence cleanly without backtick-counting. Backtick ` ```{note} ` also works for simple admonitions with no nested fence. A one-line RST admonition (`.. note:: text`) becomes `:::{note}` / `text` / `:::`.
|
||||
- **sphinx-design `tab-set` / `tab-item` / `dropdown`**: use **colon fences**, not backtick fences — `::::{tab-set}` › `:::{tab-item} Label` › ` ```code ``` `. The outer fence needs **more colons** than the one it contains (4 vs 3), and colon fences nest cleanly around backtick code fences, so you avoid backtick-counting entirely. Put directive options (`:open:`, `:sync:`, …) on their own line right after the opener. (Confirmed against Ray's RtD build.)
|
||||
- **`linkify` is OFF** (not in `myst_enable_extensions`). A bare URL will **not** autolink — wrap it as `<https://…>` to preserve the hyperlink. This includes URLs in parentheses like `Bazel 7.5.0 (https://…)` → `(<https://…>)`.
|
||||
- **The `::` literal-block marker**: docutils drops `" ::"` when it's preceded by whitespace (`"…sessions. ::"` → `"…sessions."`) and replaces `"x::"` (no space) with `"x:"`. Reproduce the resulting prose, then put the block in a plain ` ``` ` fence.
|
||||
- **`list-table`**: keep the directive (` ```{list-table} `), move options to `:key: val` lines, and de-indent the `* -` / ` -` body to column 0. Don't convert it into a native Markdown table.
|
||||
- **Nested fences**: an outer fence must use **more backticks** than any fence it contains (or use a `:::` colon fence as the outer). Inside an ordered-list item, indent a nested ` ``` ` fence to the item's content column (3 spaces under `1. `).
|
||||
- **`{eval-rst}` for autodoc** is the safe default; native `{autofunction}` is a fallback only if the build is verified clean. Keep the option indentation the RST used.
|
||||
- **Include-only content partials** (a file that exists only to be `.. include::`d, like `involvement.rst`): give the `.md` **no frontmatter and no title** — it's spliced into its includer, and frontmatter would render mid-page there. These files don't orphan-warn even though they're not in any toctree (Sphinx doesn't treat included files as standalone docs). Inline any named-reference link targets in the partial, so they don't collide with the same target defined in the includer (RST tolerated the duplicate; inlining sidesteps it).
|
||||
- **No alpha-enumerated lists**: MyST/CommonMark ordered lists are numeric only. Convert `a.`/`b.`/`c.` sub-lists to `1.`/`2.`/`3.` (a forced, minor rendering change — call it out in the PR). A column-0 code block between numbered items breaks the list, but explicit numbers still render correctly (faithful to the RST).
|
||||
|
||||
---
|
||||
|
||||
## Reference updates — what changes, what doesn't
|
||||
|
||||
**No-ops (don't touch / don't scope-creep):**
|
||||
|
||||
- **Toctree entries** — already extensionless; Sphinx resolves to whichever source exists.
|
||||
- **Extensionless links** to the converted doc (e.g. `[text](page)` with no extension) — resolve fine.
|
||||
- **`.html` URL references** — `doc/redirects/current.yaml`, `CONTRIBUTING.rst`, semgrep/lint scripts, and **absolute** `https://docs.ray.io/…/page.html#sec` links point at HTML output, which is identical regardless of source format. (Only *relative* `page.html#sec` self-links need fixing — Hard rule 2.)
|
||||
- **External `{ref}`/`:ref:` callers** — safe as long as labels are preserved (Hard rule 1).
|
||||
|
||||
**Do change (same PR as the file):**
|
||||
|
||||
- **`doc/BUILD.bazel`** — doctest `exclude` entries (Hard rule 5) and any explicit doc-code test target naming the `.rst`.
|
||||
- **`.. include::` / `{include}`** directives pointing at a file you're converting (convert both).
|
||||
- **Relative `.rst` links from sibling pages** to the file you're renaming — found via the bare-stem grep. Point them at the new doc (extensionless or `{doc}`).
|
||||
- **`.claude/` path mentions** of the file (e.g. `CLAUDE.md`, skill/rule files referencing `…/development.rst`). Re-grep `.claude/` for the stem. These are tiny string edits and `.claude/` isn't in `.buildkite/test.rules.txt`, so they don't pull extra CI suites.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
1. **Static checks** on each new `.md` — frontmatter parses (skip this for include-only partials, which have none), backtick **and** `:::` colon fences balance, no residual RST leaked outside fences, every label present, executed-directive counts match. Sketch:
|
||||
|
||||
```python
|
||||
import re, yaml
|
||||
for f in FILES: # the new .md paths
|
||||
t = open(f).read()
|
||||
if t.startswith('---\n'): # partials have no frontmatter
|
||||
assert yaml.safe_load(t.split('---\n', 2)[1])['myst']['html_meta']['description']
|
||||
L = t.splitlines()
|
||||
assert sum(ln.lstrip().startswith('```') for ln in L) % 2 == 0, f"unbalanced ``` {f}"
|
||||
assert sum(bool(re.match(r'^:{3,}\{', ln)) for ln in L) == sum(bool(re.match(r'^:{3,}\s*$', ln)) for ln in L), f"unbalanced colon fences {f}"
|
||||
infence = False # residual RST outside ``` fences
|
||||
for i, ln in enumerate(L, 1):
|
||||
if ln.lstrip().startswith('```'): infence = not infence; continue
|
||||
if infence: continue
|
||||
for pat in (r':(ref|doc):`', r'^\.\. ', r'`[^`]*<[^>]*>`_', r'[^:]::\s*$'):
|
||||
if re.search(pat, ln): print(f"RESIDUAL {f}:{i}: {ln!r}")
|
||||
```
|
||||
|
||||
Also `grep -c '^```{testcode}'` (etc.) and confirm the count equals the original's real blocks; grep for every label you noted in pre-flight; and `grep -n '\.html#\|](.*\.rst)'` to catch any link form Hard rule 2 forbids.
|
||||
|
||||
2. **Build (decisive parse check)** — `pre-commit run --files <changed>` is effectively a no-op for most `doc/source/**/*.md` (vale is scoped to `doc/source/data/`, prettier to js/ts/html/css), so the real check is Sphinx. A full local build is heavy; the practical signal is the **Read the Docs PR preview** (`docs/readthedocs.com:anyscale-ray`). With `fail_on_warning`, a green RtD build proves every label, `{ref}`, link, toctree entry, `{literalinclude}`, `{eval-rst}`, `{list-table}`, and sphinx-design directive resolved. **Read the raw RtD log on failure**: the build page lazy-loads, so fetch `https://app.readthedocs.com/api/v2/build/<BUILD_ID>.txt` and grep for `WARNING:`/`ERROR:`. `-W --keep-going` lists all warnings of a build that *completes* — but a hard-broken build (a `conf.py`/extension error, a traceback, or `SEVERE:`) aborts before producing that list, and a parse error or broken `toctree` masks the xref/orphan warnings beneath it. So if a failed build's log is empty or short, or fixing one error reveals new ones, fix the highest-severity error first and **re-run** — don't trust a single pass to be complete.
|
||||
|
||||
3. **Doctest (only for files the doctest rule includes)** — the RtD html builder does **not** execute testcode/doctest; that runs in the Buildkite doctest target (surfaces under `buildkite/microcheck` for a changed doc file). Confirm the real executed blocks pass and the illustrative ones don't run. MyST `{testcode}`/`{doctest}` in `.md` *is* exercised — `getting-started.md` and `configure-manage-dashboard.md` are tested `.md` precedents.
|
||||
|
||||
4. **Regression** — compare the RtD preview against **`/en/master`** (not `/en/latest`). Rendered content should match except where light cleanup intentionally changed it.
|
||||
|
||||
---
|
||||
|
||||
## Verified Ray-specific facts (as of mid-2026)
|
||||
|
||||
- `doc/source/conf.py`: `default_role = "code"`; `myst_enable_extensions` includes `colon_fence` but **not** `linkify`; `myst_heading_anchors = 3` (so `[text](#slug)` resolves to any h1–h3 heading).
|
||||
- `doc/BUILD.bazel` main `doctest(` rule globs `source/**/*.md` + `source/**/*.rst`, with a per-file `exclude` list (e.g. `ray-contribute/getting-involved.md`, `ray-contribute/testing-tips.md`) and whole-subtree excludes for `ray-core/`, `data/`, `rllib/`, `serve/`, `train/`, `tune/` (which have their own `doctest` rules).
|
||||
- `pre-commit` has no hook that lints `doc/source/**/*.md` outside `doc/source/data/` (vale) — so pre-commit passing is not evidence the page is correct; the Sphinx build is.
|
||||
- The `ray-contribute/` directory was the first batch fully migrated (precedent for every pattern above, including sphinx-design tabs/dropdowns in `development.md` and the shared-include + partial handling in `getting-involved.md` / `involvement.md`).
|
||||
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Heading level surprises are usually faithful, not bugs.** If a section renders one level deeper than feels right, the RST adornment order put it there. Reproduce it; don't fix it in a conversion PR.
|
||||
- **Two `#` (H1) headings in one page** is fine when the RST had two top-level (`=`) sections.
|
||||
- **Trailing whitespace inside code blocks** can be dropped (invisible, no linter on these files) — don't preserve it deliberately.
|
||||
- **A converted file in the doctest exclude list but you forgot to update BUILD.bazel** is a likely silent break: the build stays green, but the doctest target starts running blocks that were never meant to run. Always re-grep `doc/BUILD.bazel` for the stem.
|
||||
- **A sibling's *relative* link to the file you're renaming** (`[text](./page.rst)`) is the easiest reference to miss — it has no `dir/` prefix, so a path-scoped grep won't catch it. Grep the bare stem across all of `doc/`.
|
||||
- **`.html#anchor` self-links and bare `.md`→`.rst` links** are faithful-but-wrong: they render in RST but fail `fail_on_warning` in MyST. Translate them (Hard rule 2).
|
||||
- **An extensionless link to a doc that also carries a same-named label is ambiguous** (`myst.xref_ambiguous`), not missing — MyST can't tell the doc from the label. Use `{doc}` for whole-doc links so it always resolves to the document.
|
||||
- **Don't trust pre-commit's silence** as a quality signal for these `.md` files — it skips them.
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
name: sphinx-fix
|
||||
description: Diagnose a failing Ray Sphinx / Read the Docs documentation build. Parses the Sphinx warning stream (an RtD build log, a local build, or pasted text), classifies each warning against a rules table, and proposes the canonical fix in severity-tier order. Detects a hard-broken build, segregates known-benign suppressed classes, and lists every unclassified warning. Use when a `docs/readthedocs.com:anyscale-ray` check fails, when asked "why is the docs build failing?" or "what warning is breaking this PR?", or to turn a Sphinx warning dump into an ordered fix list.
|
||||
user-invocable: true
|
||||
argument-hint: <warnings file | pasted Sphinx log | '-' for stdin>
|
||||
---
|
||||
|
||||
# Diagnose and fix Ray Sphinx doc-build warnings
|
||||
|
||||
The Ray docs build with `fail_on_warning: true` (`.readthedocs.yaml`), so **any single Sphinx warning fails the build**. The expensive part of a docs change is usually not the edit — it's *finding* the warning. This skill turns a warning stream into a compact, severity-ordered list of findings, each with its canonical fix.
|
||||
|
||||
It is **deterministic and read-only**: a stdlib-only script parses and classifies; it never edits files. v0 is **human-in-the-loop** — you (or the agent) apply each fix and rebuild. Nothing is auto-applied.
|
||||
|
||||
Where it sits among the doc-build skills:
|
||||
|
||||
- **`sphinx-fix`** (this skill) — *diagnose build output*: classify warnings, propose fixes.
|
||||
- **`rst-to-myst`** — *convert source*: migrate `.rst` pages to MyST `.md`. It shares one rules source with this skill (`rules.yaml` below).
|
||||
- **`rtd_doctor.py`** (`doc/rtd_doctor.py`) — *preflight the environment*: assert the local toolchain matches Read the Docs before you build.
|
||||
|
||||
---
|
||||
|
||||
## When to use
|
||||
|
||||
- A `docs/readthedocs.com:anyscale-ray` PR check failed and you need to know why and how to fix it.
|
||||
- You have a Sphinx warning dump (from RtD, a local build, or a paste) and want it classified.
|
||||
- You're iterating on a docs PR and want to confirm the build is clean before pushing.
|
||||
|
||||
**Not for:** authoring or converting pages (use `rst-to-myst`), setting up the build environment (use `rtd_doctor.py` + `make rtd`), or fetching the rendered HTML (open the preview URL).
|
||||
|
||||
---
|
||||
|
||||
## Input modes
|
||||
|
||||
Both modes feed the same parser and rules table; only the log source differs. The parser tolerates surrounding log noise, so you can pipe a raw build log in without pre-filtering.
|
||||
|
||||
1. **A warning stream you already have** — paste it to a file, or pipe it. This includes the output of the team's Read the Docs log tooling (the private `/rtd-build-logs` skill's `warnings` subcommand), the RtD build page's raw log, or any captured Sphinx output. This is the cheapest mode and needs no local build.
|
||||
|
||||
2. **A local build** — capture stderr (and, with `-W -w warnings.txt`, a warnings file) from a full `make rtd` / `sphinx-build` run and feed it in. Use this for a tight local loop. A local build must be a **full** build for cross-reference checks to be faithful — an incremental `make local` pulls a cache and misses cross-file breakage (a rename that breaks a link in an *unchanged* file).
|
||||
|
||||
---
|
||||
|
||||
## Running the engine
|
||||
|
||||
```bash
|
||||
# From a file (or '-'/omit for stdin)
|
||||
python doc/.claude/skills/sphinx-fix/sphinx_fix.py warnings.txt
|
||||
|
||||
# Piped (e.g. a local build, or the team's RtD log tool)
|
||||
python -m sphinx -T -W --keep-going -b html . _build/html 2>&1 \
|
||||
| python doc/.claude/skills/sphinx-fix/sphinx_fix.py
|
||||
|
||||
# JSON for programmatic use
|
||||
python doc/.claude/skills/sphinx-fix/sphinx_fix.py warnings.txt --json
|
||||
|
||||
# Read one rule's cause/fix/validated-versions
|
||||
python doc/.claude/skills/sphinx-fix/sphinx_fix.py --explain myst-xref-missing-cross-extension
|
||||
```
|
||||
|
||||
Stdlib-only — no install. It prefers PyYAML to read `rules.yaml` but falls back to a bundled parser, so it runs in a bare environment. Optionally pass `--sphinx-version`/`--myst-version` to gate findings against each rule's validated range.
|
||||
|
||||
**Exit codes:** `0` clean · `1` classified findings to fix · `2` hard-broken build (abort) · `3` only unclassified warnings (no rule matched). Precedence `2 > 1 > 3 > 0`.
|
||||
|
||||
---
|
||||
|
||||
## Reading the output
|
||||
|
||||
The report has five parts, in priority order:
|
||||
|
||||
1. **Abort banner** (only if present) — a hard-broken build (a `conf.py`/extension error, a traceback, or `SEVERE:` with no completion summary) aborted before the warning pass. **Fix this first; the rest of the log is unreliable.** For an autosummary import abort, the banner adds a `hint:` — the named module is usually a decoy for an unrelated module whose import chain broke (often a dep mocked by `autodoc_mock_imports` but used at import time); trace it and make the eager import lazy.
|
||||
2. **Root-cause groups** (only if present) — a single structural root (an ungenerated `autosummary` stub for a module or class) masks a flood of downstream `py:* reference target not found` warnings for the *same* objects. Rather than list the flood as N equal-looking rows, the report collapses it: `ROOT [T2] ... — <prefix>.* (N stubs not generated, masking M downstream references)`, the root's fix once, then the masked references collapsed under it (capped in the rendered view, with the dropped count shown). **Fix the root, rebuild, and the masked references clear together** — don't chase the individual references. The full flat list is always in `--json`.
|
||||
3. **Findings**, grouped by tier (1 → 2 → 3), for everything not absorbed into a root-cause group. Each row is `[T2] <rule-id> <path:line>` plus the warning message, the canonical `fix:` (with a `Suggested:` rewrite for mechanical rules), and a `safety:` flag. `judgment` findings need your decision.
|
||||
4. **Unclassified** — warnings no rule matched. **Never silently dropped.** Resolve each with the user, then file a skill-improvement ticket so `rules.yaml` gains a rule (see below).
|
||||
5. **Suppressed** — known-benign classes the build already filters; not actionable.
|
||||
|
||||
---
|
||||
|
||||
## Severity tiers and the iterate loop
|
||||
|
||||
`-W --keep-going` lists every warning **of a build that completes**. Two things break the "one pass finds everything" assumption, so the skill triages by tier and you must iterate:
|
||||
|
||||
- **Tier 1 — fatal/abort.** The build died before the warning pass. `--keep-going` does not save you. Fix it, rebuild, re-run.
|
||||
- **Tier 2 — structural/parse.** A broken `toctree`, a duplicate label, or a failed `literalinclude` corrupts the doc graph and **masks the warnings beneath it**. Fix all tier-2 findings, then rebuild — a fresh batch of tier-3 warnings often appears.
|
||||
- **Tier 3 — plain warnings.** These mask nothing; fix them in any order.
|
||||
|
||||
**Always fix the highest tier first, then re-acquire and re-parse.** Never report "all clear" after a single pass when a tier-1 or tier-2 condition was present — the report's verdict line says so explicitly.
|
||||
|
||||
---
|
||||
|
||||
## Applying fixes (human-in-the-loop)
|
||||
|
||||
The engine proposes; you apply. For each finding:
|
||||
|
||||
- **Mechanical** fixes (link rewrites, fragment links, toctree entries, `literalinclude` ranges, image paths) — apply the `Suggested:` rewrite, confirming the target. For the MyST link rewrites, cross-check **`rst-to-myst` Hard rule 2** (the `{doc}`-role and `#fragment` rules are shared).
|
||||
- **Judgment** fixes — *you decide*: which of two duplicate labels to keep (preserve the name external `{ref}`/`:ref:` callers use), or whether an orphan page should join a toctree or be marked `:orphan:`.
|
||||
|
||||
A rule graduates to auto-apply only once there is evidence it is always safe — not in v0.
|
||||
|
||||
---
|
||||
|
||||
## Unclassified warnings → grow the rules table
|
||||
|
||||
The seed table is incomplete by construction (the Sphinx 8→9 march and contributors new to MyST will surface classes no historical log shows). When the report lists an unclassified warning:
|
||||
|
||||
1. Resolve it with the user — determine the cause and the canonical fix.
|
||||
2. Prompt the user to file a **skill-improvement ticket** capturing the new category/signature and its fix.
|
||||
3. Add a rule to `rules.yaml` (and, if it's a link class, keep it consistent with `rst-to-myst`).
|
||||
|
||||
This feedback loop is the intended way the table grows — from real misses, not a one-time corpus.
|
||||
|
||||
---
|
||||
|
||||
## conf.py suppression awareness
|
||||
|
||||
The build already suppresses or filters known-benign classes in `doc/source/conf.py` (e.g. `misc.copy_overwrite`, and docutils footnote noise from fetched `_collections` content). The engine segregates these into the **Suppressed** bucket and never proposes a fix — do not "fix" them. They are tracked for removal under their own tickets ([DOC-1257](https://anyscale1.atlassian.net/browse/DOC-1257), [DOC-1258](https://anyscale1.atlassian.net/browse/DOC-1258)); when a suppression is lifted and its warning resurfaces, convert its `suppressions:` entry in `rules.yaml` into a rule.
|
||||
|
||||
---
|
||||
|
||||
## Version-bump re-validation
|
||||
|
||||
`rules.yaml` records a `baseline` (Sphinx/myst versions) and a per-rule validated range. MyST and Sphinx bumps rename or retire `[category]` tags (the tag default itself flipped at Sphinx 8.0), so matching also keys on message **signatures** as forward-compatibility insurance. After a dependency bump, re-confirm the category strings and signatures against a real build; any finding the engine flags `(unvalidated for this version)` is the re-validation worklist. Some core-Sphinx rules are signature-only today pending confirmation of their exact `[category]` on Sphinx 8.2.3 — confirm and add the category as you encounter them.
|
||||
|
||||
---
|
||||
|
||||
## Done
|
||||
|
||||
A change is verified when a full `make rtd` (or the green RtD PR preview) shows no warnings, and a regression check against `/en/master` shows the rendered content unchanged except where you intended (per `doc/.claude/CLAUDE.md`).
|
||||
|
||||
---
|
||||
|
||||
## Maintaining this skill
|
||||
|
||||
- `rules.yaml` is the single source of truth and stays within a restricted YAML subset so the stdlib fallback can parse it (see the header comment in the file). Don't introduce flow collections or block scalars.
|
||||
- After editing `rules.yaml` or `sphinx_fix.py`, run the selftest: `python doc/.claude/skills/sphinx-fix/sphinx_fix.py --selftest`. It checks schema invariants, the fixture goldens, and that the fallback parser agrees with PyYAML. Regenerate goldens after an intentional output change with `--update-golden` (and eyeball the diff).
|
||||
@@ -0,0 +1,299 @@
|
||||
# Canonical Sphinx/MyST doc-build warning -> fix rules table.
|
||||
#
|
||||
# This is the single source of truth for the `sphinx-fix` skill's classify/fix
|
||||
# engine (sphinx_fix.py) and is referenced by the `rst-to-myst` skill as the
|
||||
# canonical category->fix catalog. Keep one table; do not duplicate it.
|
||||
#
|
||||
# GRAMMAR CONSTRAINT (important): sphinx_fix.py prefers PyYAML but falls back to
|
||||
# a small stdlib-only parser, so this file must stay within a restricted YAML
|
||||
# subset: 2-space indentation, block maps and block sequences only (no flow
|
||||
# `{...}`/`[...]` collections), no anchors/aliases, no `>`/`|` block scalars.
|
||||
# Write every value that contains special characters as a DOUBLE-QUOTED string,
|
||||
# and write every regex backslash as `\\` (e.g. `\\.` for a literal dot, `\\s`
|
||||
# for whitespace) so it is a valid YAML escape. The --selftest cross-checks the
|
||||
# fallback against PyYAML, so a grammar slip is caught.
|
||||
#
|
||||
# MATCH SEMANTICS: a rule "fires" when, per `match`, either any (default) or all
|
||||
# of its `categories`/`signatures` hit. `categories` test the trailing
|
||||
# `[category]` tag; `signatures` are regexes searched against the message text.
|
||||
# When several rules fire for one warning, the engine prefers a rule with a
|
||||
# signature hit over a category-only hit, then earliest in this file. Author
|
||||
# most-specific rules first.
|
||||
#
|
||||
# TIERS: 1 = fatal/abort (NOT represented as a rule -- the engine detects a
|
||||
# hard-broken build structurally from the log shape); 2 = structural/parse
|
||||
# errors that corrupt the toctree/label graph or fail a document's parse and so
|
||||
# MASK warnings beneath them (fix these first, then rebuild); 3 = plain warnings
|
||||
# that mask nothing.
|
||||
#
|
||||
# SAFETY: `mechanical` = a deterministic rewrite; `judgment` = needs a human
|
||||
# decision. v0 is human-in-the-loop and proposes both; nothing is auto-applied.
|
||||
#
|
||||
# VERSION RANGES: `version_sphinx`/`version_myst` record the versions a rule was
|
||||
# validated against. A warning from outside the range still matches but is
|
||||
# flagged `(unvalidated)`; dependency bumps are the trigger to re-confirm the
|
||||
# category strings and signatures.
|
||||
#
|
||||
# NOTE: the MyST categories below (`myst.xref_missing`, `myst.xref_ambiguous`)
|
||||
# are confirmed against real Ray RtD builds. The core-Sphinx classes
|
||||
# (toctree/orphan/duplicate-label/literalinclude) are matched by signature only
|
||||
# pending confirmation of their exact `[category]` tag on Sphinx 8.2.3; add the
|
||||
# category when confirmed. Unmatched warnings drive the skill-improvement loop.
|
||||
|
||||
version: 1
|
||||
baseline_sphinx: "8.2.3"
|
||||
baseline_myst: "5.1.0"
|
||||
|
||||
rules:
|
||||
- id: myst-xref-missing-cross-extension
|
||||
title: "Bare cross-extension link to an .rst/.md source"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: all
|
||||
categories:
|
||||
- myst.xref_missing
|
||||
signatures:
|
||||
- "cross-reference target not found: '[^']+\\.(?:rst|md)'"
|
||||
cause: "A bare [text](sibling.rst) or [text](sibling.md) link pointing at an .rst source. MyST flags it as a missing xref and fail_on_warning turns it into a build failure."
|
||||
fix: "Rewrite as a {doc} role so it resolves to the document and is never ambiguous; drop the extension on the target."
|
||||
fix_template: "{doc}`text <{target_stem}>`"
|
||||
target_extract: "cross-reference target not found: '(?P<target>[^']+)'"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
version_myst: ">=5.0,<6"
|
||||
notes: "Shared with rst-to-myst Hard rule 2. target_stem = target minus its extension. On a category rename, re-validate (this rule needs the myst.xref_missing tag)."
|
||||
|
||||
- id: myst-xref-missing-html-fragment
|
||||
title: "Relative .html# self-link read as a cross-reference"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: all
|
||||
categories:
|
||||
- myst.xref_missing
|
||||
signatures:
|
||||
- "cross-reference target not found: '[^']*\\.html#"
|
||||
cause: "A relative page.html#section URL. RST renders it, but MyST treats it as an xref target and cannot find it."
|
||||
fix: "Rewrite as a same-page fragment link [text](#section), resolved via myst_heading_anchors (h1-h3)."
|
||||
fix_template: "[text](#{sec})"
|
||||
target_extract: "cross-reference target not found: '[^']*\\.html#(?P<sec>[^']+)'"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
version_myst: ">=5.0,<6"
|
||||
notes: "Shared with rst-to-myst Hard rule 2."
|
||||
|
||||
- id: myst-xref-missing-renamed-doc
|
||||
title: "Missing MyST xref (residual: stale/renamed doc target)"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- myst.xref_missing
|
||||
cause: "A myst.xref_missing whose target is not a cross-extension or .html# link -- usually a stale relative link to a doc that was renamed or moved (often from a sibling page)."
|
||||
fix: "Repoint the link to the current document, preferring the {doc} role (or an extensionless target). Grep the bare stem across doc/ to find every caller."
|
||||
fix_template: "{doc}`text <DOC>`"
|
||||
target_extract: "cross-reference target not found: '(?P<target>[^']+)'"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
version_myst: ">=5.0,<6"
|
||||
notes: "Residual within the myst.xref_missing family: category-only, so it loses to the cross-extension and html-fragment rules when their signature hits."
|
||||
|
||||
- id: myst-xref-ambiguous
|
||||
title: "Ambiguous extensionless link (doc and same-named label)"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- myst.xref_ambiguous
|
||||
signatures:
|
||||
- "(?:not unique|matches more than one)"
|
||||
cause: "An extensionless [text](name) link where `name` is both a document and a (name)= label, so MyST cannot tell which you meant."
|
||||
fix: "Use the {doc} role for the whole-doc link so it always resolves to the document."
|
||||
fix_template: "{doc}`text <{target_stem}>`"
|
||||
target_extract: "target (?:is not unique|not unique): '?(?P<target>[^'\\s]+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
version_myst: ">=5.0,<6"
|
||||
notes: "Shared with rst-to-myst Hard rule 2."
|
||||
|
||||
- id: unknown-document
|
||||
title: "{doc}/toctree reference to an unknown document"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- ref.doc
|
||||
signatures:
|
||||
- "unknown document:"
|
||||
cause: "A {doc} role, a :doc: reference, or a toctree entry points at a document path that does not resolve -- a renamed or moved page, a typo, or a relative path that is wrong from the referencing file. Distinct from toctree-nonexisting, which is the toctree-directive-specific wording."
|
||||
fix: "Repoint to the correct document: a path relative to the referencing page, or an absolute path from doc/source with a leading slash. Remove the reference if the target is gone. Grep the bare stem across doc/ to find every caller."
|
||||
target_extract: "unknown document: '?(?P<target>[^'\\s]+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Category ref.doc confirmed on real Ray RtD builds; the tag is sometimes absent on the first emission, so the signature carries the match."
|
||||
|
||||
- id: toctree-nonexisting
|
||||
title: "toctree references a nonexisting document"
|
||||
tier: 2
|
||||
safety: mechanical
|
||||
match: any
|
||||
signatures:
|
||||
- "toctree contains reference to nonexisting document"
|
||||
cause: "A toctree entry points at a document that does not exist -- a typo or a renamed/removed file. A broken toctree masks the warnings beneath it."
|
||||
fix: "Fix the toctree entry (entries are extensionless) to the current document name, or remove it."
|
||||
target_extract: "nonexisting document '?(?P<target>[^'\\s]+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Signature-only pending confirmation of the [category] tag on Sphinx 8.2.3."
|
||||
|
||||
- id: orphan-not-in-toctree
|
||||
title: "Document not included in any toctree (orphan)"
|
||||
tier: 3
|
||||
safety: judgment
|
||||
match: any
|
||||
signatures:
|
||||
- "document isn't included in any toctree"
|
||||
cause: "A page that no toctree references. Sphinx warns so pages are not silently unreachable."
|
||||
fix: "Add the page to the appropriate toctree, OR add `:orphan:` (RST) / an `orphan: true` frontmatter field (MyST) if it is intentionally standalone. Human decides which."
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Signature-only pending [category] confirmation."
|
||||
|
||||
- id: duplicate-label
|
||||
title: "Duplicate label or object description"
|
||||
tier: 2
|
||||
safety: judgment
|
||||
match: any
|
||||
signatures:
|
||||
- "duplicate (?:label|object description)"
|
||||
cause: "The same label (or autodoc object) is defined in two places, so references to it are ambiguous and the label graph is corrupted."
|
||||
fix: "Dedupe -- remove or rename one definition. Which one to keep is a judgment call; preserve the name external {ref}/:ref: callers use."
|
||||
target_extract: "duplicate (?:label|object description) (?:of )?'?(?P<target>[^'\\s]+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Signature-only pending [category] confirmation. The intentional ray.actor.ActorMethod.bind duplicate is pre-filtered by a suppression below."
|
||||
|
||||
- id: undefined-ref-label
|
||||
title: "Undefined label in a {ref}"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- ref.ref
|
||||
signatures:
|
||||
- "undefined label"
|
||||
cause: "A {ref}/:ref: points at a label that does not exist -- usually a renamed or dropped label. Label names are load-bearing for every caller."
|
||||
fix: "Restore the exact label name (do not invent a new one), or repoint the {ref} at the correct existing label."
|
||||
target_extract: "undefined label:?\\s+'?(?P<target>[^'\\s]+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Category ref.ref is a best-effort guess; signature carries the match."
|
||||
|
||||
- id: py-xref-target-not-found
|
||||
title: "Python-domain reference target not found (meth/func/class/obj/exc)"
|
||||
tier: 3
|
||||
safety: judgment
|
||||
match: any
|
||||
categories:
|
||||
- ref.meth
|
||||
- ref.obj
|
||||
- ref.func
|
||||
- ref.class
|
||||
- ref.exc
|
||||
- ref.attr
|
||||
- ref.mod
|
||||
- ref.python
|
||||
signatures:
|
||||
- "py:\\w+ reference target not found:"
|
||||
cause: "A Python-domain role ({meth}/{class}/{func}/{obj}/{exc}/:py:*:) points at an object not in the resolved API inventory. Two very different roots: (1) the object is documented elsewhere but its autosummary stub was not generated, or its module aborted import, so the inventory entry is missing -- these arrive as a large flood sharing an object prefix and are DOWNSTREAM symptoms, not separate defects; (2) the reference is genuinely wrong -- a renamed, moved, or mistyped target, or a stdlib/third-party name that needs intersphinx."
|
||||
fix: "Check for a co-occurring tier-2 autosummary-stub-not-found or tier-1 import abort for the same module first. If present, fix that root (up to and including reverting an in-progress API-ref restructuring) and rebuild -- the flood clears at once. Only when the build is otherwise healthy is the reference itself wrong: repoint to the correct dotted path, or fix the intersphinx target for a stdlib/third-party name."
|
||||
target_extract: "py:\\w+ reference target not found: (?P<target>\\S+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Categories confirmed on real Ray RtD builds (ref.meth, ref.obj, ref.func, ref.python); the signature carries the rest of the py domain (ref.exc, ref.attr). safety=judgment: both the correct target and the correct root (edit one reference vs revert a restructuring) need a human decision -- never auto-apply. A correlated flood sharing an object prefix with autosummary-stub-not-found warnings is that rule's masked downstream; triage the tier-2 root, not each reference."
|
||||
|
||||
- id: image-not-readable
|
||||
title: "Image file not readable"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- image.not_readable
|
||||
signatures:
|
||||
- "image file (?:not readable|.* not readable)"
|
||||
cause: "An image path does not resolve from the referencing document."
|
||||
fix: "Fix the image path (relative to the referencing page, or an absolute /_static-style path)."
|
||||
target_extract: "image file (?:not readable|.* not readable):?\\s+(?P<target>\\S+)"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Category image.not_readable is a best-effort guess; signature carries the match."
|
||||
|
||||
- id: docutils-inline-markup
|
||||
title: "Unbalanced inline emphasis/strong markup"
|
||||
tier: 3
|
||||
safety: mechanical
|
||||
match: any
|
||||
categories:
|
||||
- docutils
|
||||
signatures:
|
||||
- "Inline .+ start-string without end-string"
|
||||
cause: "A docstring or RST body has an unbalanced inline marker -- a lone * (emphasis) or ** (strong) with no closing marker. Common in API docstrings that write *args or **kwargs, or a bare asterisk, outside a code span. It surfaces from an <autosummary> generated stub at the docstring's location."
|
||||
fix: "Balance or escape the marker in the source docstring, not in the generated stub: wrap literals in double backticks (``*args``, ``**kwargs``), or escape a standalone asterisk (\\*)."
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Category docutils confirmed on real Ray RtD builds. No target extraction: the message does not carry the offending text, so locate the unbalanced marker at the cited docstring."
|
||||
|
||||
- id: intersphinx-inventory-unreachable
|
||||
title: "intersphinx inventory not fetchable (network/transient)"
|
||||
tier: 3
|
||||
safety: judgment
|
||||
match: any
|
||||
signatures:
|
||||
- "failed to reach any of the inventories"
|
||||
cause: "Sphinx could not download an external intersphinx objects.inv (for example docs.python.org or spark.apache.org). On the RtD build network this is usually a transient connectivity failure, not a defect in the docs; with fail_on_warning it still fails the build."
|
||||
fix: "Re-run the build first -- this is usually transient. If it persists, the inventory URL in intersphinx_mapping (conf.py) is unreachable or wrong: confirm the URL. This is not a per-page content fix."
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Untagged (no [category]); signature-only. The URL and error detail sit on an unindented continuation line the parser does not capture as a warning record, so match on the WARNING line. safety=judgment: re-run vs investigate intersphinx_mapping is a human call."
|
||||
|
||||
- id: literalinclude-out-of-range
|
||||
title: "literalinclude line/marker out of range"
|
||||
tier: 2
|
||||
safety: mechanical
|
||||
match: any
|
||||
signatures:
|
||||
- "Line spec '[^']*' out of range"
|
||||
- "Object named '[^']*' not found in include file"
|
||||
cause: "A {literalinclude} :lines:/:start-after:/:end-before: no longer matches the source -- the included code moved or changed. The directive fails the document's parse."
|
||||
fix: "Update the :lines: range or the :start-after:/:end-before: markers to match the current source file."
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Signature-only pending [category] confirmation."
|
||||
|
||||
- id: autosummary-stub-not-found
|
||||
title: "autosummary stub file not generated for a listed member"
|
||||
tier: 2
|
||||
safety: judgment
|
||||
match: any
|
||||
signatures:
|
||||
- "autosummary: stub file not found '[^']+'\\. Check your autosummary_generate"
|
||||
cause: "An autosummary directive lists a member whose stub .rst was never generated. A lone entry usually means the member was renamed or removed. A bulk failure across a whole module or API-ref page means autosummary could not introspect the module -- commonly the module failed to import under the build's autodoc_mock_imports set, or an in-progress API-ref restructuring moved or renamed the source. It is a tier-2 structural condition: the ungenerated stubs mask the py:* reference-target-not-found warnings for the same objects (see py-xref-target-not-found)."
|
||||
fix: "Do not fix per entry when the failure is in bulk. Determine scope first: many missing stubs sharing a module prefix point at one root -- confirm the module imports under the build's mock set (autodoc_mock_imports) and that autosummary_generate is on; when an API-ref restructuring is in flight, the correct move may be to revert or repair the file moves rather than edit each entry. A lone missing stub means the member no longer exists -- update or drop the autosummary entry."
|
||||
target_extract: "stub file not found '(?P<target>[^']+)'"
|
||||
version_sphinx: ">=8.0,<9"
|
||||
notes: "Untagged core-Sphinx warning (no [category]); signature-only. Seen at scale in a Ray API-ref rework where dozens of missing stubs masked hundreds of downstream py:ref failures. Fix this tier-2 root first, rebuild, and most of the py-xref-target-not-found flood clears."
|
||||
|
||||
# Known-benign warning classes the build already suppresses or filters in
|
||||
# doc/source/conf.py. The engine segregates these into a not-actionable bucket
|
||||
# and never proposes a fix. Suppressions are checked BEFORE rules, so an
|
||||
# intentional duplicate (e.g. ActorMethod.bind) never reaches duplicate-label.
|
||||
# When a suppression is lifted (its tracking ticket), convert its row to a rule.
|
||||
suppressions:
|
||||
- id: misc-copy-overwrite
|
||||
categories:
|
||||
- misc.copy_overwrite
|
||||
reason: "Sphinx 8 copy-overwrite check fires on a redundant sphinxcontrib-redoc asset. Benign."
|
||||
tracked_by: "DOC-1257"
|
||||
|
||||
- id: config-cache
|
||||
categories:
|
||||
- config.cache
|
||||
reason: "sphinx_design stores an unpicklable value in config; benign cache warning. Pre-existing."
|
||||
|
||||
- id: actor-bind-duplicate
|
||||
signatures:
|
||||
- "duplicate object description of ray\\.actor\\.ActorMethod\\.bind"
|
||||
reason: "Intentional duplicate autodoc object; not a real defect."
|
||||
|
||||
- id: collections-fetched-footnotes
|
||||
signatures:
|
||||
- "autonumbered footnote references"
|
||||
- "Unknown target name"
|
||||
location_contains: "_collections"
|
||||
reason: "Docutils footnote errors in fetched example content under _collections; benign noise filtered in conf.py."
|
||||
tracked_by: "DOC-1258"
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,7 @@
|
||||
Build: 4100500 State: Finished Success: False Duration: 38s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 2
|
||||
|
||||
Running Sphinx v8.2.3
|
||||
Extension error:
|
||||
Could not import extension myst_parser (exception: No module named 'markdown_it')
|
||||
@@ -0,0 +1,17 @@
|
||||
Build: 4154865 State: Finished Success: False Duration: 113s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Running Sphinx v8.2.3
|
||||
[autosummary] generating autosummary for: data/api/loading_data.rst, train/api/doc/ray.train.Trainer.rst
|
||||
|
||||
Extension error (sphinx.ext.autosummary)!
|
||||
|
||||
Traceback
|
||||
=========
|
||||
|
||||
sphinx.ext.autosummary.ImportExceptionGroup: no module named ray.air.integrations.comet.CometLoggerCallback
|
||||
|
||||
sphinx.ext.autosummary.ImportExceptionGroup: no module named ray.air.integrations.comet
|
||||
|
||||
sphinx.errors.ExtensionError: Handler <function process_generate_options at 0x7f00> for event 'builder-inited' threw an exception (exception: no module named ray.air.integrations.comet)
|
||||
@@ -0,0 +1,22 @@
|
||||
Build: 4156675 State: Finished Success: False Duration: 1042s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (15):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.map_batches'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.map'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.filter'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.flat_map'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.groupby'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.sort'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.map_batches [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.map [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.filter [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.flat_map [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/transform.rst:10: WARNING: py:meth reference target not found: ray.data.Dataset.groupby [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/transform.rst:10: WARNING: py:meth reference target not found: ray.data.Dataset.sort [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/transform.rst:10: WARNING: py:meth reference target not found: ray.data.Dataset.limit [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/transform.rst:10: WARNING: py:meth reference target not found: ray.data.Dataset.show [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/python/ray/train/collective/__init__.py:docstring of ray.train.collective.broadcast_from_rank_zero:: WARNING: py:exc reference target not found: pickle.PicklingError [ref.exc]
|
||||
|
||||
Summary line: build finished with problems, 15 warnings.
|
||||
@@ -0,0 +1,11 @@
|
||||
Build: 4156675 State: Finished Success: False Duration: 1004s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (4):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.map_batches'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/ray.data.Dataset.rst:20: WARNING: autosummary: stub file not found 'ray.data.Dataset.map'. Check your autosummary_generate setting.
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.map_batches [ref.meth]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64387/doc/source/apis/data/aggregate.rst:6: WARNING: py:meth reference target not found: ray.data.Dataset.map [ref.meth]
|
||||
|
||||
Summary line: build finished with problems, 4 warnings.
|
||||
@@ -0,0 +1,7 @@
|
||||
Build: 4023375 State: Finished Success: True Duration: 536s
|
||||
Command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 0
|
||||
|
||||
Sphinx warnings (0):
|
||||
|
||||
Summary line: build succeeded.
|
||||
@@ -0,0 +1,9 @@
|
||||
Build: 4144240 State: Finished Success: False Duration: 604s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (2):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/releases-2.56.0/doc/source/data/api/doc/ray.data.ReaderFormat.rst:74:<autosummary>:1: WARNING: Inline emphasis start-string without end-string. [docutils]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/releases-2.56.0/doc/source/data/api/doc/ray.data.ReaderFormat.rst:74:<autosummary>:1: WARNING: Inline strong start-string without end-string. [docutils]
|
||||
|
||||
Summary line: build finished with problems, 2 warnings.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4159413 State: Finished Success: False Duration: 1078s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
WARNING: failed to reach any of the inventories with the following issues:
|
||||
intersphinx inventory 'https://docs.python.org/3/objects.inv' not fetchable due to <class 'requests.exceptions.ConnectionError'>: HTTPSConnectionPool(host='docs.python.org', port=443): Max retries exceeded with url: /3/objects.inv (Caused by NewConnectionError('Failed to establish a new connection: [Errno 101] Network is unreachable'))
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,9 @@
|
||||
Build: 4102000 State: Finished Success: False Duration: 460s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (2):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64220/doc/source/_static/redoc.html: WARNING: copying overwrites an existing file [misc.copy_overwrite]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64220/doc/source/_collections/example.md:3: ERROR: Unknown target name: "foo".
|
||||
|
||||
Summary line: build finished with problems, 2 warnings.
|
||||
@@ -0,0 +1,9 @@
|
||||
Build: 4101200 State: Finished Success: False Duration: 455s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (2):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64210/doc/source/ray-contribute/index.md:8: WARNING: toctree contains reference to nonexisting document 'ray-contribute/fixme' [toc.not_readable]
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64210/doc/source/ray-contribute/getting-involved.md:40: WARNING: 'myst' cross-reference target not found: 'development.rst' [myst.xref_missing]
|
||||
|
||||
Summary line: build finished with problems, 2 warnings.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4161499 State: Finished Success: False Duration: 669s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (1):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/releases-2.56.0/python/ray/train/collective/__init__.py:docstring of ray.train.collective.broadcast_from_rank_zero:: WARNING: py:exc reference target not found: pickle.PicklingError [ref.exc]
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,6 @@
|
||||
Running Sphinx v8.2.3
|
||||
2026-06-16 10:00:01,234 WARNING worker.py:123 -- Ray runtime noise, ignore me
|
||||
building [html]: targets for 5 source files that are out of date
|
||||
{"levelname": "WARNING", "message": "json log line noise from a notebook"}
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64240/doc/source/data/api.md:55: WARNING: image file not readable: images/missing.png [image.not_readable]
|
||||
build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4014890 State: Finished Success: False Duration: 456s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (1):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64014/doc/source/404.md:9: WARNING: 'myst' cross-reference target not found: 'index.md' [myst.xref_missing]
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4099001 State: Finished Success: False Duration: 412s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (1):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64200/doc/source/index.md:12: WARNING: toctree contains reference to nonexisting document 'ray-contribute/old-name' [toc.not_readable]
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4103000 State: Finished Success: False Duration: 450s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (1):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64230/doc/source/api/foo.md:1: WARNING: some unrecognized problem here [app.unknown_thing]
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,8 @@
|
||||
Build: 4155333 State: Finished Success: False Duration: 628s
|
||||
Failing command: python -m sphinx -T -W --keep-going -j auto -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
|
||||
Exit code: 1
|
||||
|
||||
Sphinx warnings (1):
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/63617/doc/source/serve/llm/examples.md:17: WARNING: unknown document: '/_collections/serve/tutorials/deployment-serve-llm/npu-ascend/README' [ref.doc]
|
||||
|
||||
Summary line: build finished with problems, 1 warning.
|
||||
@@ -0,0 +1,2 @@
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64147/doc/source/tune/index.md:42: SEVERE: Title level inconsistent:
|
||||
/home/docs/checkouts/readthedocs.org/user_builds/anyscale-ray/checkouts/64147/doc/source/tune/api.md:7: WARNING: 'myst' cross-reference target not found: 'development.rst' [myst.xref_missing]
|
||||
@@ -0,0 +1,16 @@
|
||||
✗ HARD-BROKEN BUILD — fix this first; the rest of the log is unreliable.
|
||||
signal: extension-error
|
||||
Extension error:
|
||||
Could not import extension myst_parser (exception: No module named 'markdown_it')
|
||||
The warning list below is incomplete until this is fixed.
|
||||
|
||||
Build: state=Finished success=False exit=2
|
||||
|
||||
Partial findings (warning pass did not complete) (0):
|
||||
(none)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix the hard-broken build above, then rebuild and re-run.
|
||||
@@ -0,0 +1,20 @@
|
||||
✗ HARD-BROKEN BUILD — fix this first; the rest of the log is unreliable.
|
||||
signal: extension-error
|
||||
Extension error (sphinx.ext.autosummary)!
|
||||
|
||||
Traceback
|
||||
=========
|
||||
|
||||
hint: the named module is usually a DECOY. autosummary imports every documented object at builder-inited, so the real cause is typically an unrelated module whose import chain breaks under autodoc_mock_imports (a dep mocked for the doc build -- numpy/pandas/etc. -- but used at import time), aborting a shared import (in Ray, ray.air imports ray.data). Trace the import to the offending eager import and make it lazy; the named module is just the first to trip it.
|
||||
The warning list below is incomplete until this is fixed.
|
||||
|
||||
Build: state=Finished success=False exit=1
|
||||
|
||||
Partial findings (warning pass did not complete) (0):
|
||||
(none)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix the hard-broken build above, then rebuild and re-run.
|
||||
@@ -0,0 +1,28 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 15 warnings.
|
||||
|
||||
Root-cause groups (1) — a structural root masks a downstream flood; fix the root, rebuild, and the masked references clear together:
|
||||
ROOT [T2] autosummary-stub-not-found — ray.data.Dataset.* (6 stubs not generated, masking 8 downstream references)
|
||||
where: doc/source/apis/data/ray.data.Dataset.rst:20
|
||||
stubs: ray.data.Dataset.map_batches, ray.data.Dataset.map, ray.data.Dataset.filter, ray.data.Dataset.flat_map, ray.data.Dataset.groupby, ... and 1 more
|
||||
fix: Do not fix per entry when the failure is in bulk. Determine scope first: many missing stubs sharing a module prefix point at one root -- confirm the module imports under the build's mock set (autodoc_mock_imports) and that autosummary_generate is on; when an API-ref restructuring is in flight, the correct move may be to revert or repair the file moves rather than edit each entry. A lone missing stub means the member no longer exists -- update or drop the autosummary entry.
|
||||
safety: judgment (needs your call)
|
||||
masks 8 downstream py:* reference(s) — collapsed (full list in --json):
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.map_batches
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.map
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.filter
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.flat_map
|
||||
doc/source/apis/data/transform.rst:10 ray.data.Dataset.groupby
|
||||
... and 3 more (see --json)
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] py-xref-target-not-found python/ray/train/collective/__init__.py:docstring of ray.train.collective.broadcast_from_rank_zero:
|
||||
msg: py:exc reference target not found: pickle.PicklingError
|
||||
fix: Check for a co-occurring tier-2 autosummary-stub-not-found or tier-1 import abort for the same module first. If present, fix that root (up to and including reverting an in-progress API-ref restructuring) and rebuild -- the flood clears at once. Only when the build is otherwise healthy is the reference itself wrong: repoint to the correct dotted path, or fix the intersphinx target for a stdlib/third-party name.
|
||||
safety: judgment (needs your call)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix Tier 2 first (it masks others), then rebuild and re-run — do not assume one pass is complete.
|
||||
@@ -0,0 +1,20 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 4 warnings.
|
||||
|
||||
Root-cause groups (1) — a structural root masks a downstream flood; fix the root, rebuild, and the masked references clear together:
|
||||
ROOT [T2] autosummary-stub-not-found — ray.data.Dataset.* (2 stubs not generated, masking 2 downstream references)
|
||||
where: doc/source/apis/data/ray.data.Dataset.rst:20
|
||||
stubs: ray.data.Dataset.map_batches, ray.data.Dataset.map
|
||||
fix: Do not fix per entry when the failure is in bulk. Determine scope first: many missing stubs sharing a module prefix point at one root -- confirm the module imports under the build's mock set (autodoc_mock_imports) and that autosummary_generate is on; when an API-ref restructuring is in flight, the correct move may be to revert or repair the file moves rather than edit each entry. A lone missing stub means the member no longer exists -- update or drop the autosummary entry.
|
||||
safety: judgment (needs your call)
|
||||
masks 2 downstream py:* reference(s) — collapsed (full list in --json):
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.map_batches
|
||||
doc/source/apis/data/aggregate.rst:6 ray.data.Dataset.map
|
||||
|
||||
Findings (0):
|
||||
(none)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix Tier 2 first (it masks others), then rebuild and re-run — do not assume one pass is complete.
|
||||
@@ -0,0 +1,10 @@
|
||||
Build: state=Finished success=True exit=0 build succeeded.
|
||||
|
||||
Findings (0):
|
||||
(none)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
✓ No actionable warnings.
|
||||
@@ -0,0 +1,18 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 2 warnings.
|
||||
|
||||
Findings (2):
|
||||
Tier 3 — warnings:
|
||||
[T3] docutils-inline-markup doc/source/data/api/doc/ray.data.ReaderFormat.rst:74:<autosummary>:1
|
||||
msg: Inline emphasis start-string without end-string.
|
||||
fix: Balance or escape the marker in the source docstring, not in the generated stub: wrap literals in double backticks (``*args``, ``**kwargs``), or escape a standalone asterisk (\*).
|
||||
safety: mechanical
|
||||
[T3] docutils-inline-markup doc/source/data/api/doc/ray.data.ReaderFormat.rst:74:<autosummary>:1
|
||||
msg: Inline strong start-string without end-string.
|
||||
fix: Balance or escape the marker in the source docstring, not in the generated stub: wrap literals in double backticks (``*args``, ``**kwargs``), or escape a standalone asterisk (\*).
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] intersphinx-inventory-unreachable (no location)
|
||||
msg: failed to reach any of the inventories with the following issues:
|
||||
fix: Re-run the build first -- this is usually transient. If it persists, the inventory URL in intersphinx_mapping (conf.py) is unreachable or wrong: confirm the URL. This is not a per-page content fix.
|
||||
safety: judgment (needs your call)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,12 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 2 warnings.
|
||||
|
||||
Findings (0):
|
||||
(none)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (2) — known-benign, not actionable:
|
||||
collections-fetched-footnotes (DOC-1258): 1
|
||||
misc-copy-overwrite (DOC-1257): 1
|
||||
|
||||
✓ No actionable warnings.
|
||||
@@ -0,0 +1,19 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 2 warnings.
|
||||
|
||||
Findings (2):
|
||||
Tier 2 — structural/parse (fix first; these mask warnings beneath them):
|
||||
[T2] toctree-nonexisting doc/source/ray-contribute/index.md:8
|
||||
msg: toctree contains reference to nonexisting document 'ray-contribute/fixme'
|
||||
fix: Fix the toctree entry (entries are extensionless) to the current document name, or remove it.
|
||||
safety: mechanical
|
||||
Tier 3 — warnings:
|
||||
[T3] myst-xref-missing-cross-extension doc/source/ray-contribute/getting-involved.md:40
|
||||
msg: 'myst' cross-reference target not found: 'development.rst'
|
||||
fix: Rewrite as a {doc} role so it resolves to the document and is never ambiguous; drop the extension on the target. Suggested: {doc}`text <development>`
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix Tier 2 first (it masks others), then rebuild and re-run — do not assume one pass is complete.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] py-xref-target-not-found python/ray/train/collective/__init__.py:docstring of ray.train.collective.broadcast_from_rank_zero:
|
||||
msg: py:exc reference target not found: pickle.PicklingError
|
||||
fix: Check for a co-occurring tier-2 autosummary-stub-not-found or tier-1 import abort for the same module first. If present, fix that root (up to and including reverting an in-progress API-ref restructuring) and rebuild -- the flood clears at once. Only when the build is otherwise healthy is the reference itself wrong: repoint to the correct dotted path, or fix the intersphinx target for a stdlib/third-party name.
|
||||
safety: judgment (needs your call)
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] image-not-readable doc/source/data/api.md:55
|
||||
msg: image file not readable: images/missing.png
|
||||
fix: Fix the image path (relative to the referencing page, or an absolute /_static-style path).
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] myst-xref-missing-cross-extension doc/source/404.md:9
|
||||
msg: 'myst' cross-reference target not found: 'index.md'
|
||||
fix: Rewrite as a {doc} role so it resolves to the document and is never ambiguous; drop the extension on the target. Suggested: {doc}`text <index>`
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 2 — structural/parse (fix first; these mask warnings beneath them):
|
||||
[T2] toctree-nonexisting doc/source/index.md:12
|
||||
msg: toctree contains reference to nonexisting document 'ray-contribute/old-name'
|
||||
fix: Fix the toctree entry (entries are extensionless) to the current document name, or remove it.
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: fix Tier 2 first (it masks others), then rebuild and re-run — do not assume one pass is complete.
|
||||
@@ -0,0 +1,11 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (0):
|
||||
(none)
|
||||
|
||||
Unclassified (1) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
doc/source/api/foo.md:1: WARNING: some unrecognized problem here [app.unknown_thing]
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
No rule matched the warning(s) above — resolve with the user and extend rules.yaml.
|
||||
@@ -0,0 +1,14 @@
|
||||
Build: state=Finished success=False exit=1 build finished with problems, 1 warning.
|
||||
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] unknown-document doc/source/serve/llm/examples.md:17
|
||||
msg: unknown document: '/_collections/serve/tutorials/deployment-serve-llm/npu-ascend/README'
|
||||
fix: Repoint to the correct document: a path relative to the referencing page, or an absolute path from doc/source with a leading slash. Remove the reference if the target is gone. Grep the bare stem across doc/ to find every caller.
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (0) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,13 @@
|
||||
Findings (1):
|
||||
Tier 3 — warnings:
|
||||
[T3] myst-xref-missing-cross-extension doc/source/tune/api.md:7
|
||||
msg: 'myst' cross-reference target not found: 'development.rst'
|
||||
fix: Rewrite as a {doc} role so it resolves to the document and is never ambiguous; drop the extension on the target. Suggested: {doc}`text <development>`
|
||||
safety: mechanical
|
||||
|
||||
Unclassified (1) — no rule matched; resolve with the user, then file a skill-improvement ticket to add a rule:
|
||||
doc/source/tune/index.md:42: SEVERE: Title level inconsistent:
|
||||
|
||||
Suppressed (0) — known-benign, not actionable:
|
||||
|
||||
Next: apply the fixes above, then rebuild and re-run to confirm clean.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
description: Ray documentation style guide for technical content
|
||||
alwaysApply: true
|
||||
path: doc/**/*
|
||||
---
|
||||
|
||||
# Ray documentation style
|
||||
|
||||
The full style and grammar standard for Ray docs is `doc/source/ray-contribute/writing-style.md`. Read it before writing or editing any file under `doc/`. It's the source of truth and supersedes prior style guidance. It builds on the Google developer documentation style guide as a fallback.
|
||||
|
||||
The rules below are the highest-frequency corrections. The full guide covers word choice, sentence structure, links, admonitions, tables, and human-writing patterns in depth.
|
||||
|
||||
## Voice
|
||||
|
||||
- Active voice with a named actor: "The scheduler retries failed tasks," not "Failed tasks are retried."
|
||||
- Present tense, second person ("you"), imperative for instructions.
|
||||
- No first-person plural. Rewrite to remove "we" and "our." Use "The Ray project" only when a subject is unavoidable.
|
||||
|
||||
## Word choice
|
||||
|
||||
- Use contractions (don't, isn't, can't), except in warnings and error messages.
|
||||
- "such as," not "like," for examples.
|
||||
- Cut filler: simply, just, basically, actually, really, very.
|
||||
- Plain words: "use" (not utilize or leverage), "to" (not in order to), "through" (not via), "before" (not prior to).
|
||||
- "ID," not "id," in prose.
|
||||
- No time-relative words: currently, now, recently, new.
|
||||
- Angle-bracket placeholders (`<head-node-ip>`), not UPPERCASE.
|
||||
|
||||
## Headings
|
||||
|
||||
- Sentence case: "Configure the runtime environment," not "Configure the Runtime Environment."
|
||||
- Imperative for task headings, questions for conceptual headings.
|
||||
|
||||
## Structure and formatting
|
||||
|
||||
- Short sentences. No dashes, parentheticals, or semicolons in prose. A colon only introduces a list or code block.
|
||||
- List items that are complete sentences or contain verbs get periods. Lead-ins are complete sentences ending in a colon.
|
||||
- MyST admonitions (`:::{note}`, `:::{warning}`), not bold inline labels.
|
||||
- Tag code blocks with a language. No `$` shell prompt. Code comments start capitalized and use the imperative.
|
||||
- Reserve bold for UI elements, admonition labels, and definition-list terms. Italicize a term on first use.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- `{doc}` for whole pages (required when the target's source is `.rst`), `{ref}` for labeled targets, `[text](page.md)` between `.md` pages.
|
||||
|
||||
## Never fabricate technical details
|
||||
|
||||
Every technical claim needs a verifiable source: the existing docs, the source code, or a config file. When you can't verify a detail, leave it out.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Generated documentation files
|
||||
_build
|
||||
source/_static/thumbs
|
||||
.ipynb_checkpoints/
|
||||
|
||||
# Ignore generated gallery txt files
|
||||
eco-gallery.txt
|
||||
|
||||
# Ignore CSS linter
|
||||
node_modules/
|
||||
package-lock.json
|
||||
|
||||
# Ignore generated API files
|
||||
source/data/api/doc/
|
||||
source/cluster/running-applications/job-submission/doc/
|
||||
source/serve/api/doc/
|
||||
source/workflows/api/doc/
|
||||
source/data/api/ray.data.*.rst
|
||||
|
||||
# Ignore generated examples files
|
||||
source/data/examples.rst
|
||||
source/train/examples.rst
|
||||
source/serve/examples.rst
|
||||
|
||||
# Ignore generated versions
|
||||
source/_static/versions.json
|
||||
|
||||
# Ignore rllib generated files
|
||||
source/rllib/package_ref/
|
||||
|
||||
# Ignore generated collections
|
||||
source/_collections/
|
||||
+785
@@ -0,0 +1,785 @@
|
||||
load("@py_deps_py310//:requirements.bzl", ci_require = "requirement")
|
||||
load("@rules_python//python:defs.bzl", "py_test")
|
||||
load("//bazel:python.bzl", "doctest", "doctest_each", "py_test_run_all_notebooks", "py_test_run_all_subdirectory")
|
||||
|
||||
exports_files(["test_myst_doc.py"])
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Tests from the doc directory.
|
||||
# Please keep these sorted alphabetically, but start with the
|
||||
# root directory.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test(
|
||||
name = "highly_parallel",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/highly_parallel.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"highly_parallel",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "plot_hyperparameter",
|
||||
size = "small",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/plot_hyperparameter.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "batch_prediction",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/batch_prediction.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "plot_parameter_server",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/plot_parameter_server.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "plot_pong_example",
|
||||
size = "large",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/plot_pong_example.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "gentle_walkthrough",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/gentle_walkthrough.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "map_reduce",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/map_reduce.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "web_crawler",
|
||||
size = "medium",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/ray-core/examples/web_crawler.ipynb",
|
||||
],
|
||||
data = ["//doc/source/ray-core/examples:core_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/ray-observability/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = ["source/ray-observability/doc_code/*.py"],
|
||||
exclude = ["source/ray-observability/doc_code/ray-distributed-debugger.py"],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/ray-core/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test(
|
||||
name = "doc_code_runtime_env_example",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/runtime_env_example.py"],
|
||||
main = "source/ray-core/doc_code/runtime_env_example.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"post_wheel_build",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_ray_oom_prevention",
|
||||
size = "medium",
|
||||
srcs = ["source/ray-core/doc_code/ray_oom_prevention.py"],
|
||||
main = "source/ray-core/doc_code/ray_oom_prevention.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"mem_pressure",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_cgraph_profiling",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/cgraph_profiling.py"],
|
||||
main = "source/ray-core/doc_code/cgraph_profiling.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"multi_gpu",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_cgraph_nccl",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/cgraph_nccl.py"],
|
||||
main = "source/ray-core/doc_code/cgraph_nccl.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"multi_gpu",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_cgraph_overlap",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/cgraph_overlap.py"],
|
||||
main = "source/ray-core/doc_code/cgraph_overlap.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"multi_gpu",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_direct_transport_gloo",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/direct_transport_gloo.py"],
|
||||
main = "source/ray-core/doc_code/direct_transport_gloo.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_direct_transport_nccl",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/direct_transport_nccl.py"],
|
||||
main = "source/ray-core/doc_code/direct_transport_nccl.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"multi_gpu",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test(
|
||||
name = "doc_code_direct_transport_nixl",
|
||||
size = "small",
|
||||
srcs = ["source/ray-core/doc_code/direct_transport_nixl.py"],
|
||||
main = "source/ray-core/doc_code/direct_transport_nixl.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"multi_gpu",
|
||||
"custom_setup",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = ["source/ray-core/doc_code/*.py"],
|
||||
exclude = [
|
||||
"source/ray-core/doc_code/runtime_env_example.py",
|
||||
"source/ray-core/doc_code/cross_language.py",
|
||||
"source/ray-core/doc_code/ray_oom_prevention.py",
|
||||
"source/ray-core/doc_code/cgraph_profiling.py",
|
||||
"source/ray-core/doc_code/cgraph_nccl.py",
|
||||
"source/ray-core/doc_code/cgraph_overlap.py",
|
||||
# not testing this as it purposefully segfaults
|
||||
"source/ray-core/doc_code/cgraph_troubleshooting.py",
|
||||
"source/ray-core/doc_code/direct_transport_nccl.py",
|
||||
"source/ray-core/doc_code/direct_transport_nixl.py",
|
||||
],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:core",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/serve/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = ["source/serve/doc_code/**/*.py"],
|
||||
exclude = [
|
||||
"source/serve/doc_code/aws_neuron_core_inference_serve.py",
|
||||
"source/serve/doc_code/aws_neuron_core_inference_serve_stable_diffusion.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_serve.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_serve_deepspeed.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_client.py",
|
||||
"source/serve/doc_code/distilbert.py",
|
||||
"source/serve/doc_code/stable_diffusion.py",
|
||||
"source/serve/doc_code/object_detection.py",
|
||||
"source/serve/doc_code/vllm_example.py",
|
||||
"source/serve/doc_code/cross_node_parallelism_example.py",
|
||||
"source/serve/doc_code/llm/llm_yaml_config_example.py",
|
||||
"source/serve/doc_code/llm/qwen_example.py",
|
||||
"source/serve/doc_code/capacity_queue_request_router_app.py",
|
||||
],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:serve",
|
||||
],
|
||||
)
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = [
|
||||
"source/serve/doc_code/distilbert.py",
|
||||
"source/serve/doc_code/stable_diffusion.py",
|
||||
"source/serve/doc_code/object_detection.py",
|
||||
],
|
||||
env = {"RAY_SERVE_PROXY_READY_CHECK_TIMEOUT_S": "60"},
|
||||
exclude = [
|
||||
"source/serve/doc_code/aws_neuron_core_inference_serve.py",
|
||||
"source/serve/doc_code/aws_neuron_core_inference_serve_stable_diffusion.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_serve.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_serve_deepspeed.py",
|
||||
"source/serve/doc_code/intel_gaudi_inference_client.py",
|
||||
],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"team:serve",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/llm/doc_code/serve code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
filegroup(
|
||||
name = "serve_llm_examples",
|
||||
srcs = glob(["source/llm/doc_code/serve/**/*.py"]),
|
||||
visibility = ["//doc:__subpackages__"],
|
||||
)
|
||||
|
||||
# GPU Tests (standard GPU tests)
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/llm/doc_code/serve/**/*.py"],
|
||||
exclude = [
|
||||
"source/llm/doc_code/serve/direct_streaming/**/*.py",
|
||||
"source/llm/doc_code/serve/multi_gpu/**/*.py",
|
||||
"source/llm/doc_code/serve/sglang/**/*.py",
|
||||
# custom_vllm runs in its own direct-streaming target below.
|
||||
"source/llm/doc_code/serve/custom_vllm/**/*.py",
|
||||
],
|
||||
extra_srcs = [],
|
||||
data = ["source/llm/doc_code/serve/qwen/llm_config_example.yaml"],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"team:llm",
|
||||
],
|
||||
)
|
||||
|
||||
# Custom vLLM model (Qwen3 reward model) GPU tests
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/llm/doc_code/serve/custom_vllm/**/*.py"],
|
||||
exclude = [
|
||||
"source/llm/doc_code/serve/custom_vllm/qwen3_reward_plugin/**/*.py",
|
||||
# setup.py packages the plugin; it is not a runnable example.
|
||||
"source/llm/doc_code/serve/custom_vllm/setup.py",
|
||||
],
|
||||
extra_srcs = [],
|
||||
data = ["source/llm/doc_code/serve/custom_vllm/custom_vllm_config.yaml"],
|
||||
env = {
|
||||
"RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING": "1",
|
||||
"RAY_SERVE_ENABLE_HA_PROXY": "1",
|
||||
},
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"custom_vllm_plugin",
|
||||
"team:llm",
|
||||
],
|
||||
)
|
||||
|
||||
# Direct streaming GPU Tests. Isolated into their own target so they run with
|
||||
# the direct streaming flags set, which exercises the real direct streaming
|
||||
# path. The flags can't go on the shared target above because they would force
|
||||
# every other serve example into direct streaming mode, and build_openai_app
|
||||
# rejects multi-config apps (such as the qwen example) under direct streaming.
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/llm/doc_code/serve/direct_streaming/**/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
data = ["source/llm/doc_code/serve/direct_streaming/direct_streaming_config.yaml"],
|
||||
env = {
|
||||
"RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING": "1",
|
||||
# Direct streaming wires LLMRouter as the ingress_request_router,
|
||||
# which build_app.py rejects unless HAProxy is enabled.
|
||||
"RAY_SERVE_ENABLE_HA_PROXY": "1",
|
||||
},
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"team:llm",
|
||||
],
|
||||
)
|
||||
|
||||
# Multi-GPU Tests (4+ GPUs)
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/llm/doc_code/serve/multi_gpu/**/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"multi_gpu_4",
|
||||
"team:llm",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/data/doc_code/working-with-llms code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
filegroup(
|
||||
name = "data_llm_examples",
|
||||
srcs = glob(["source/data/doc_code/working-with-llms/**/*.py"]),
|
||||
visibility = ["//doc:__subpackages__"],
|
||||
)
|
||||
|
||||
# GPU Tests
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/data/doc_code/working-with-llms/**/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"team:llm"
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/tune/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = ["source/tune/doc_code/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
env = {"RAY_TRAIN_V2_ENABLED": "1"},
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/rllib/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "medium",
|
||||
include = ["source/rllib/doc_code/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:rllib",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/train/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
env = {"RAY_TRAIN_V2_ENABLED": "1"},
|
||||
include = ["source/train/doc_code/*.py"],
|
||||
exclude = [
|
||||
"source/train/doc_code/hvd_trainer.py", # CI do not have Horovod
|
||||
"source/train/doc_code/asynchronous_validation.py", # pseudocode snippets, not runnable
|
||||
],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/data/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/data/doc_code/*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"exclusive",
|
||||
"team:data",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/source/ray-more-libs/doc_code code included in rst/md files.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
py_test_run_all_subdirectory(
|
||||
size = "large",
|
||||
include = ["source/ray-more-libs/doc_code/dask_on_ray_*.py"],
|
||||
exclude = [],
|
||||
extra_srcs = [],
|
||||
tags = [
|
||||
"dask",
|
||||
"custom_setup",
|
||||
"exclusive",
|
||||
"team:data",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------
|
||||
# Run GPU tests
|
||||
# --------------
|
||||
|
||||
py_test(
|
||||
name = "pytorch_resnet_finetune",
|
||||
size = "large",
|
||||
srcs = ["test_myst_doc.py"],
|
||||
args = [
|
||||
"--path",
|
||||
"doc/source/train/examples/pytorch/pytorch_resnet_finetune.ipynb",
|
||||
],
|
||||
data = ["//doc/source/train/examples/pytorch:train_pytorch_examples"],
|
||||
main = "test_myst_doc.py",
|
||||
tags = [
|
||||
"exclusive",
|
||||
"gpu",
|
||||
"ray_air",
|
||||
"team:ml",
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Test all doc/external code
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
# py_test_run_all_subdirectory(
|
||||
# size = "enormous",
|
||||
# include = ["external/*.py"],
|
||||
# exclude = ["external/test_hashes.py"],
|
||||
# extra_srcs = [],
|
||||
# tags = [
|
||||
# "exclusive",
|
||||
# "external",
|
||||
# "team:ml",
|
||||
# ],
|
||||
# )
|
||||
|
||||
py_test(
|
||||
name = "test_external_hashes",
|
||||
srcs = ["external/test_hashes.py"],
|
||||
data = glob(
|
||||
["external/*.py"],
|
||||
exclude = ["external/test_hashes.py"],
|
||||
),
|
||||
exec_compatible_with = ["//bazel:py3"],
|
||||
main = "external/test_hashes.py",
|
||||
tags = ["team:ml"],
|
||||
deps = [
|
||||
ci_require("pytest"),
|
||||
ci_require("bazel-runfiles"),
|
||||
],
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Tests code snippets in user guides.
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
doctest(
|
||||
size = "large",
|
||||
files = glob(
|
||||
include = [
|
||||
"source/**/*.md",
|
||||
"source/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
"source/ray-contribute/getting-involved.md",
|
||||
"source/ray-contribute/testing-tips.md",
|
||||
"source/ray-contribute/writing-code-snippets.md",
|
||||
"source/ray-observability/user-guides/ray-tracing.rst",
|
||||
"source/ray-observability/user-guides/cli-sdk.rst",
|
||||
"source/templates/04_finetuning_llms_with_deepspeed/README.md",
|
||||
"source/ray-core/**/*.md",
|
||||
"source/ray-core/**/*.rst",
|
||||
"source/data/**/*.md",
|
||||
"source/data/**/*.rst",
|
||||
"source/rllib/**/*.md",
|
||||
"source/rllib/**/*.rst",
|
||||
"source/serve/**/*.md",
|
||||
"source/serve/**/*.rst",
|
||||
"source/train/**/*.md",
|
||||
"source/train/**/*.rst",
|
||||
"source/tune/**/*.md",
|
||||
"source/tune/**/*.rst",
|
||||
],
|
||||
),
|
||||
tags = ["team:none"],
|
||||
# NOTE(edoakes): the global glossary and some tutorials use Ray Data,
|
||||
# so we use its pytest plugin file (which is a superset of the default).
|
||||
pytest_plugin_file = "//python/ray/data:tests/doctest_pytest_plugin.py",
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[core]",
|
||||
size = "large",
|
||||
files = glob(
|
||||
include = [
|
||||
"source/ray-core/**/*.md",
|
||||
"source/ray-core/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
"source/ray-core/handling-dependencies.rst",
|
||||
# The `doc_code/` snippet for `nested-tasks.rst` is tested.
|
||||
"source/ray-core/tasks/nested-tasks.rst",
|
||||
],
|
||||
),
|
||||
tags = ["team:core"],
|
||||
)
|
||||
|
||||
doctest_each(
|
||||
files = glob(
|
||||
include = [
|
||||
"source/data/**/*.md",
|
||||
"source/data/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
# These tests run on GPU (see below).
|
||||
"source/data/batch_inference.rst",
|
||||
"source/data/transforming-data.rst",
|
||||
# These don't contain code snippets.
|
||||
"source/data/api/**/*.rst",
|
||||
],
|
||||
),
|
||||
pytest_plugin_file = "//python/ray/data:tests/doctest_pytest_plugin.py",
|
||||
tags = ["team:data"],
|
||||
# Adding to use keras 2 & tensorflow >= 2.16 (https://keras.io/getting_started)
|
||||
env = {"TF_USE_LEGACY_KERAS": "1"},
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[data-gpu]",
|
||||
files = [
|
||||
"source/data/batch_inference.rst",
|
||||
"source/data/transforming-data.rst",
|
||||
],
|
||||
pytest_plugin_file = "//python/ray/data:tests/doctest_pytest_plugin.py",
|
||||
tags = ["team:data"],
|
||||
gpu = True,
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[rllib]",
|
||||
size = "large",
|
||||
data = ["//rllib:cartpole-v1_large"],
|
||||
files = glob(
|
||||
include = [
|
||||
"source/rllib/**/*.md",
|
||||
"source/rllib/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
"source/rllib/getting-started.rst",
|
||||
],
|
||||
),
|
||||
tags = ["team:rllib"],
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[rllib2]",
|
||||
size = "enormous",
|
||||
files = glob(
|
||||
include = [
|
||||
"source/rllib/getting-started.rst",
|
||||
],
|
||||
),
|
||||
# TODO(elliot-barn): Re-enable once getting-started.rst doctest hang is fixed.
|
||||
tags = ["manual", "team:rllib"],
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[serve]",
|
||||
files = glob(
|
||||
include = [
|
||||
"source/serve/**/*.md",
|
||||
"source/serve/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
"source/serve/advanced-guides/inplace-updates.md",
|
||||
"source/serve/deploy-many-models/multi-app.md",
|
||||
"source/serve/production-guide/deploy-vm.md",
|
||||
"source/serve/production-guide/fault-tolerance.md",
|
||||
],
|
||||
),
|
||||
tags = ["team:serve"],
|
||||
)
|
||||
|
||||
|
||||
doctest(
|
||||
name = "doctest[train]",
|
||||
# TODO: [V2] Migrate
|
||||
env = {
|
||||
"RAY_TRAIN_V2_ENABLED": "0",
|
||||
"TF_USE_LEGACY_KERAS": "1",
|
||||
},
|
||||
files = glob(
|
||||
include = [
|
||||
"source/train/**/*.md",
|
||||
"source/train/**/*.rst",
|
||||
],
|
||||
exclude = [
|
||||
# CI does not have Horovod installed.
|
||||
"source/train/horovod.rst",
|
||||
# These tests run on GPU (see below).
|
||||
"source/train/user-guides/data-loading-preprocessing.rst",
|
||||
"source/train/user-guides/using-accelerators.rst",
|
||||
],
|
||||
),
|
||||
tags = ["team:ml"],
|
||||
)
|
||||
|
||||
doctest(
|
||||
name = "doctest[train-gpu]",
|
||||
files = [
|
||||
"source/train/user-guides/data-loading-preprocessing.rst",
|
||||
"source/train/user-guides/using-accelerators.rst",
|
||||
],
|
||||
env = {"RAY_TRAIN_V2_ENABLED": "0"},
|
||||
tags = ["team:ml"],
|
||||
gpu = True,
|
||||
)
|
||||
|
||||
|
||||
doctest(
|
||||
name = "doctest[tune]",
|
||||
files = [
|
||||
"source/tune/**/*.md",
|
||||
"source/tune/**/*.rst",
|
||||
],
|
||||
tags = ["team:ml"],
|
||||
env = {"RAY_TRAIN_V2_ENABLED": "1"},
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------
|
||||
# Discover the Anyscale Jobs compute configs .yaml for release tests in CI
|
||||
# --------------------------------------------------------------------
|
||||
|
||||
# Comprehensive filegroup that aggregates all CI compute configs from across the docs.
|
||||
# This includes a general glob pattern + references to filegroups in existing child BUILD.bazel
|
||||
# Following the standard structure: doc/source/<path>/my_example/ci/{aws,gce}.yaml
|
||||
filegroup(
|
||||
name = "all_examples_ci_configs",
|
||||
srcs = glob([
|
||||
"source/ray-overview/examples/**/ci/aws.yaml",
|
||||
"source/ray-overview/examples/**/ci/gce.yaml",
|
||||
"source/serve/tutorials/**/ci/aws.yaml",
|
||||
"source/serve/tutorials/**/ci/gce.yaml",
|
||||
]) + [
|
||||
"//doc/source/data/examples:data_examples_ci_configs",
|
||||
"//doc/source/tune/examples:tune_examples_ci_configs",
|
||||
"//doc/source/ray-core/examples:core_examples_ci_configs",
|
||||
"//doc/source/train/examples:train_examples_ci_configs",
|
||||
],
|
||||
visibility = ["//release:__pkg__"],
|
||||
)
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
# Makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line.
|
||||
# Allow linkcheck to run without treating warnings as errors; -W will
|
||||
# fast-fail if enabled; it's better to gather the whole list of bad links at once.
|
||||
SPHINXOPTS = -a -E -W -j auto
|
||||
LOCALSPHINXOPTS = -W -j auto
|
||||
LINKCHECKOPTS = -a -E -j auto
|
||||
# RtD-faithful build: our full-build options plus the two flags Read the Docs
|
||||
# adds to its synthesized html command (-T full tracebacks, -D language=en).
|
||||
RTDSPHINXOPTS = $(SPHINXOPTS) -T -D language=en
|
||||
# Extra args forwarded to the preflight doctor (e.g. RTD_DOCTOR_ARGS=--warn-only
|
||||
# to build despite environment drift).
|
||||
RTD_DOCTOR_ARGS =
|
||||
SPHINXBUILD = sphinx-build
|
||||
SPHINXAUTOBUILD = sphinx-autobuild --port 0 --open-browser --ignore "*.log" --ignore "*data/examples*" --ignore "*train/examples*" --ignore "*serve/examples*"
|
||||
PAPER =
|
||||
BUILDDIR = _build
|
||||
# Output directory for the html builder. Overridable so Read the Docs can point
|
||||
# it at $READTHEDOCS_OUTPUT/html; defaults to the local _build/html.
|
||||
HTMLDIR ?= $(BUILDDIR)/html
|
||||
|
||||
# User-friendly check for sphinx-build
|
||||
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
|
||||
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
|
||||
endif
|
||||
|
||||
# Internal variables.
|
||||
PAPEROPT_a4 = -D latex_paper_size=a4
|
||||
PAPEROPT_letter = -D latex_paper_size=letter
|
||||
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
|
||||
ALLLOCALSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(LOCALSPHINXOPTS) source
|
||||
ALLRTDSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(RTDSPHINXOPTS) source
|
||||
ALLLINKCHECKOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(LINKCHECKOPTS) source
|
||||
# the i18n builder cannot share the environment and doctrees with the others
|
||||
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source
|
||||
|
||||
.PHONY: help clean html rtd rtd-fallback rtd-build rtd-doctor dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext show
|
||||
|
||||
help:
|
||||
@echo "Please use \`make <target>' where <target> is one of"
|
||||
@echo " html to make standalone HTML files"
|
||||
@echo " rtd-build to run a Read-the-Docs-faithful full build (preflight doctor + html)"
|
||||
@echo " rtd-doctor to check this environment matches the Read the Docs build"
|
||||
@echo " dirhtml to make HTML files named index.html in directories"
|
||||
@echo " singlehtml to make a single large HTML file"
|
||||
@echo " pickle to make pickle files"
|
||||
@echo " json to make JSON files"
|
||||
@echo " htmlhelp to make HTML files and a HTML help project"
|
||||
@echo " qthelp to make HTML files and a qthelp project"
|
||||
@echo " applehelp to make an Apple Help Book"
|
||||
@echo " devhelp to make HTML files and a Devhelp project"
|
||||
@echo " epub to make an epub"
|
||||
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
|
||||
@echo " latexpdf to make LaTeX files and run them through pdflatex"
|
||||
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
|
||||
@echo " text to make text files"
|
||||
@echo " man to make manual pages"
|
||||
@echo " texinfo to make Texinfo files"
|
||||
@echo " info to make Texinfo files and run them through makeinfo"
|
||||
@echo " gettext to make PO message catalogs"
|
||||
@echo " changes to make an overview of all changed/added/deprecated items"
|
||||
@echo " xml to make Docutils-native XML files"
|
||||
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
|
||||
@echo " linkcheck to check all external links for integrity"
|
||||
@echo " linkcheck_all to check all external links for integrity"
|
||||
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
|
||||
@echo " coverage to run coverage check of the documentation (if enabled)"
|
||||
|
||||
clean:
|
||||
rm -rf $(BUILDDIR)/*
|
||||
rm -rf ./source/__pycache__/
|
||||
rm -rf ./source/_ext/__pycache__/
|
||||
rm -rf ./source/*/api/doc/*
|
||||
rm -rf ./source/ray-core/compiled-graph/doc/*
|
||||
rm -rf ./source/ray-references/api/*/doc/*
|
||||
rm -rf ./source/cluster/running-applications/doc/*
|
||||
rm -rf ./source/cluster/running-applications/job-submission/doc/*
|
||||
rm -rf ./source/ray-observability/api/state/doc*
|
||||
rm -rf ./source/rllib/package_ref/doc*
|
||||
rm -rf ./source/ray-more-libs/doc/*
|
||||
rm -rf ./source/ray-observability/reference/doc/*
|
||||
rm -f ./source/data/examples.rst
|
||||
rm -f ./source/serve/examples.rst
|
||||
rm -f ./source/train/examples.rst
|
||||
|
||||
html:
|
||||
@# Remove cached serve API stubs so autosummary regenerates them with correct templates
|
||||
find source/serve/api/doc/ -name "ray.serve.*.rst" -delete 2>/dev/null || true
|
||||
$(SPHINXBUILD) -W --keep-going -b html $(ALLSPHINXOPTS) $(HTMLDIR)
|
||||
@echo
|
||||
@echo "Build finished. The HTML pages are in $(HTMLDIR)."
|
||||
@echo "View the documentation by opening a browser and going to $(HTMLDIR)/index.html."
|
||||
|
||||
develop: html
|
||||
|
||||
RAY_DIR := $(shell pwd | rev | cut -d'/' -f2- | rev)
|
||||
|
||||
local:
|
||||
python load_doc_cache.py --ray-dir=$(RAY_DIR)
|
||||
@# Remove cached serve API stubs so autosummary regenerates them with correct templates
|
||||
@# (pydantic v2 migration changed templates; cached stubs use old template with nested
|
||||
@# autosummary :toctree: that generates unwanted per-member stub files)
|
||||
find source/serve/api/doc/ -name "ray.serve.*.rst" -delete 2>/dev/null || true
|
||||
python update_cache_env.py --ray-dir=$(RAY_DIR)
|
||||
$(SPHINXAUTOBUILD) -W --keep-going -b html $(ALLLOCALSPHINXOPTS) $(BUILDDIR)/html
|
||||
|
||||
rtd-doctor:
|
||||
python rtd_doctor.py $(RTD_DOCTOR_ARGS)
|
||||
|
||||
# RtD-faithful full build for local verification: reproduces the clean full
|
||||
# build Read the Docs runs from a fresh checkout, so a clean build here means a
|
||||
# clean build on RtD. Distinct from the `rtd` target below, which RtD itself
|
||||
# drives as an incremental PR-preview build. The preflight doctor checks Python
|
||||
# 3.11 and the docs lock, and blocks on drift unless you pass
|
||||
# RTD_DOCTOR_ARGS=--warn-only.
|
||||
rtd-build: rtd-doctor
|
||||
@# Remove cached serve API stubs so autosummary regenerates them with correct templates
|
||||
find source/serve/api/doc/ -name "ray.serve.*.rst" -delete 2>/dev/null || true
|
||||
$(SPHINXBUILD) --keep-going -b html $(ALLRTDSPHINXOPTS) $(BUILDDIR)/html
|
||||
@echo
|
||||
@echo "RtD-faithful build finished. The HTML pages are in $(BUILDDIR)/html."
|
||||
|
||||
# Incremental one-shot build for Read the Docs PR previews: restore the latest
|
||||
# master build cache, mark only the PR-changed files dirty (load_doc_cache.py +
|
||||
# update_cache_env.py), then build once WITHOUT -a/-E so Sphinx reuses the cached
|
||||
# environment and rebuilds only those files. Mirrors `local`, but builds once
|
||||
# instead of running sphinx-autobuild.
|
||||
#
|
||||
# No --keep-going: if the restored cache is incompatible (e.g. it predates an
|
||||
# autosummary template change and carries stale generated stubs), fail on the
|
||||
# first warning so .readthedocs.yaml falls back to rtd-fallback (a clean build)
|
||||
# quickly instead of grinding through the whole build first.
|
||||
rtd:
|
||||
python load_doc_cache.py --ray-dir=$(RAY_DIR)
|
||||
@# Remove cached serve API stubs so autosummary regenerates them with correct templates
|
||||
find source/serve/api/doc/ -name "ray.serve.*.rst" -delete 2>/dev/null || true
|
||||
python update_cache_env.py --ray-dir=$(RAY_DIR)
|
||||
@# Build into $(BUILDDIR)/html, where load_doc_cache.py extracted the full prior
|
||||
@# site and update_cache_env.py refreshed the output timestamps, so Sphinx rewrites
|
||||
@# only the PR-changed pages. Then publish the whole tree to $(HTMLDIR): building
|
||||
@# straight into an empty $(HTMLDIR) (RTD's output dir) would emit only the changed
|
||||
@# pages and drop the rest of the site.
|
||||
$(SPHINXBUILD) -b html $(ALLLOCALSPHINXOPTS) $(BUILDDIR)/html
|
||||
@if [ "$(HTMLDIR)" != "$(BUILDDIR)/html" ]; then \
|
||||
echo "Publishing $(BUILDDIR)/html -> $(HTMLDIR)"; \
|
||||
mkdir -p $(HTMLDIR) && cp -a $(BUILDDIR)/html/. $(HTMLDIR)/; \
|
||||
fi
|
||||
|
||||
# Cold rebuild used by .readthedocs.yaml when the incremental `rtd` build fails.
|
||||
# git clean -dfX removes only git-ignored files -- the restored cache's _build tree
|
||||
# and all generated source/**/doc stubs -- so the tree matches a fresh checkout,
|
||||
# then a full clean build runs (identical to what postmerge CI builds and caches).
|
||||
rtd-fallback:
|
||||
git clean -dfX .
|
||||
$(MAKE) HTMLDIR="$(HTMLDIR)" html
|
||||
|
||||
dirhtml:
|
||||
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
|
||||
@echo
|
||||
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
|
||||
|
||||
singlehtml:
|
||||
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
|
||||
@echo
|
||||
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
|
||||
|
||||
pickle:
|
||||
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
|
||||
@echo
|
||||
@echo "Build finished; now you can process the pickle files."
|
||||
|
||||
json:
|
||||
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
|
||||
@echo
|
||||
@echo "Build finished; now you can process the JSON files."
|
||||
|
||||
htmlhelp:
|
||||
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
|
||||
@echo
|
||||
@echo "Build finished; now you can run HTML Help Workshop with the" \
|
||||
".hhp project file in $(BUILDDIR)/htmlhelp."
|
||||
|
||||
qthelp:
|
||||
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
|
||||
@echo
|
||||
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
|
||||
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
|
||||
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Ray.qhcp"
|
||||
@echo "To view the help file:"
|
||||
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Ray.qhc"
|
||||
|
||||
applehelp:
|
||||
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
|
||||
@echo
|
||||
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
|
||||
@echo "N.B. You won't be able to view it unless you put it in" \
|
||||
"~/Library/Documentation/Help or install it in your application" \
|
||||
"bundle."
|
||||
|
||||
devhelp:
|
||||
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
|
||||
@echo
|
||||
@echo "Build finished."
|
||||
@echo "To view the help file:"
|
||||
@echo "# mkdir -p $$HOME/.local/share/devhelp/Ray"
|
||||
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Ray"
|
||||
@echo "# devhelp"
|
||||
|
||||
epub:
|
||||
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
|
||||
@echo
|
||||
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
|
||||
|
||||
latex:
|
||||
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
||||
@echo
|
||||
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
|
||||
@echo "Run \`make' in that directory to run these through (pdf)latex" \
|
||||
"(use \`make latexpdf' here to do that automatically)."
|
||||
|
||||
latexpdf:
|
||||
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
||||
@echo "Running LaTeX files through pdflatex..."
|
||||
$(MAKE) -C $(BUILDDIR)/latex all-pdf
|
||||
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
|
||||
|
||||
latexpdfja:
|
||||
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
||||
@echo "Running LaTeX files through platex and dvipdfmx..."
|
||||
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
|
||||
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
|
||||
|
||||
text:
|
||||
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
|
||||
@echo
|
||||
@echo "Build finished. The text files are in $(BUILDDIR)/text."
|
||||
|
||||
man:
|
||||
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
|
||||
@echo
|
||||
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
|
||||
|
||||
texinfo:
|
||||
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
|
||||
@echo
|
||||
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
|
||||
@echo "Run \`make' in that directory to run these through makeinfo" \
|
||||
"(use \`make info' here to do that automatically)."
|
||||
|
||||
info:
|
||||
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
|
||||
@echo "Running Texinfo files through makeinfo..."
|
||||
make -C $(BUILDDIR)/texinfo info
|
||||
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
|
||||
|
||||
gettext:
|
||||
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
|
||||
@echo
|
||||
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
|
||||
|
||||
changes:
|
||||
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
|
||||
@echo
|
||||
@echo "The overview file is in $(BUILDDIR)/changes."
|
||||
|
||||
linkcheck:
|
||||
$(SPHINXBUILD) -b linkcheck $(ALLLINKCHECKOPTS) $(BUILDDIR)/linkcheck
|
||||
@echo
|
||||
@echo "Link check complete; look for any errors in the above output " \
|
||||
"or in $(BUILDDIR)/linkcheck/output.txt."
|
||||
|
||||
linkcheck_all:
|
||||
LINKCHECK_ALL=1 $(SPHINXBUILD) -b linkcheck $(ALLLINKCHECKOPTS) $(BUILDDIR)/linkcheck
|
||||
@echo
|
||||
@echo "Link check complete; look for any errors in the above output " \
|
||||
"or in $(BUILDDIR)/linkcheck/output.txt."
|
||||
|
||||
doctest:
|
||||
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
|
||||
@echo "Testing of doctests in the sources finished, look at the " \
|
||||
"results in $(BUILDDIR)/doctest/output.txt."
|
||||
|
||||
coverage:
|
||||
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
|
||||
@echo "Testing of coverage in the sources finished, look at the " \
|
||||
"results in $(BUILDDIR)/coverage/python.txt."
|
||||
|
||||
xml:
|
||||
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
|
||||
@echo
|
||||
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
|
||||
|
||||
pseudoxml:
|
||||
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
|
||||
@echo
|
||||
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
|
||||
|
||||
show:
|
||||
python -m http.server -d _build/html/
|
||||
+139
@@ -0,0 +1,139 @@
|
||||
# Ray Documentation
|
||||
|
||||
Repository for documentation of the Ray project, hosted at [docs.ray.io](https://docs.ray.io).
|
||||
|
||||
## Contributing new documentation pages
|
||||
|
||||
Author new documentation files under `doc/source/` as MyST Markdown (`.md`), not reStructuredText (`.rst`). A lint check rejects newly added `.rst` files. Edits to existing `.rst` files continue to work. `git mv` is not flagged for any rename, which covers file renames and directory reorganization.
|
||||
|
||||
MyST is a Markdown superset that supports the Sphinx directives used throughout the docs. See the [MyST Parser documentation](https://myst-parser.readthedocs.io/) for syntax reference.
|
||||
|
||||
## Installation
|
||||
|
||||
To build the documentation, make sure you have `ray` installed first.
|
||||
For building the documentation locally install the following dependencies:
|
||||
|
||||
```bash
|
||||
pip install -r requirements-doc.lock.txt
|
||||
```
|
||||
|
||||
## Building the documentation
|
||||
|
||||
To compile the documentation and open it locally, run the following command from this directory.
|
||||
|
||||
```bash
|
||||
make develop && open _build/html/index.html
|
||||
```
|
||||
|
||||
## Building just one subproject
|
||||
|
||||
Often your changes in documentation just concern one subproject, such as Tune or Train.
|
||||
To build just this one subproject, and ignore the rest
|
||||
(leading to build warnings due to broken references etc.), run the following command:
|
||||
|
||||
```shell
|
||||
DOC_LIB=<project> sphinx-build -b html -d _build/doctrees source _build/html
|
||||
```
|
||||
where `<project>` is the name of the subproject and can be any of the docs projects in the `source/`
|
||||
directory either called `tune`, `rllib`, `train`, `cluster`, `serve`, `data` or the ones starting
|
||||
with `ray-`, e.g. `ray-observability`.
|
||||
|
||||
## Announcements and includes
|
||||
|
||||
To add new announcements and other messaging to the top or bottom of a documentation page,
|
||||
check the `_includes` folder first to see if the message you want is already there (like "get help"
|
||||
or "we're hiring" etc.)
|
||||
If not, add the template you want and include it accordingly, i.e. with
|
||||
|
||||
```markdown
|
||||
.. include:: /_includes/<my-announcement>
|
||||
```
|
||||
|
||||
This ensures consistent messaging across documentation pages.
|
||||
|
||||
## Checking for broken links
|
||||
|
||||
To check if there are broken links, run the following (we are currently not running this
|
||||
in the CI since there are false positives).
|
||||
|
||||
```bash
|
||||
make linkcheck
|
||||
```
|
||||
|
||||
## Running doctests
|
||||
|
||||
To run tests for examples shipping with docstrings in Python files, run the following command:
|
||||
|
||||
```shell
|
||||
make doctest
|
||||
```
|
||||
|
||||
## Adding examples as MyST Markdown Notebooks
|
||||
|
||||
You can now add [executable notebooks](https://myst-nb.readthedocs.io/en/latest/use/markdown.html) to this project,
|
||||
which will get built into the documentation.
|
||||
<!--An [example can be found here](./source/serve/tutorials/rllib.md).-->
|
||||
By default, building the docs with `make develop` will not run those notebooks.
|
||||
If you set the `RUN_NOTEBOOKS` environment variable to `"cache"`, each notebook cell will be run when you build the
|
||||
documentation, and outputs will be cached into `_build/.jupyter_cache`.
|
||||
|
||||
```bash
|
||||
RUN_NOTEBOOKS="cache" make develop
|
||||
```
|
||||
|
||||
To force re-running the notebooks, use `RUN_NOTEBOOKS="force"`.
|
||||
|
||||
Using caching, this means the first time you build the documentation, it might take a while to run the notebooks.
|
||||
After that, notebook execution is only triggered when you change the notebook source file.
|
||||
|
||||
The benefits of working with notebooks for examples are that you don't separate the code from the documentation, but can still easily smoke-test the code.
|
||||
|
||||
## Adding Markdown docs from external (ecosystem) repositories
|
||||
|
||||
In order to avoid a situation where duplicate documentation files live in both the `doc/` folder
|
||||
in this repository and in external repositories of ecosystem libraries (eg. xgboost-ray), you can
|
||||
specify Markdown files that will be downloaded from other GitHub repositories during the build process.
|
||||
|
||||
In order to do that, simply edit the `EXTERNAL_MARKDOWN_FILES` list in `source/custom_directives.py`
|
||||
using the format in the comment. Before build process, the specified files will be downloaded, preprocessed
|
||||
and saved to given paths. The build process will then proceed as normal.
|
||||
|
||||
While both GitHub Markdown and MyST are supersets of Common Markdown, there are differences in syntax.
|
||||
Furthermore, some contents such as Sphinx headers are not desirable to be displayed on GitHub.
|
||||
In order to deal with this, simple preprocessing is performed to allow for differences
|
||||
in rendering on GitHub and in docs. You can use two commands (`$UNCOMMENT` and `$REMOVE`/`$END_REMOVE`)
|
||||
in the Markdown file, specified in the following way:
|
||||
|
||||
### `$UNCOMMENT`
|
||||
|
||||
GitHub:
|
||||
|
||||
```html
|
||||
<!--$UNCOMMENTthis will be uncommented--> More text
|
||||
```
|
||||
|
||||
In docs, this will become:
|
||||
|
||||
```html
|
||||
this will be uncommented More text
|
||||
```
|
||||
|
||||
### `$REMOVE`/`$END_REMOVE`
|
||||
|
||||
GitHub:
|
||||
|
||||
```html
|
||||
<!--$REMOVE-->This will be removed<!--$END_REMOVE--> More text
|
||||
```
|
||||
|
||||
In docs, this will become:
|
||||
|
||||
```html
|
||||
More text
|
||||
```
|
||||
|
||||
Please note that the parsing is extremely simple (regex replace) and will not support nesting.
|
||||
|
||||
## Testing changes locally
|
||||
|
||||
If you want to run the preprocessing locally on a specific file (to eg. see how it will render after docs have been built), run `source/preprocess_github_markdown.py PATH_TO_MARKDOWN_FILE PATH_TO_PREPROCESSED_MARKDOWN_FILE`. Make sure to also edit `EXTERNAL_MARKDOWN_FILES` in `source/custom_directives.py` so that your file does not get overwritten by one downloaded from GitHub.
|
||||
Executable
+99
@@ -0,0 +1,99 @@
|
||||
#!/bin/sh
|
||||
|
||||
USERNAME=$1
|
||||
CONDA_ENV=$2
|
||||
WHEEL=$3
|
||||
RAY_HEAD_IP=$4
|
||||
TYPE=$5
|
||||
|
||||
echo "Installing wheel..."
|
||||
sudo -u "$USERNAME" -i /bin/bash -l -c "conda init bash"
|
||||
sudo -u "$USERNAME" -i /bin/bash -l -c "conda activate $CONDA_ENV; pip install $WHEEL"
|
||||
|
||||
echo "Setting up service scripts..."
|
||||
cat > /home/"$USERNAME"/ray-head.sh << EOM
|
||||
#!/bin/bash
|
||||
|
||||
eval "$(conda shell.bash hook)"
|
||||
conda activate $CONDA_ENV
|
||||
|
||||
NUM_GPUS=\`nvidia-smi -L | wc -l\`
|
||||
|
||||
ray stop
|
||||
ulimit -n 65536
|
||||
ray start --head --port=6379 --object-manager-port=8076 --num-gpus=\$NUM_GPUS --block --dashboard-host 0.0.0.0
|
||||
EOM
|
||||
|
||||
cat > /home/"$USERNAME"/ray-worker.sh << EOM
|
||||
#!/bin/bash
|
||||
|
||||
eval "$(conda shell.bash hook)"
|
||||
conda activate $CONDA_ENV
|
||||
|
||||
NUM_GPUS=\`nvidia-smi -L | wc -l\`
|
||||
|
||||
ray stop
|
||||
ulimit -n 65536
|
||||
|
||||
while true
|
||||
do
|
||||
ray start --address=$RAY_HEAD_IP:6379 --object-manager-port=8076 --num-gpus=\$NUM_GPUS --block
|
||||
echo Ray exited. Auto-restarting in 1 second...
|
||||
sleep 1
|
||||
done
|
||||
EOM
|
||||
|
||||
cat > /home/"$USERNAME"/tensorboard.sh << EOM
|
||||
#!/bin/bash
|
||||
|
||||
eval "$(conda shell.bash hook)"
|
||||
conda activate $CONDA_ENV
|
||||
mkdir -p /home/$USERNAME/ray_results
|
||||
|
||||
tensorboard --bind_all --logdir=/home/$USERNAME/ray_results
|
||||
EOM
|
||||
|
||||
chmod +x /home/"$USERNAME"/ray-head.sh
|
||||
chmod +x /home/"$USERNAME"/ray-worker.sh
|
||||
chmod +x /home/"$USERNAME"/tensorboard.sh
|
||||
|
||||
cat > /lib/systemd/system/ray.service << EOM
|
||||
[Unit]
|
||||
Description=Ray
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=$USERNAME
|
||||
ExecStart=/bin/bash -l /home/$USERNAME/ray-$TYPE.sh
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOM
|
||||
|
||||
cat > /lib/systemd/system/tensorboard.service << EOM
|
||||
[Unit]
|
||||
Description=TensorBoard
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=$USERNAME
|
||||
ExecStart=/bin/bash -l /home/$USERNAME/tensorboard.sh
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOM
|
||||
|
||||
echo "Configure ray to start at boot..."
|
||||
systemctl enable ray
|
||||
|
||||
echo "Starting ray..."
|
||||
systemctl start ray
|
||||
|
||||
# shellcheck disable=SC2154
|
||||
if [ "$type" = "head" ]; then
|
||||
echo "Configure TensorBoard to start at boot..."
|
||||
systemctl enable tensorboard
|
||||
|
||||
echo "Starting TensorBoard..."
|
||||
systemctl start tensorboard
|
||||
fi
|
||||
@@ -0,0 +1,520 @@
|
||||
{
|
||||
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
|
||||
"contentVersion": "1.0.0.0",
|
||||
"parameters": {
|
||||
"adminUsername": {
|
||||
"type": "string",
|
||||
"defaultValue": "ubuntu",
|
||||
"metadata": {
|
||||
"description": "Username for the Virtual Machine."
|
||||
}
|
||||
},
|
||||
"publicKey": {
|
||||
"type": "securestring",
|
||||
"metadata": {
|
||||
"description": "SSH Key for the Virtual Machine"
|
||||
}
|
||||
},
|
||||
"adminPassword": {
|
||||
"type": "securestring",
|
||||
"metadata": {
|
||||
"description": "Password for the Virtual Machine and JupyterLab"
|
||||
}
|
||||
},
|
||||
"headNodeSize": {
|
||||
"type": "string",
|
||||
"defaultValue": "Standard_D2s_v3",
|
||||
"metadata": {
|
||||
"description": "The size of the head-node Virtual Machine"
|
||||
}
|
||||
},
|
||||
"headNodePriority": {
|
||||
"type": "string",
|
||||
"defaultValue": "Regular",
|
||||
"allowedValues": ["Regular", "Low", "Spot"],
|
||||
"metadata": {
|
||||
"description": "Use Azure Spot instance for worker nodes"
|
||||
}
|
||||
},
|
||||
"workerNodeSize": {
|
||||
"type": "string",
|
||||
"defaultValue": "Standard_D2s_v3",
|
||||
"metadata": {
|
||||
"description": "The size of the worker node Virtual Machine"
|
||||
}
|
||||
},
|
||||
"workerNodePriority": {
|
||||
"type": "string",
|
||||
"defaultValue": "Spot",
|
||||
"allowedValues": ["Regular", "Low", "Spot"],
|
||||
"metadata": {
|
||||
"description": "Use Azure Spot instance for worker nodes"
|
||||
}
|
||||
},
|
||||
"workerInitial": {
|
||||
"type": "int",
|
||||
"defaultValue": 1,
|
||||
"minValue": 0,
|
||||
"maxValue": 1000,
|
||||
"metadata": {
|
||||
"description": "Initial number of worker nodes"
|
||||
}
|
||||
},
|
||||
"workerMin": {
|
||||
"type": "int",
|
||||
"defaultValue": 1,
|
||||
"minValue": 0,
|
||||
"maxValue": 1000,
|
||||
"metadata": {
|
||||
"description": "Minimum number of worker nodes"
|
||||
}
|
||||
},
|
||||
"workerMax": {
|
||||
"type": "int",
|
||||
"defaultValue": 1,
|
||||
"minValue": 0,
|
||||
"maxValue": 1000,
|
||||
"metadata": {
|
||||
"description": "Maximum number of worker nodes"
|
||||
}
|
||||
},
|
||||
"condaEnv": {
|
||||
"type": "string",
|
||||
"defaultValue": "py38_tensorflow",
|
||||
"allowedValues": [
|
||||
"azureml_py36_automl",
|
||||
"azureml_py36_pytorch",
|
||||
"azureml_py36_tensorflow",
|
||||
"py38_default",
|
||||
"py38_pytorch",
|
||||
"py38_tensorflow"
|
||||
],
|
||||
"metadata": {
|
||||
"description": "Conda environment to select (installed on DSVM)"
|
||||
}
|
||||
},
|
||||
"PythonPackages": {
|
||||
"type": "string",
|
||||
"defaultValue": "ray[rllib] gym[atari]",
|
||||
"metadata": {
|
||||
"description": "Python packages to install (space separated)"
|
||||
}
|
||||
},
|
||||
"PublicWebUI": {
|
||||
"type": "bool",
|
||||
"defaultValue": true,
|
||||
"metadata": {
|
||||
"description": "Open port for web UI"
|
||||
}
|
||||
}
|
||||
},
|
||||
"variables": {
|
||||
"azureScriptInitUrl": "https://raw.githubusercontent.com/ray-project/ray/master/doc/azure/azure-init.sh",
|
||||
"location": "[resourceGroup().location]",
|
||||
"vmName": "ray-node",
|
||||
"subnetWorkers": "10.32.0.0/16",
|
||||
"subnetHead": "10.33.0.0/16",
|
||||
"publicIpAddressName": "[concat(variables('vmName'), '-ip' )]",
|
||||
"networkIpConfig": "[guid(resourceGroup().id, variables('vmName'))]",
|
||||
"subnetName": "ray-subnet",
|
||||
"subnetHeadName": "ray-subnet-head",
|
||||
"subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vNetName'), variables('subnetName'))]",
|
||||
"subnetHeadRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('vNetName'), variables('subnetHeadName'))]",
|
||||
"osDiskType": "Standard_LRS",
|
||||
"vmNameHead": "[concat(variables('vmName'), '-head')]",
|
||||
"vmNameWorker": "[concat(variables('vmName'), '-workers')]",
|
||||
"networkInterfaceName": "[concat(variables('vmName'), '-nic')]",
|
||||
"networkSecurityGroupName": "ray-nsg",
|
||||
"vNetName": "ray-vnet",
|
||||
"subnetNetwork": "[split(variables('subnetHead'), '/')[0]]",
|
||||
"headInternalIP": "[concat(substring(variables('subnetNetwork'), 0, lastIndexOf(variables('subnetNetwork'), '.')), '.5')]",
|
||||
"imagePublisher": "microsoft-dsvm",
|
||||
"imageOffer": "ubuntu-1804",
|
||||
"imageSku": "1804",
|
||||
"imageVersion": "latest"
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.Network/networkSecurityGroups",
|
||||
"apiVersion": "2019-02-01",
|
||||
"name": "[variables('networkSecurityGroupName')]",
|
||||
"location": "[variables('location')]",
|
||||
"properties": {
|
||||
"securityRules": [
|
||||
{
|
||||
"name": "SSH",
|
||||
"properties": {
|
||||
"priority": 1000,
|
||||
"protocol": "TCP",
|
||||
"access": "Allow",
|
||||
"direction": "Inbound",
|
||||
"sourceAddressPrefix": "*",
|
||||
"sourcePortRange": "*",
|
||||
"destinationAddressPrefix": "*",
|
||||
"destinationPortRange": "22"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "JupyterLab",
|
||||
"properties": {
|
||||
"priority": 1001,
|
||||
"protocol": "TCP",
|
||||
"access": "Allow",
|
||||
"direction": "Inbound",
|
||||
"sourceAddressPrefix": "*",
|
||||
"sourcePortRange": "*",
|
||||
"destinationAddressPrefix": "*",
|
||||
"destinationPortRange": "8000"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "RayWebUI",
|
||||
"properties": {
|
||||
"priority": 1002,
|
||||
"protocol": "TCP",
|
||||
"access": "[if(parameters('PublicWebUI'), 'Allow', 'Deny')]",
|
||||
"direction": "Inbound",
|
||||
"sourceAddressPrefix": "*",
|
||||
"sourcePortRange": "*",
|
||||
"destinationAddressPrefix": "*",
|
||||
"destinationPortRange": "8265"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "TensorBoard",
|
||||
"properties": {
|
||||
"priority": 1003,
|
||||
"protocol": "TCP",
|
||||
"access": "[if(parameters('PublicWebUI'), 'Allow', 'Deny')]",
|
||||
"direction": "Inbound",
|
||||
"sourceAddressPrefix": "*",
|
||||
"sourcePortRange": "*",
|
||||
"destinationAddressPrefix": "*",
|
||||
"destinationPortRange": "6006"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Network/virtualNetworks",
|
||||
"apiVersion": "2019-11-01",
|
||||
"name": "[variables('vNetName')]",
|
||||
"location": "[variables('location')]",
|
||||
"properties": {
|
||||
"addressSpace": {
|
||||
"addressPrefixes": [
|
||||
"[variables('subnetHead')]",
|
||||
"[variables('subnetWorkers')]"
|
||||
]
|
||||
},
|
||||
"subnets": [
|
||||
{
|
||||
"name": "[variables('subnetName')]",
|
||||
"properties": {
|
||||
"addressPrefix": "[variables('subnetWorkers')]"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "[variables('subnetHeadName')]",
|
||||
"properties": {
|
||||
"addressPrefix": "[variables('subnetHead')]"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Network/publicIpAddresses",
|
||||
"apiVersion": "2019-02-01",
|
||||
"name": "[variables('publicIpAddressName')]",
|
||||
"location": "[variables('location')]",
|
||||
"properties": {
|
||||
"publicIpAllocationMethod": "Static",
|
||||
"publicIPAddressVersion": "IPv4"
|
||||
},
|
||||
"sku": {
|
||||
"name": "Basic",
|
||||
"tier": "Regional"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Network/networkInterfaces",
|
||||
"apiVersion": "2020-06-01",
|
||||
"name": "[variables('networkInterfaceName')]",
|
||||
"location": "[variables('location')]",
|
||||
"dependsOn": [
|
||||
"[resourceId('Microsoft.Network/publicIpAddresses', variables('publicIpAddressName'))]",
|
||||
"[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
|
||||
],
|
||||
"properties": {
|
||||
"ipConfigurations": [
|
||||
{
|
||||
"name": "[variables('networkIpConfig')]",
|
||||
"properties": {
|
||||
"subnet": {
|
||||
"id": "[variables('subnetHeadRef')]"
|
||||
},
|
||||
"privateIPAllocationMethod": "Static",
|
||||
"privateIPAddress": "[variables('headInternalIP')]",
|
||||
"publicIpAddress": {
|
||||
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]"
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"networkSecurityGroup": {
|
||||
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Compute/virtualMachines",
|
||||
"apiVersion": "2020-06-01",
|
||||
"name": "[variables('vmNameHead')]",
|
||||
"location": "[variables('location')]",
|
||||
"dependsOn": [
|
||||
"[resourceId('Microsoft.Network/networkInterfaces', variables('networkInterfaceName'))]"
|
||||
],
|
||||
"properties": {
|
||||
"hardwareProfile": {
|
||||
"vmSize": "[parameters('headNodeSize')]"
|
||||
},
|
||||
"priority": "[parameters('headNodePriority')]",
|
||||
"storageProfile": {
|
||||
"osDisk": {
|
||||
"createOption": "fromImage",
|
||||
"managedDisk": {
|
||||
"storageAccountType": "[variables('osDiskType')]"
|
||||
}
|
||||
},
|
||||
"imageReference": {
|
||||
"publisher": "[variables('imagePublisher')]",
|
||||
"offer": "[variables('imageOffer')]",
|
||||
"sku": "[variables('imageSku')]",
|
||||
"version": "[variables('imageVersion')]"
|
||||
}
|
||||
},
|
||||
"networkProfile": {
|
||||
"networkInterfaces": [
|
||||
{
|
||||
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkInterfaceName'))]"
|
||||
}
|
||||
]
|
||||
},
|
||||
"osProfile": {
|
||||
"computerName": "[variables('vmNameHead')]",
|
||||
"adminUsername": "[parameters('adminUsername')]",
|
||||
"adminPassword": "[parameters('adminPassword')]",
|
||||
"linuxConfiguration": {
|
||||
"disablePasswordAuthentication": false,
|
||||
"ssh": {
|
||||
"publicKeys": [
|
||||
{
|
||||
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
|
||||
"keyData": "[parameters('publicKey')]"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"resources": [
|
||||
{
|
||||
"type": "Microsoft.Compute/virtualMachines/extensions",
|
||||
"name": "[concat(variables('vmNameHead'), '/HeadNodeInitScript')]",
|
||||
"apiVersion": "2020-06-01",
|
||||
"location": "[variables('location')]",
|
||||
"dependsOn": [
|
||||
"[resourceId('Microsoft.Compute/virtualMachines', variables('vmNameHead'))]"
|
||||
],
|
||||
"properties": {
|
||||
"publisher": "Microsoft.Azure.Extensions",
|
||||
"type": "CustomScript",
|
||||
"typeHandlerVersion": "2.1",
|
||||
"autoUpgradeMinorVersion": true,
|
||||
"settings": {
|
||||
"commandToExecute": "[concat('sh azure-init.sh ', parameters('adminUsername'), ' ', parameters('condaEnv'), ' \"', parameters('PythonPackages'), '\" ignore head 2>&1 >/var/log/ray-head.log')]",
|
||||
"fileUris": [
|
||||
"[variables('azureScriptInitUrl')]"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Compute/virtualMachineScaleSets",
|
||||
"name": "[variables('vmNameWorker')]",
|
||||
"location": "[variables('location')]",
|
||||
"apiVersion": "2019-07-01",
|
||||
"dependsOn": [
|
||||
"[resourceId('Microsoft.Network/virtualNetworks', variables('vNetName'))]"
|
||||
],
|
||||
"sku": {
|
||||
"name": "[parameters('workerNodeSize')]",
|
||||
"tier": "Standard",
|
||||
"capacity": "[parameters('workerInitial')]"
|
||||
},
|
||||
"properties": {
|
||||
"upgradePolicy": {
|
||||
"mode": "Manual"
|
||||
},
|
||||
"virtualMachineProfile": {
|
||||
"priority": "[parameters('workerNodePriority')]",
|
||||
"storageProfile": {
|
||||
"osDisk": {
|
||||
"createOption": "fromImage",
|
||||
"managedDisk": {
|
||||
"storageAccountType": "[variables('osDiskType')]"
|
||||
}
|
||||
},
|
||||
"imageReference": {
|
||||
"publisher": "[variables('imagePublisher')]",
|
||||
"offer": "[variables('imageOffer')]",
|
||||
"sku": "[variables('imageSku')]",
|
||||
"version": "[variables('imageVersion')]"
|
||||
}
|
||||
},
|
||||
"osProfile": {
|
||||
"computerNamePrefix": "[variables('vmNameWorker')]",
|
||||
"adminUsername": "[parameters('adminUsername')]",
|
||||
"adminPassword": "[parameters('adminPassword')]",
|
||||
"linuxConfiguration": {
|
||||
"disablePasswordAuthentication": false,
|
||||
"ssh": {
|
||||
"publicKeys": [
|
||||
{
|
||||
"path": "[concat('/home/', parameters('adminUsername'), '/.ssh/authorized_keys')]",
|
||||
"keyData": "[parameters('publicKey')]"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
},
|
||||
"networkProfile": {
|
||||
"networkInterfaceConfigurations": [
|
||||
{
|
||||
"name": "[concat(variables('vmNameWorker'),'-nic')]",
|
||||
"properties": {
|
||||
"primary": true,
|
||||
"ipConfigurations": [
|
||||
{
|
||||
"name": "worker-ip-config",
|
||||
"properties": {
|
||||
"subnet": {
|
||||
"id": "[variables('subnetRef')]"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
"extensionProfile": {
|
||||
"extensions": [
|
||||
{
|
||||
"name": "RayWorkerInitScript",
|
||||
"properties": {
|
||||
"publisher": "Microsoft.Azure.Extensions",
|
||||
"type": "CustomScript",
|
||||
"typeHandlerVersion": "2.1",
|
||||
"autoUpgradeMinorVersion": true,
|
||||
"settings": {
|
||||
"commandToExecute": "[concat('sh azure-init.sh ', parameters('adminUsername'), ' ', parameters('condaEnv'), ' \"', parameters('PythonPackages'), '\" ', variables('headInternalIP'), ' worker 2>&1 >/var/log/ray-worker.log')]",
|
||||
"fileUris": [
|
||||
"[variables('azureScriptInitUrl')]"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "Microsoft.Insights/autoscaleSettings",
|
||||
"apiVersion": "2015-04-01",
|
||||
"name": "cpuautoscale",
|
||||
"location": "[variables('location')]",
|
||||
"dependsOn": [
|
||||
"[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('vmNameWorker'))]"
|
||||
],
|
||||
"properties": {
|
||||
"name": "cpuautoscale",
|
||||
"targetResourceUri": "[concat(resourceGroup().id, '/providers/Microsoft.Compute/virtualMachineScaleSets/', variables('vmNameWorker'))]",
|
||||
"enabled": true,
|
||||
"profiles": [
|
||||
{
|
||||
"name": "Profile1",
|
||||
"capacity": {
|
||||
"minimum": "[parameters('workerMin')]",
|
||||
"maximum": "[parameters('workerMax')]",
|
||||
"default": "[parameters('workerInitial')]"
|
||||
},
|
||||
"rules": [
|
||||
{
|
||||
"metricTrigger": {
|
||||
"metricName": "Percentage CPU",
|
||||
"metricResourceUri": "[concat(resourceGroup().id, '/providers/Microsoft.Compute/virtualMachineScaleSets/', variables('vmNameWorker'))]",
|
||||
"timeGrain": "PT1M",
|
||||
"statistic": "Average",
|
||||
"timeWindow": "PT10M",
|
||||
"timeAggregation": "Average",
|
||||
"operator": "GreaterThan",
|
||||
"threshold": 80
|
||||
},
|
||||
"scaleAction": {
|
||||
"direction": "Increase",
|
||||
"type": "ChangeCount",
|
||||
"value": "1",
|
||||
"cooldown": "PT5M"
|
||||
}
|
||||
},
|
||||
{
|
||||
"metricTrigger": {
|
||||
"metricName": "Percentage CPU",
|
||||
"metricResourceUri": "[concat(resourceGroup().id, '/providers/Microsoft.Compute/virtualMachineScaleSets/', variables('vmNameWorker'))]",
|
||||
"timeGrain": "PT1M",
|
||||
"statistic": "Average",
|
||||
"timeWindow": "PT30M",
|
||||
"timeAggregation": "Average",
|
||||
"operator": "LessThan",
|
||||
"threshold": 20
|
||||
},
|
||||
"scaleAction": {
|
||||
"direction": "Decrease",
|
||||
"type": "ChangeCount",
|
||||
"value": "1",
|
||||
"cooldown": "PT5M"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
],
|
||||
"outputs": {
|
||||
"JupyterLabURL": {
|
||||
"type": "string",
|
||||
"value": "[concat('https://', reference(variables('publicIpAddressName')).ipAddress, ':8000')]"
|
||||
},
|
||||
"SSH": {
|
||||
"type": "string",
|
||||
"value": "[concat('ssh -t -L 8265:localhost:8265 -L 8888:localhost:8888 ', parameters('adminUsername'),'@', reference(variables('publicIpAddressName')).ipAddress)]"
|
||||
},
|
||||
"RayWebUIURL": {
|
||||
"type": "string",
|
||||
"value": "[concat('http://', reference(variables('publicIpAddressName')).ipAddress, ':8265')]",
|
||||
"condition": "[parameters('PublicWebUI')]"
|
||||
},
|
||||
"TensorBoard": {
|
||||
"type": "string",
|
||||
"value": "[concat('http://', reference(variables('publicIpAddressName')).ipAddress, ':6006')]",
|
||||
"condition": "[parameters('PublicWebUI')]"
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,502 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# flake8: noqa
|
||||
"""
|
||||
Hyperparameter tuning with Ray Tune
|
||||
===================================
|
||||
|
||||
Hyperparameter tuning can make the difference between an average model and a highly
|
||||
accurate one. Often simple things like choosing a different learning rate or changing
|
||||
a network layer size can have a dramatic impact on your model performance.
|
||||
|
||||
Fortunately, there are tools that help with finding the best combination of parameters.
|
||||
`Ray Tune <https://docs.ray.io/en/latest/tune.html>`_ is an industry standard tool for
|
||||
distributed hyperparameter tuning. Ray Tune includes the latest hyperparameter search
|
||||
algorithms, integrates with TensorBoard and other analysis libraries, and natively
|
||||
supports distributed training through `Ray's distributed machine learning engine
|
||||
<https://ray.io/>`_.
|
||||
|
||||
In this tutorial, we will show you how to integrate Ray Tune into your PyTorch
|
||||
training workflow. We will extend `this tutorial from the PyTorch documentation
|
||||
<https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html>`_ for training
|
||||
a CIFAR10 image classifier.
|
||||
|
||||
As you will see, we only need to add some slight modifications. In particular, we
|
||||
need to
|
||||
|
||||
1. wrap data loading and training in functions,
|
||||
2. make some network parameters configurable,
|
||||
3. add checkpointing (optional),
|
||||
4. and define the search space for the model tuning
|
||||
|
||||
|
|
||||
|
||||
To run this tutorial, please make sure the following packages are
|
||||
installed:
|
||||
|
||||
- ``ray[tune]``: Distributed hyperparameter tuning library
|
||||
- ``torchvision``: For the data transformers
|
||||
|
||||
Setup / Imports
|
||||
---------------
|
||||
Let's start with the imports:
|
||||
"""
|
||||
from functools import partial
|
||||
import os
|
||||
from tempfile import TemporaryDirectory
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
import torch.optim as optim
|
||||
from torch.utils.data import random_split
|
||||
import torchvision
|
||||
import torchvision.transforms as transforms
|
||||
from ray import train, tune
|
||||
from ray.train import Checkpoint
|
||||
from ray.tune.schedulers import ASHAScheduler
|
||||
|
||||
######################################################################
|
||||
# Most of the imports are needed for building the PyTorch model. Only the last three
|
||||
# imports are for Ray Tune.
|
||||
#
|
||||
# Data loaders
|
||||
# ------------
|
||||
# We wrap the data loaders in their own function and pass a global data directory.
|
||||
# This way we can share a data directory between different trials.
|
||||
|
||||
|
||||
def load_data(data_dir="./data"):
|
||||
transform = transforms.Compose(
|
||||
[transforms.ToTensor(), transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))]
|
||||
)
|
||||
|
||||
trainset = torchvision.datasets.CIFAR10(
|
||||
root=data_dir, train=True, download=True, transform=transform
|
||||
)
|
||||
|
||||
testset = torchvision.datasets.CIFAR10(
|
||||
root=data_dir, train=False, download=True, transform=transform
|
||||
)
|
||||
|
||||
return trainset, testset
|
||||
|
||||
|
||||
######################################################################
|
||||
# Configurable neural network
|
||||
# ---------------------------
|
||||
# We can only tune those parameters that are configurable.
|
||||
# In this example, we can specify
|
||||
# the layer sizes of the fully connected layers:
|
||||
|
||||
|
||||
class Net(nn.Module):
|
||||
def __init__(self, l1=120, l2=84):
|
||||
super(Net, self).__init__()
|
||||
self.conv1 = nn.Conv2d(3, 6, 5)
|
||||
self.pool = nn.MaxPool2d(2, 2)
|
||||
self.conv2 = nn.Conv2d(6, 16, 5)
|
||||
self.fc1 = nn.Linear(16 * 5 * 5, l1)
|
||||
self.fc2 = nn.Linear(l1, l2)
|
||||
self.fc3 = nn.Linear(l2, 10)
|
||||
|
||||
def forward(self, x):
|
||||
x = self.pool(F.relu(self.conv1(x)))
|
||||
x = self.pool(F.relu(self.conv2(x)))
|
||||
x = torch.flatten(x, 1) # flatten all dimensions except batch
|
||||
x = F.relu(self.fc1(x))
|
||||
x = F.relu(self.fc2(x))
|
||||
x = self.fc3(x)
|
||||
return x
|
||||
|
||||
|
||||
######################################################################
|
||||
# The train function
|
||||
# ------------------
|
||||
# Now it gets interesting, because we introduce some changes to the example `from the PyTorch
|
||||
# documentation <https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html>`_.
|
||||
#
|
||||
# We wrap the training script in a function ``train_cifar(config, data_dir=None)``.
|
||||
# The ``config`` parameter will receive the hyperparameters we would like to
|
||||
# train with. The ``data_dir`` specifies the directory where we load and store the data,
|
||||
# so that multiple runs can share the same data source.
|
||||
# We also load the model and optimizer state at the start of the run, if a checkpoint
|
||||
# is provided. Further down in this tutorial you will find information on how
|
||||
# to save the checkpoint and what it is used for.
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# net = Net(config["l1"], config["l2"])
|
||||
#
|
||||
# checkpoint = train.get_checkpoint()
|
||||
#
|
||||
# if checkpoint:
|
||||
# checkpoint_dir = checkpoint.to_directory()
|
||||
# checkpoint_path = os.path.join(checkpoint_dir, "checkpoint.pt")
|
||||
# checkpoint_state = torch.load(checkpoint_path)
|
||||
# start_epoch = checkpoint_state["epoch"]
|
||||
# net.load_state_dict(checkpoint_state["net_state_dict"])
|
||||
# optimizer.load_state_dict(checkpoint_state["optimizer_state_dict"])
|
||||
# else:
|
||||
# start_epoch = 0
|
||||
#
|
||||
# The learning rate of the optimizer is made configurable, too:
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)
|
||||
#
|
||||
# We also split the training data into a training and validation subset. We thus train on
|
||||
# 80% of the data and calculate the validation loss on the remaining 20%. The batch sizes
|
||||
# with which we iterate through the training and test sets are configurable as well.
|
||||
#
|
||||
# Adding (multi) GPU support with DataParallel
|
||||
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
# Image classification benefits largely from GPUs. Luckily, we can continue to use
|
||||
# PyTorch's abstractions in Ray Tune. Thus, we can wrap our model in ``nn.DataParallel``
|
||||
# to support data parallel training on multiple GPUs:
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# device = "cpu"
|
||||
# if torch.cuda.is_available():
|
||||
# device = "cuda:0"
|
||||
# if torch.cuda.device_count() > 1:
|
||||
# net = nn.DataParallel(net)
|
||||
# net.to(device)
|
||||
#
|
||||
# By using a ``device`` variable we make sure that training also works when we have
|
||||
# no GPUs available. PyTorch requires us to send our data to the GPU memory explicitly,
|
||||
# like this:
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# for i, data in enumerate(trainloader, 0):
|
||||
# inputs, labels = data
|
||||
# inputs, labels = inputs.to(device), labels.to(device)
|
||||
#
|
||||
# The code now supports training on CPUs, on a single GPU, and on multiple GPUs. Notably, Ray
|
||||
# also supports `fractional GPUs <https://docs.ray.io/en/master/using-ray-with-gpus.html#fractional-gpus>`_
|
||||
# so we can share GPUs among trials, as long as the model still fits on the GPU memory. We'll come back
|
||||
# to that later.
|
||||
#
|
||||
# Communicating with Ray Tune
|
||||
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
#
|
||||
# The most interesting part is the communication with Ray Tune:
|
||||
#
|
||||
# .. code-block:: python
|
||||
|
||||
# checkpoint_data = {
|
||||
# "epoch": epoch,
|
||||
# "net_state_dict": net.state_dict(),
|
||||
# "optimizer_state_dict": optimizer.state_dict(),
|
||||
# }
|
||||
|
||||
# with TemporaryDirectory() as tmpdir:
|
||||
# torch.save(checkpoint_data, os.path.join(tmpdir, "checkpoint.pt"))
|
||||
|
||||
# train.report(
|
||||
# {"loss": val_loss / val_steps, "accuracy": correct / total},
|
||||
# checkpoint=Checkpoint.from_directory(tmpdir),
|
||||
# )
|
||||
#
|
||||
# Here we first save a checkpoint and then report some metrics back to Ray Tune. Specifically,
|
||||
# we send the validation loss and accuracy back to Ray Tune. Ray Tune can then use these metrics
|
||||
# to decide which hyperparameter configuration lead to the best results. These metrics
|
||||
# can also be used to stop bad performing trials early in order to avoid wasting
|
||||
# resources on those trials.
|
||||
#
|
||||
# The checkpoint saving is optional, however, it is necessary if we wanted to use advanced
|
||||
# schedulers like
|
||||
# `Population Based Training <https://docs.ray.io/en/master/tune/tutorials/tune-advanced-tutorial.html>`_.
|
||||
# Also, by saving the checkpoint we can later load the trained models and validate them
|
||||
# on a test set. Lastly, saving checkpoints is useful for fault tolerance, and it allows
|
||||
# us to interrupt training and continue training later.
|
||||
#
|
||||
# Full training function
|
||||
# ~~~~~~~~~~~~~~~~~~~~~~
|
||||
#
|
||||
# The full code example looks like this:
|
||||
|
||||
|
||||
def train_cifar(config, data_dir=None):
|
||||
net = Net(config["l1"], config["l2"])
|
||||
|
||||
device = "cpu"
|
||||
if torch.cuda.is_available():
|
||||
device = "cuda:0"
|
||||
if torch.cuda.device_count() > 1:
|
||||
net = nn.DataParallel(net)
|
||||
net.to(device)
|
||||
|
||||
criterion = nn.CrossEntropyLoss()
|
||||
optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)
|
||||
|
||||
checkpoint = train.get_checkpoint()
|
||||
|
||||
if checkpoint:
|
||||
checkpoint_dir = checkpoint.to_directory()
|
||||
checkpoint_path = os.path.join(checkpoint_dir, "checkpoint.pt")
|
||||
checkpoint_state = torch.load(checkpoint_path)
|
||||
start_epoch = checkpoint_state["epoch"]
|
||||
net.load_state_dict(checkpoint_state["net_state_dict"])
|
||||
optimizer.load_state_dict(checkpoint_state["optimizer_state_dict"])
|
||||
else:
|
||||
start_epoch = 0
|
||||
|
||||
trainset, testset = load_data(data_dir)
|
||||
|
||||
test_abs = int(len(trainset) * 0.8)
|
||||
train_subset, val_subset = random_split(
|
||||
trainset, [test_abs, len(trainset) - test_abs]
|
||||
)
|
||||
|
||||
trainloader = torch.utils.data.DataLoader(
|
||||
train_subset, batch_size=int(config["batch_size"]), shuffle=True, num_workers=8
|
||||
)
|
||||
valloader = torch.utils.data.DataLoader(
|
||||
val_subset, batch_size=int(config["batch_size"]), shuffle=True, num_workers=8
|
||||
)
|
||||
|
||||
for epoch in range(start_epoch, 10): # loop over the dataset multiple times
|
||||
running_loss = 0.0
|
||||
epoch_steps = 0
|
||||
for i, data in enumerate(trainloader, 0):
|
||||
# get the inputs; data is a list of [inputs, labels]
|
||||
inputs, labels = data
|
||||
inputs, labels = inputs.to(device), labels.to(device)
|
||||
|
||||
# zero the parameter gradients
|
||||
optimizer.zero_grad()
|
||||
|
||||
# forward + backward + optimize
|
||||
outputs = net(inputs)
|
||||
loss = criterion(outputs, labels)
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
|
||||
# print statistics
|
||||
running_loss += loss.item()
|
||||
epoch_steps += 1
|
||||
if i % 2000 == 1999: # print every 2000 mini-batches
|
||||
print(
|
||||
"[%d, %5d] loss: %.3f"
|
||||
% (epoch + 1, i + 1, running_loss / epoch_steps)
|
||||
)
|
||||
running_loss = 0.0
|
||||
|
||||
# Validation loss
|
||||
val_loss = 0.0
|
||||
val_steps = 0
|
||||
total = 0
|
||||
correct = 0
|
||||
for i, data in enumerate(valloader, 0):
|
||||
with torch.no_grad():
|
||||
inputs, labels = data
|
||||
inputs, labels = inputs.to(device), labels.to(device)
|
||||
|
||||
outputs = net(inputs)
|
||||
_, predicted = torch.max(outputs.data, 1)
|
||||
total += labels.size(0)
|
||||
correct += (predicted == labels).sum().item()
|
||||
|
||||
loss = criterion(outputs, labels)
|
||||
val_loss += loss.cpu().numpy()
|
||||
val_steps += 1
|
||||
|
||||
checkpoint_data = {
|
||||
"epoch": epoch,
|
||||
"net_state_dict": net.state_dict(),
|
||||
"optimizer_state_dict": optimizer.state_dict(),
|
||||
}
|
||||
with TemporaryDirectory() as tmpdir:
|
||||
torch.save(checkpoint_data, os.path.join(tmpdir, "checkpoint.pt"))
|
||||
train.report(
|
||||
{"loss": val_loss / val_steps, "accuracy": correct / total},
|
||||
checkpoint=Checkpoint.from_directory(tmpdir),
|
||||
)
|
||||
print("Finished Training")
|
||||
|
||||
|
||||
######################################################################
|
||||
# As you can see, most of the code is adapted directly from the original example.
|
||||
#
|
||||
# Test set accuracy
|
||||
# -----------------
|
||||
# Commonly the performance of a machine learning model is tested on a hold-out test
|
||||
# set with data that has not been used for training the model. We also wrap this in a
|
||||
# function:
|
||||
|
||||
|
||||
def test_accuracy(net, device="cpu"):
|
||||
trainset, testset = load_data()
|
||||
|
||||
testloader = torch.utils.data.DataLoader(
|
||||
testset, batch_size=4, shuffle=False, num_workers=2
|
||||
)
|
||||
|
||||
correct = 0
|
||||
total = 0
|
||||
with torch.no_grad():
|
||||
for data in testloader:
|
||||
images, labels = data
|
||||
images, labels = images.to(device), labels.to(device)
|
||||
outputs = net(images)
|
||||
_, predicted = torch.max(outputs.data, 1)
|
||||
total += labels.size(0)
|
||||
correct += (predicted == labels).sum().item()
|
||||
|
||||
return correct / total
|
||||
|
||||
|
||||
######################################################################
|
||||
# The function also expects a ``device`` parameter, so we can do the
|
||||
# test set validation on a GPU.
|
||||
#
|
||||
# Configuring the search space
|
||||
# ----------------------------
|
||||
# Lastly, we need to define Ray Tune's search space. Here is an example:
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# config = {
|
||||
# "l1": tune.choice([2 ** i for i in range(9)]),
|
||||
# "l2": tune.choice([2 ** i for i in range(9)]),
|
||||
# "lr": tune.loguniform(1e-4, 1e-1),
|
||||
# "batch_size": tune.choice([2, 4, 8, 16])
|
||||
# }
|
||||
#
|
||||
# The ``tune.choice()`` accepts a list of values that are uniformly sampled from.
|
||||
# In this example, the ``l1`` and ``l2`` parameters
|
||||
# should be powers of 2 between 4 and 256, so either 4, 8, 16, 32, 64, 128, or 256.
|
||||
# The ``lr`` (learning rate) should be uniformly sampled between 0.0001 and 0.1. Lastly,
|
||||
# the batch size is a choice between 2, 4, 8, and 16.
|
||||
#
|
||||
# At each trial, Ray Tune will now randomly sample a combination of parameters from these
|
||||
# search spaces. It will then train a number of models in parallel and find the best
|
||||
# performing one among these. We also use the ``ASHAScheduler`` which will terminate bad
|
||||
# performing trials early.
|
||||
#
|
||||
# We wrap the ``train_cifar`` function with ``functools.partial`` to set the constant
|
||||
# ``data_dir`` parameter. We can also tell Ray Tune what resources should be
|
||||
# available for each trial:
|
||||
#
|
||||
# .. code-block:: python
|
||||
#
|
||||
# gpus_per_trial = 2
|
||||
# # ...
|
||||
# result = tune.run(
|
||||
# partial(train_cifar, data_dir=data_dir),
|
||||
# resources_per_trial={"cpu": 8, "gpu": gpus_per_trial},
|
||||
# config=config,
|
||||
# num_samples=num_samples,
|
||||
# scheduler=scheduler,
|
||||
# checkpoint_at_end=True)
|
||||
#
|
||||
# You can specify the number of CPUs, which are then available e.g.
|
||||
# to increase the ``num_workers`` of the PyTorch ``DataLoader`` instances. The selected
|
||||
# number of GPUs are made visible to PyTorch in each trial. Trials do not have access to
|
||||
# GPUs that haven't been requested for them - so you don't have to care about two trials
|
||||
# using the same set of resources.
|
||||
#
|
||||
# Here we can also specify fractional GPUs, so something like ``gpus_per_trial=0.5`` is
|
||||
# completely valid. The trials will then share GPUs among each other.
|
||||
# You just have to make sure that the models still fit in the GPU memory.
|
||||
#
|
||||
# After training the models, we will find the best performing one and load the trained
|
||||
# network from the checkpoint file. We then obtain the test set accuracy and report
|
||||
# everything by printing.
|
||||
#
|
||||
# The full main function looks like this:
|
||||
|
||||
|
||||
def main(num_samples=10, max_num_epochs=10, gpus_per_trial=2):
|
||||
data_dir = os.path.abspath("./data")
|
||||
load_data(data_dir)
|
||||
config = {
|
||||
"l1": tune.choice([2**i for i in range(9)]),
|
||||
"l2": tune.choice([2**i for i in range(9)]),
|
||||
"lr": tune.loguniform(1e-4, 1e-1),
|
||||
"batch_size": tune.choice([2, 4, 8, 16]),
|
||||
}
|
||||
scheduler = ASHAScheduler(
|
||||
metric="loss",
|
||||
mode="min",
|
||||
max_t=max_num_epochs,
|
||||
grace_period=1,
|
||||
reduction_factor=2,
|
||||
)
|
||||
result = tune.run(
|
||||
partial(train_cifar, data_dir=data_dir),
|
||||
resources_per_trial={"cpu": 2, "gpu": gpus_per_trial},
|
||||
config=config,
|
||||
num_samples=num_samples,
|
||||
scheduler=scheduler,
|
||||
)
|
||||
|
||||
best_trial = result.get_best_trial("loss", "min", "last")
|
||||
print(f"Best trial config: {best_trial.config}")
|
||||
print(f"Best trial final validation loss: {best_trial.last_result['loss']}")
|
||||
print(f"Best trial final validation accuracy: {best_trial.last_result['accuracy']}")
|
||||
|
||||
best_trained_model = Net(best_trial.config["l1"], best_trial.config["l2"])
|
||||
device = "cpu"
|
||||
if torch.cuda.is_available():
|
||||
device = "cuda:0"
|
||||
if gpus_per_trial > 1:
|
||||
best_trained_model = nn.DataParallel(best_trained_model)
|
||||
best_trained_model.to(device)
|
||||
|
||||
best_checkpoint = best_trial.checkpoint
|
||||
best_checkpoint_dir = best_checkpoint.to_directory()
|
||||
best_checkpoint_path = os.path.join(best_checkpoint_dir, "checkpoint.pt")
|
||||
best_checkpoint_data = torch.load(best_checkpoint_path)
|
||||
|
||||
best_trained_model.load_state_dict(best_checkpoint_data["net_state_dict"])
|
||||
|
||||
test_acc = test_accuracy(best_trained_model, device)
|
||||
print("Best trial test set accuracy: {}".format(test_acc))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
# sphinx_gallery_start_ignore
|
||||
# Fixes ``AttributeError: '_LoggingTee' object has no attribute 'fileno'``.
|
||||
# This is only needed to run with sphinx-build.
|
||||
import sys
|
||||
|
||||
sys.stdout.fileno = lambda: False
|
||||
# sphinx_gallery_end_ignore
|
||||
# You can change the number of GPUs per trial here:
|
||||
main(num_samples=10, max_num_epochs=10, gpus_per_trial=0)
|
||||
|
||||
|
||||
######################################################################
|
||||
# If you run the code, an example output could look like this:
|
||||
#
|
||||
# ::
|
||||
#
|
||||
# Number of trials: 10/10 (10 TERMINATED)
|
||||
# +-----+--------------+------+------+-------------+--------+---------+------------+
|
||||
# | ... | batch_size | l1 | l2 | lr | iter | loss | accuracy |
|
||||
# |-----+--------------+------+------+-------------+--------+---------+------------|
|
||||
# | ... | 2 | 1 | 256 | 0.000668163 | 1 | 2.31479 | 0.0977 |
|
||||
# | ... | 4 | 64 | 8 | 0.0331514 | 1 | 2.31605 | 0.0983 |
|
||||
# | ... | 4 | 2 | 1 | 0.000150295 | 1 | 2.30755 | 0.1023 |
|
||||
# | ... | 16 | 32 | 32 | 0.0128248 | 10 | 1.66912 | 0.4391 |
|
||||
# | ... | 4 | 8 | 128 | 0.00464561 | 2 | 1.7316 | 0.3463 |
|
||||
# | ... | 8 | 256 | 8 | 0.00031556 | 1 | 2.19409 | 0.1736 |
|
||||
# | ... | 4 | 16 | 256 | 0.00574329 | 2 | 1.85679 | 0.3368 |
|
||||
# | ... | 8 | 2 | 2 | 0.00325652 | 1 | 2.30272 | 0.0984 |
|
||||
# | ... | 2 | 2 | 2 | 0.000342987 | 2 | 1.76044 | 0.292 |
|
||||
# | ... | 4 | 64 | 32 | 0.003734 | 8 | 1.53101 | 0.4761 |
|
||||
# +-----+--------------+------+------+-------------+--------+---------+------------+
|
||||
#
|
||||
# Best trial config: {'l1': 64, 'l2': 32, 'lr': 0.0037339984519545164, 'batch_size': 4}
|
||||
# Best trial final validation loss: 1.5310075663924216
|
||||
# Best trial final validation accuracy: 0.4761
|
||||
# Best trial test set accuracy: 0.4737
|
||||
#
|
||||
# Most trials have been stopped early in order to avoid wasting resources.
|
||||
# The best performing trial achieved a validation accuracy of about 47%, which could
|
||||
# be confirmed on the test set.
|
||||
#
|
||||
# So that's it! You can now tune the parameters of your PyTorch models.
|
||||
Vendored
+47
@@ -0,0 +1,47 @@
|
||||
import hashlib
|
||||
import sys
|
||||
from typing import TypedDict
|
||||
import os
|
||||
|
||||
import runfiles
|
||||
import pytest
|
||||
|
||||
_REPO_NAME = "io_ray"
|
||||
|
||||
_runfiles = runfiles.Create()
|
||||
|
||||
|
||||
class ExternalDoc(TypedDict):
|
||||
file: str
|
||||
digest: str
|
||||
ref: str
|
||||
|
||||
|
||||
# Files here are referenced on external pages as examples, and are tested
|
||||
# to make sure exteranl referenced Ray examples are working with latest version
|
||||
# of Ray. If you need to make changes, make sure to update the external examples
|
||||
# too, and then update the digests here as a confirmation.
|
||||
docs = [
|
||||
ExternalDoc(
|
||||
file="pytorch_tutorials_hyperparameter_tuning_tutorial.py",
|
||||
digest="04f8bab9fda98bceaf541984482faacab7bd8d35d6e5850ae610bfea08709743",
|
||||
ref="https://pytorch.org/tutorials/beginner/hyperparameter_tuning_tutorial.html"
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def test_hashes():
|
||||
for doc in docs:
|
||||
path = os.path.join(_REPO_NAME, "doc", "external", doc["file"])
|
||||
runfile = _runfiles.Rlocation(path)
|
||||
with open(runfile, "rb") as f:
|
||||
content = f.read()
|
||||
want = doc["digest"]
|
||||
got = hashlib.sha256(content).hexdigest()
|
||||
name = doc["file"]
|
||||
ref = doc["ref"]
|
||||
assert got == want, f"{name} ({ref}) has sha256 {got}, want {want}"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(pytest.main(["-v", __file__]))
|
||||
@@ -0,0 +1,170 @@
|
||||
import subprocess
|
||||
import tarfile
|
||||
import os
|
||||
import time
|
||||
|
||||
import click
|
||||
import requests
|
||||
|
||||
LAST_BUILD_CUTOFF = 3 # how many days ago to consider a build outdated
|
||||
PENDING_FILES_PATH = "pending_files.txt"
|
||||
ENVIRONMENT_PICKLE = "_build/doctrees/environment.pickle"
|
||||
DOC_BUILD_CACHE_URL = "https://ci.ray.io/ray/doc/build-cache"
|
||||
|
||||
|
||||
def _build_cache_url(commit: str):
|
||||
return f"{DOC_BUILD_CACHE_URL}/{commit}.tgz"
|
||||
|
||||
|
||||
def find_latest_master_commit():
|
||||
"""Find the latest origin/master commit that has a build cache uploaded.
|
||||
|
||||
Walk origin/master, not HEAD. Read the Docs checks out PRs with a shallow
|
||||
``git clone --depth 1``, so HEAD's history is grafted -- a plain ``git log``
|
||||
then sees ~1 commit and never reaches a cached master commit. post_checkout
|
||||
in .readthedocs.yaml deepens origin/master (``git fetch --depth=500 origin
|
||||
master``), so that ref has real history to probe. Fall back to HEAD when
|
||||
origin/master is unavailable (e.g. a local checkout without that
|
||||
remote-tracking ref).
|
||||
"""
|
||||
has_origin_master = (
|
||||
subprocess.run(
|
||||
["git", "rev-parse", "--verify", "origin/master"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
).returncode
|
||||
== 0
|
||||
)
|
||||
ref = "origin/master" if has_origin_master else "HEAD"
|
||||
|
||||
latest_commits = (
|
||||
subprocess.check_output(["git", "log", "-n", "100", "--format=%H", ref])
|
||||
.strip()
|
||||
.decode("utf-8")
|
||||
.split("\n")
|
||||
)
|
||||
for commit in latest_commits:
|
||||
with requests.head(_build_cache_url(commit), allow_redirects=True) as response:
|
||||
if response.status_code == 200:
|
||||
return commit
|
||||
raise Exception(
|
||||
"No cache found for latest master commit. "
|
||||
"Please merge with upstream master or use 'make develop'."
|
||||
)
|
||||
|
||||
|
||||
def fetch_cache(commit, target_file_path):
|
||||
"""
|
||||
Fetch doc cache archive from ci.ray.io
|
||||
|
||||
Args:
|
||||
commit: The commit hash of the doc cache to fetch
|
||||
target_file_path: The file path to save the doc cache archive
|
||||
"""
|
||||
|
||||
with requests.get(
|
||||
_build_cache_url(commit), allow_redirects=True, stream=True
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
with open(target_file_path, "wb") as f:
|
||||
for chunk in response.iter_content(chunk_size=8192):
|
||||
f.write(chunk)
|
||||
print(f"Successfully downloaded {target_file_path}")
|
||||
|
||||
|
||||
def extract_cache(cache_path: str, doc_dir: str):
|
||||
"""
|
||||
Extract the doc cache archive to overwrite the ray/doc directory
|
||||
|
||||
Args:
|
||||
file_path: The file path of the doc cache archive
|
||||
"""
|
||||
with tarfile.open(cache_path, "r:gz") as tar:
|
||||
tar.extractall(doc_dir)
|
||||
print(f"Extracted {cache_path} to {doc_dir}")
|
||||
|
||||
|
||||
def list_changed_and_added_files(ray_dir: str, latest_master_commit: str):
|
||||
"""
|
||||
List all changed and added untracked files in the repo.
|
||||
This is to prevent cache environment from updating timestamp of these files.
|
||||
"""
|
||||
untracked_files = (
|
||||
subprocess.check_output(
|
||||
["git", "ls-files", "--others"],
|
||||
cwd=ray_dir,
|
||||
)
|
||||
.decode("utf-8")
|
||||
.split(os.linesep)
|
||||
)
|
||||
modified_files = (
|
||||
subprocess.check_output(
|
||||
["git", "ls-files", "--modified"],
|
||||
cwd=ray_dir,
|
||||
)
|
||||
.decode("utf-8")
|
||||
.split(os.linesep)
|
||||
)
|
||||
diff_files_with_master = (
|
||||
subprocess.check_output(
|
||||
["git", "diff", "--name-only", latest_master_commit],
|
||||
cwd=ray_dir,
|
||||
)
|
||||
.decode("utf-8")
|
||||
.split(os.linesep)
|
||||
)
|
||||
filenames = []
|
||||
for file in untracked_files + modified_files + diff_files_with_master:
|
||||
filename = file
|
||||
if filename.startswith("doc/"): # Remove "doc/" prefix
|
||||
filename = filename.replace("doc/", "")
|
||||
if filename.startswith("source/"): # Remove "doc/" prefix
|
||||
filename = filename.replace("source/", "")
|
||||
filenames.append(filename)
|
||||
return filenames
|
||||
|
||||
|
||||
def should_load_cache(ray_dir: str):
|
||||
"""
|
||||
Check if cache should be loaded based on the timestamp of last build.
|
||||
"""
|
||||
ray_doc_dir = os.path.join(ray_dir, "doc")
|
||||
if not os.path.exists(f"{ray_doc_dir}/{ENVIRONMENT_PICKLE}"):
|
||||
print("Doc build environment pickle file does not exist.")
|
||||
return True
|
||||
last_build_time = os.path.getmtime(f"{ray_doc_dir}/{ENVIRONMENT_PICKLE}")
|
||||
current_time = time.time()
|
||||
# Load cache if last build was more than LAST_BUILD_CUTOFF days ago
|
||||
print("time diff: ", current_time - last_build_time)
|
||||
if current_time - last_build_time > LAST_BUILD_CUTOFF * 60 * 60 * 24:
|
||||
print(f"Last build was more than {LAST_BUILD_CUTOFF} days ago.")
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.option("--ray-dir", default="/ray", help="Path to Ray repo")
|
||||
def main(ray_dir: str) -> None:
|
||||
if not should_load_cache(ray_dir):
|
||||
print("Skip loading global cache...")
|
||||
return
|
||||
print("Loading global cache ...")
|
||||
latest_master_commit = find_latest_master_commit()
|
||||
# List all changed and added files in the repo
|
||||
filenames = list_changed_and_added_files(ray_dir, latest_master_commit)
|
||||
with open(
|
||||
f"{ray_dir}/{PENDING_FILES_PATH}", "w"
|
||||
) as f: # Save to file to be used when updating cache environment
|
||||
f.write("\n".join(filenames))
|
||||
|
||||
cache_path = f"{ray_dir}/doc.tgz"
|
||||
# Fetch cache of that commit from build cache archive to cache_path
|
||||
print(f"Use build cache for commit {latest_master_commit}")
|
||||
fetch_cache(latest_master_commit, cache_path)
|
||||
# Extract cache to override ray/doc directory
|
||||
extract_cache(cache_path, f"{ray_dir}/doc")
|
||||
os.remove(cache_path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+263
@@ -0,0 +1,263 @@
|
||||
@ECHO OFF
|
||||
|
||||
REM Command file for Sphinx documentation
|
||||
|
||||
if "%SPHINXBUILD%" == "" (
|
||||
set SPHINXBUILD=sphinx-build
|
||||
)
|
||||
set BUILDDIR=_build
|
||||
set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% .
|
||||
set I18NSPHINXOPTS=%SPHINXOPTS% .
|
||||
if NOT "%PAPER%" == "" (
|
||||
set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS%
|
||||
set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS%
|
||||
)
|
||||
|
||||
if "%1" == "" goto help
|
||||
|
||||
if "%1" == "help" (
|
||||
:help
|
||||
echo.Please use `make ^<target^>` where ^<target^> is one of
|
||||
echo. html to make standalone HTML files
|
||||
echo. dirhtml to make HTML files named index.html in directories
|
||||
echo. singlehtml to make a single large HTML file
|
||||
echo. pickle to make pickle files
|
||||
echo. json to make JSON files
|
||||
echo. htmlhelp to make HTML files and a HTML help project
|
||||
echo. qthelp to make HTML files and a qthelp project
|
||||
echo. devhelp to make HTML files and a Devhelp project
|
||||
echo. epub to make an epub
|
||||
echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter
|
||||
echo. text to make text files
|
||||
echo. man to make manual pages
|
||||
echo. texinfo to make Texinfo files
|
||||
echo. gettext to make PO message catalogs
|
||||
echo. changes to make an overview over all changed/added/deprecated items
|
||||
echo. xml to make Docutils-native XML files
|
||||
echo. pseudoxml to make pseudoxml-XML files for display purposes
|
||||
echo. linkcheck to check all external links for integrity
|
||||
echo. doctest to run all doctests embedded in the documentation if enabled
|
||||
echo. coverage to run coverage check of the documentation if enabled
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "clean" (
|
||||
for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i
|
||||
del /q /s %BUILDDIR%\*
|
||||
goto end
|
||||
)
|
||||
|
||||
|
||||
REM Check if sphinx-build is available and fallback to Python version if any
|
||||
%SPHINXBUILD% 2> nul
|
||||
if errorlevel 9009 goto sphinx_python
|
||||
goto sphinx_ok
|
||||
|
||||
:sphinx_python
|
||||
|
||||
set SPHINXBUILD=python -m sphinx.__init__
|
||||
%SPHINXBUILD% 2> nul
|
||||
if errorlevel 9009 (
|
||||
echo.
|
||||
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
|
||||
echo.installed, then set the SPHINXBUILD environment variable to point
|
||||
echo.to the full path of the 'sphinx-build' executable. Alternatively you
|
||||
echo.may add the Sphinx directory to PATH.
|
||||
echo.
|
||||
echo.If you don't have Sphinx installed, grab it from
|
||||
echo.http://sphinx-doc.org/
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
:sphinx_ok
|
||||
|
||||
|
||||
if "%1" == "html" (
|
||||
%SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The HTML pages are in %BUILDDIR%/html.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "dirhtml" (
|
||||
%SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "singlehtml" (
|
||||
%SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "pickle" (
|
||||
%SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished; now you can process the pickle files.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "json" (
|
||||
%SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished; now you can process the JSON files.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "htmlhelp" (
|
||||
%SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished; now you can run HTML Help Workshop with the ^
|
||||
.hhp project file in %BUILDDIR%/htmlhelp.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "qthelp" (
|
||||
%SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished; now you can run "qcollectiongenerator" with the ^
|
||||
.qhcp project file in %BUILDDIR%/qthelp, like this:
|
||||
echo.^> qcollectiongenerator %BUILDDIR%\qthelp\Ray.qhcp
|
||||
echo.To view the help file:
|
||||
echo.^> assistant -collectionFile %BUILDDIR%\qthelp\Ray.ghc
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "devhelp" (
|
||||
%SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "epub" (
|
||||
%SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The epub file is in %BUILDDIR%/epub.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "latex" (
|
||||
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished; the LaTeX files are in %BUILDDIR%/latex.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "latexpdf" (
|
||||
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
|
||||
cd %BUILDDIR%/latex
|
||||
make all-pdf
|
||||
cd %~dp0
|
||||
echo.
|
||||
echo.Build finished; the PDF files are in %BUILDDIR%/latex.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "latexpdfja" (
|
||||
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
|
||||
cd %BUILDDIR%/latex
|
||||
make all-pdf-ja
|
||||
cd %~dp0
|
||||
echo.
|
||||
echo.Build finished; the PDF files are in %BUILDDIR%/latex.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "text" (
|
||||
%SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The text files are in %BUILDDIR%/text.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "man" (
|
||||
%SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The manual pages are in %BUILDDIR%/man.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "texinfo" (
|
||||
%SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "gettext" (
|
||||
%SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The message catalogs are in %BUILDDIR%/locale.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "changes" (
|
||||
%SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.The overview file is in %BUILDDIR%/changes.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "linkcheck" (
|
||||
%SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Link check complete; look for any errors in the above output ^
|
||||
or in %BUILDDIR%/linkcheck/output.txt.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "doctest" (
|
||||
%SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Testing of doctests in the sources finished, look at the ^
|
||||
results in %BUILDDIR%/doctest/output.txt.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "coverage" (
|
||||
%SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Testing of coverage in the sources finished, look at the ^
|
||||
results in %BUILDDIR%/coverage/python.txt.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "xml" (
|
||||
%SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The XML files are in %BUILDDIR%/xml.
|
||||
goto end
|
||||
)
|
||||
|
||||
if "%1" == "pseudoxml" (
|
||||
%SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml
|
||||
if errorlevel 1 exit /b 1
|
||||
echo.
|
||||
echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml.
|
||||
goto end
|
||||
)
|
||||
|
||||
:end
|
||||
@@ -0,0 +1,41 @@
|
||||
# Read the Docs redirects for docs.ray.io
|
||||
|
||||
`current.yaml` is the source of truth for the HTTP redirects configured on the
|
||||
`anyscale-ray` Read the Docs project, which serves docs.ray.io. The file is
|
||||
managed with [rtd-redirects](https://github.com/anyscale/rtd-redirects) and
|
||||
mirrors the live configuration exactly.
|
||||
|
||||
## Policy
|
||||
|
||||
- Change redirects by editing `current.yaml` in a pull request, not in the
|
||||
Read the Docs dashboard. Dashboard edits are out-of-process: they aren't
|
||||
reviewed, and the next reconciliation overwrites them.
|
||||
- If an urgent fix has to land through the dashboard, record why in the PR
|
||||
that follows, and open that reconcile PR within 24 hours
|
||||
(`rtd-redirects dump --project anyscale-ray -o doc/redirects/current.yaml`
|
||||
regenerates this file from live state).
|
||||
|
||||
## Adding or changing a redirect
|
||||
|
||||
1. Edit `current.yaml`. Prefer `type: page` rules with version-less paths;
|
||||
they apply to every docs version. Point `to` at the final destination, not
|
||||
at another redirect's source.
|
||||
2. Validate locally: `rtd-redirects validate doc/redirects/current.yaml`.
|
||||
No Read the Docs credentials needed.
|
||||
3. Open a PR. After it merges, a maintainer applies the change with
|
||||
`rtd-redirects apply --project anyscale-ray --file doc/redirects/current.yaml --strict`.
|
||||
CI automation for this step is planned; application is manual for now.
|
||||
|
||||
## Auditing drift
|
||||
|
||||
`rtd-redirects plan --project anyscale-ray --file doc/redirects/current.yaml`
|
||||
shows any difference between this file and the live configuration. An empty
|
||||
plan means no drift.
|
||||
|
||||
The ruleset began as a May-June 2026 cleanup that reduced 287 inherited rules
|
||||
to a curated set of 169 version-agnostic `page` rules plus 3 intentional
|
||||
version-pinned `exact` rules, all returning 301. The pre-cleanup snapshot is
|
||||
preserved in git history. A later legacy-version 404-coverage pass
|
||||
added 21 `page` catch-all rules that land high-traffic out-of-support docs
|
||||
paths, which have no equivalent on current docs, on the nearest surviving
|
||||
section index.
|
||||
@@ -0,0 +1,584 @@
|
||||
schema_version: 1
|
||||
redirects:
|
||||
- from: /train/user-guides/using-gpus.html
|
||||
to: /train/user-guides/using-accelerators.html
|
||||
type: page
|
||||
- from: /serve/llm/prefix-aware-request-router.html
|
||||
to: /serve/llm/user-guides/prefix-aware-routing.html
|
||||
type: page
|
||||
- from: /serve/llm/pd-dissagregation.html
|
||||
to: /serve/llm/user-guides/prefill-decode.html
|
||||
type: page
|
||||
- from: /serve/llm/api.html
|
||||
to: /serve/api/index.html#llm-api
|
||||
type: page
|
||||
- from: /serve/tutorials/intel-habana-gaudi-inference.html
|
||||
to: /serve/tutorials/intel-gaudi-inference.html
|
||||
type: page
|
||||
- from: /train/examples/hpu/resnet.html
|
||||
to: /train/examples/intel_gaudi/resnet.html
|
||||
type: page
|
||||
force: true
|
||||
description: redirect for remapping of "hpu" to "intel-gaudi" brand
|
||||
- from: /ray-observability/ray-logging.html
|
||||
to: /ray-observability/user-guides/configure-logging.html
|
||||
type: page
|
||||
- from: /ray-air/examples/convert_existing_pytorch_code_to_ray_air.html
|
||||
to: /train/examples/pytorch/convert_existing_pytorch_code_to_ray_train.html
|
||||
type: page
|
||||
- from: /ray-observability/ray-logging.html#logging-directory-structure
|
||||
to: /ray-observability/user-guides/configure-logging.html#logging-directory-structure
|
||||
type: page
|
||||
- from: /ray-core/ray-dashboard.html
|
||||
to: /ray-observability/getting-started.html
|
||||
type: page
|
||||
- from: /ray-observability/ray-metrics.html
|
||||
to: /cluster/metrics.html
|
||||
type: page
|
||||
- from: /ray-observability/monitoring-debugging/monitoring-debugging.html
|
||||
to: /ray-observability/index.html
|
||||
type: page
|
||||
- from: /data/dataset.html
|
||||
to: /en/master/data/api/dataset.html
|
||||
type: page
|
||||
- from: /en/releases-0.8.7/cluster/launcher.html
|
||||
to: /en/latest/cluster/getting-started.html
|
||||
type: exact
|
||||
- from: /cluster/cloud.html
|
||||
to: /cluster/getting-started.html
|
||||
type: page
|
||||
- from: /en/releases-2.0.0rc0/ray-air/config-scaling.html
|
||||
to: /en/latest/train/config_guide.html#scaling-configurations-in-train-scalingconfig
|
||||
type: exact
|
||||
- from: /en/releases-2.0.0rc0/cluster/cluster_under_construction/getting-started.html
|
||||
to: /en/latest/cluster/vms/getting-started.html
|
||||
type: exact
|
||||
- from: /ray-core/actors/patterns/fault-tolerance-actor-checkpointing.html
|
||||
to: /ray-core/fault_tolerance/actors.html
|
||||
type: page
|
||||
- from: /ray-observability/state/ray-state-api-reference.html
|
||||
to: /ray-observability/reference/api.html
|
||||
type: page
|
||||
- from: /ray-observability/state/cli.html
|
||||
to: /ray-observability/reference/cli.html
|
||||
type: page
|
||||
- from: /ray-core/cli.html
|
||||
to: /ray-core/api/cli.html
|
||||
type: page
|
||||
- from: /ray-core/package-ref.html
|
||||
to: /ray-core/api/index.html
|
||||
type: page
|
||||
- from: /train/api.html
|
||||
to: /train/api/api.html
|
||||
type: page
|
||||
- from: /ray-air/package-ref.html
|
||||
to: /ray-air/getting-started.html
|
||||
type: page
|
||||
- from: /tune/tutorials/tune-checkpoints.html
|
||||
to: /tune/tutorials/tune-storage.html
|
||||
type: page
|
||||
- from: /tune/api_docs/overview.html
|
||||
to: /tune/api/api.html
|
||||
type: page
|
||||
- from: /tune/api_docs/*
|
||||
to: /tune/api/:splat
|
||||
type: page
|
||||
- from: /hyperband.html
|
||||
to: /tune/api/schedulers.html#asynchronous-hyperband
|
||||
type: page
|
||||
- from: /pbt.html
|
||||
to: /tune/api/schedulers.html#population-based-training-pbt
|
||||
type: page
|
||||
- from: /tune-config.html
|
||||
to: /tune/index.html
|
||||
type: page
|
||||
- from: /contact.html
|
||||
to: /index.html#contact
|
||||
type: page
|
||||
- from: /tutorial.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /tempfile.html
|
||||
to: /ray-core/configure.html
|
||||
type: page
|
||||
- from: /serialization.html
|
||||
to: /ray-core/objects/serialization.html
|
||||
type: page
|
||||
- from: /install-on-docker.html
|
||||
to: /ray-overview/installation.html
|
||||
type: page
|
||||
- from: /installation-troubleshooting.html
|
||||
to: /ray-overview/installation.html
|
||||
type: page
|
||||
- from: /resources.html
|
||||
to: /ray-core/scheduling/resources.html
|
||||
type: page
|
||||
- from: /plasma-object-store.html
|
||||
to: /ray-core/configure.html#using-the-object-store-with-huge-pages
|
||||
type: page
|
||||
- from: /tune-package-ref.html
|
||||
to: /tune/api/api.html
|
||||
type: page
|
||||
- from: /tune-distributed.html
|
||||
to: /tune/tutorials/tune-distributed.html
|
||||
type: page
|
||||
- from: /tune-advanced-tutorial.html
|
||||
to: /tune/tutorials/overview.html
|
||||
type: page
|
||||
- from: /tune-tutorial.html
|
||||
to: /tune/getting-started.html
|
||||
type: page
|
||||
- from: /tune.html
|
||||
to: /tune/index.html
|
||||
type: page
|
||||
- from: /tune-searchalg.html
|
||||
to: /tune/api/suggestion.html
|
||||
type: page
|
||||
- from: /tune-schedulers.html
|
||||
to: /tune/api/schedulers.html
|
||||
type: page
|
||||
- from: /tune-examples.html
|
||||
to: /tune/tutorials/overview.html
|
||||
type: page
|
||||
- from: /tune-usage.html
|
||||
to: /tune/tutorials/overview.html
|
||||
type: page
|
||||
- from: /autoscaling.html
|
||||
to: /cluster/vms/user-guides/configuring-autoscaling.html
|
||||
type: page
|
||||
- from: /rllib-toc.html
|
||||
to: /rllib/index.html
|
||||
type: page
|
||||
- from: /rllib-training.html
|
||||
to: /rllib/getting-started.html
|
||||
type: page
|
||||
- from: /rllib-models.html
|
||||
to: /rllib/rl-modules.html
|
||||
type: page
|
||||
- from: /rllib-sample-collection.html
|
||||
to: /rllib/rllib-examples.html
|
||||
type: page
|
||||
- from: /rllib-concepts.html
|
||||
to: /rllib/key-concepts.html
|
||||
type: page
|
||||
- from: /using-ray.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /async_api.html
|
||||
to: /ray-core/actors/async_api.html
|
||||
type: page
|
||||
- from: /concurrency_group_api.html
|
||||
to: /ray-core/actors/concurrency_group_api.html
|
||||
type: page
|
||||
- from: /using-ray-with-gpus.html
|
||||
to: /ray-core/scheduling/accelerators.html
|
||||
type: page
|
||||
- from: /memory-management.html
|
||||
to: /ray-core/scheduling/memory-management.html
|
||||
type: page
|
||||
- from: /troubleshooting.html
|
||||
to: /ray-core/user-guide.html
|
||||
type: page
|
||||
- from: /advanced.html
|
||||
to: /ray-core/advanced-topics.html
|
||||
type: page
|
||||
- from: /using-ray-with-jupyter.html
|
||||
to: /ray-core/using-ray-with-jupyter.html
|
||||
type: page
|
||||
- from: /auto_examples/tips-for-first-time.html
|
||||
to: /ray-core/tips-for-first-time.html
|
||||
type: page
|
||||
- from: /auto_examples/testing-tips.html
|
||||
to: /ray-contribute/testing-tips.html
|
||||
type: page
|
||||
- from: /auto_examples/plot_lbfgs.html
|
||||
to: /ray-core/examples/overview.html
|
||||
type: page
|
||||
- from: /auto_examples/plot_example-lm.html
|
||||
to: /ray-core/examples/overview.html
|
||||
type: page
|
||||
- from: /auto_examples/dask_xgboost/dask_xgboost.html
|
||||
to: /train/getting-started-xgboost.html
|
||||
type: page
|
||||
- from: /auto_examples/modin_xgboost/modin_xgboost.html
|
||||
to: /train/getting-started-xgboost.html
|
||||
type: page
|
||||
- from: /auto_examples/plot_example-a3c.html
|
||||
to: /ray-core/examples/overview.html
|
||||
type: page
|
||||
- from: /auto_examples/using-ray-with-pytorch-lightning.html
|
||||
to: /train/getting-started-pytorch-lightning.html
|
||||
type: page
|
||||
- from: /ray-job-submission/overview.html
|
||||
to: /cluster/running-applications/job-submission/index.html
|
||||
type: page
|
||||
- from: /xgboost-ray.html
|
||||
to: /train/getting-started-xgboost.html
|
||||
type: page
|
||||
- from: /lightgbm-ray.html
|
||||
to: /train/getting-started-lightgbm.html
|
||||
type: page
|
||||
- from: /ray-lightning.html
|
||||
to: /train/getting-started-pytorch-lightning.html
|
||||
type: page
|
||||
- from: /ray-collective.html
|
||||
to: /ray-more-libs/ray-collective.html
|
||||
type: page
|
||||
- from: /ray-debugging.html
|
||||
to: /ray-observability/user-guides/debug-apps/ray-debugging.html
|
||||
type: page
|
||||
- from: /ray-tracing.html
|
||||
to: /ray-observability/user-guides/ray-tracing.html
|
||||
type: page
|
||||
- from: /debugging.html
|
||||
to: /ray-contribute/debugging.html
|
||||
type: page
|
||||
- from: /fake-autoscaler.html
|
||||
to: /ray-contribute/fake-autoscaler.html
|
||||
type: page
|
||||
- from: /data/dataset-pipeline.html
|
||||
to: /data/quickstart.html
|
||||
type: page
|
||||
- from: /data/dataset-execution-model.html
|
||||
to: /data/key-concepts.html
|
||||
type: page
|
||||
- from: /actors.html
|
||||
to: /ray-core/actors.html
|
||||
type: page
|
||||
- from: /installation.html
|
||||
to: /ray-overview/installation.html
|
||||
type: page
|
||||
- from: /rllib.html
|
||||
to: /rllib/index.html
|
||||
type: page
|
||||
- from: /placement-group.html
|
||||
to: /en/master/ray-core/scheduling/placement-group.html
|
||||
type: page
|
||||
- from: /joblib.html
|
||||
to: /ray-more-libs/joblib.html
|
||||
type: page
|
||||
- from: /development.html
|
||||
to: /ray-contribute/development.html
|
||||
type: page
|
||||
- from: /rllib-env.html
|
||||
to: /rllib/rllib-env.html
|
||||
type: page
|
||||
- from: /fault-tolerance.html
|
||||
to: /ray-core/fault-tolerance.html
|
||||
type: page
|
||||
- from: /getting-involved.html
|
||||
to: /ray-contribute/getting-involved.html
|
||||
type: page
|
||||
- from: /tune/user-guide.html
|
||||
to: /tune/tutorials/overview.html
|
||||
type: page
|
||||
- from: /ray-logging.html
|
||||
to: /ray-observability/user-guides/configure-logging.html
|
||||
type: page
|
||||
- from: /ray-metrics.html
|
||||
to: /cluster/metrics.html
|
||||
type: page
|
||||
- from: /ray-dashboard.html
|
||||
to: /ray-observability/getting-started.html
|
||||
type: page
|
||||
- from: /ray-more-libs/actors.html
|
||||
to: /ray-core/actors.html
|
||||
type: page
|
||||
- from: /ray-more-libs/cluster/index.html
|
||||
to: /cluster/getting-started.html
|
||||
type: page
|
||||
- from: /ray-more-libs/installation.html
|
||||
to: /ray-overview/installation.html
|
||||
type: page
|
||||
- from: /rllib/actors.html
|
||||
to: /ray-core/actors.html
|
||||
type: page
|
||||
- from: /rllib/rllib-package-ref.html
|
||||
to: /rllib/package_ref/index.html
|
||||
type: page
|
||||
- from: /auto_examples/placement-group.html
|
||||
to: /ray-core/scheduling/placement-group.html
|
||||
type: page
|
||||
- from: /namespaces.html
|
||||
to: /ray-core/namespaces.html
|
||||
type: page
|
||||
- from: /using-ray-with-pytorch.html
|
||||
to: /train/examples.html
|
||||
type: page
|
||||
- from: /cross-language.html
|
||||
to: /ray-core/cross-language.html
|
||||
type: page
|
||||
- from: /multiprocessing.html
|
||||
to: /ray-more-libs/multiprocessing.html
|
||||
type: page
|
||||
- from: /starting-ray.html
|
||||
to: /ray-core/starting-ray.html
|
||||
type: page
|
||||
- from: /auto_examples/plot_newsreader.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /auto_examples/plot_streaming.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /whitepaper.html
|
||||
to: /ray-contribute/whitepaper.html
|
||||
type: page
|
||||
- from: /tune/tutorials/tune-tutorial.html
|
||||
to: /tune/getting-started.html
|
||||
type: page
|
||||
- from: /using-ray-with-tensorflow.html
|
||||
to: /train/examples.html
|
||||
type: page
|
||||
- from: /auto_examples/progress_bar.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /configure.html
|
||||
to: /ray-core/configure.html
|
||||
type: page
|
||||
- from: /profiling.html
|
||||
to: /ray-contribute/profiling.html
|
||||
type: page
|
||||
- from: /tune/contrib.html
|
||||
to: /ray-contribute/getting-involved.html
|
||||
type: page
|
||||
- from: /walkthrough.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /rllib-offline.html
|
||||
to: /rllib/rllib-offline.html
|
||||
type: page
|
||||
- from: /package-ref.html
|
||||
to: /ray-core/api/index.html
|
||||
type: page
|
||||
- from: /rllib-dev.html
|
||||
to: /rllib/rllib-dev.html
|
||||
type: page
|
||||
- from: /rllib-algorithms.html
|
||||
to: /rllib/rllib-algorithms.html
|
||||
type: page
|
||||
- from: /ray-libraries.html
|
||||
to: /ray-overview/ray-libraries.html
|
||||
type: page
|
||||
- from: /using-ray-with-jupyter.rst
|
||||
to: /ray-core/using-ray-with-jupyter.html
|
||||
type: page
|
||||
- from: /rllib/package_ref/evaluation/rllib-concepts.html
|
||||
to: /rllib/package_ref/index.html
|
||||
type: page
|
||||
- from: /rllib/package_ref/evaluation/rllib-training.html
|
||||
to: /rllib/package_ref/index.html
|
||||
type: page
|
||||
- from: /tf_distributed_training.html
|
||||
to: /train/train.html
|
||||
type: page
|
||||
- from: /tune/tune-scheduler-hyperband
|
||||
to: /tune/api/schedulers.html#tune-original-hyperband
|
||||
type: page
|
||||
- from: /ray-overview/joblib.html
|
||||
to: /ray-more-libs/joblib.html
|
||||
type: page
|
||||
- from: /ray-overview/rllib/index.html
|
||||
to: /rllib/index.html
|
||||
type: page
|
||||
- from: /ray-overview/tune.html
|
||||
to: /tune/index.html
|
||||
type: page
|
||||
- from: /tune.io
|
||||
to: /tune/index.html
|
||||
type: page
|
||||
- from: /rllib-examples.html
|
||||
to: /rllib/rllib-examples.html
|
||||
type: page
|
||||
- from: /iter.html
|
||||
to: /index.html
|
||||
type: page
|
||||
- from: /ray-core/tf_distributed_training.html
|
||||
to: /train/train.html
|
||||
type: page
|
||||
- from: /rllib-package-ref.html
|
||||
to: /rllib/package_ref/index.html
|
||||
type: page
|
||||
- from: /ray-more-libs/tune.io
|
||||
to: /tune/index.html
|
||||
type: page
|
||||
- from: /ray-core/ray-metrics.html
|
||||
to: /cluster/metrics.html
|
||||
type: page
|
||||
- from: /ray-core/cluster/index.html
|
||||
to: /cluster/getting-started.html
|
||||
type: page
|
||||
- from: /ray-core/using-ray-with-jupyter.rst
|
||||
to: /ray-core/using-ray-with-jupyter.html
|
||||
type: page
|
||||
- from: /dask-on-ray.html
|
||||
to: /ray-more-libs/dask-on-ray.html
|
||||
type: page
|
||||
- from: /mars-on-ray.html
|
||||
to: /ray-more-libs/mars-on-ray.html
|
||||
type: page
|
||||
- from: /cluster/ray-client
|
||||
to: /en/master/cluster/running-applications/job-submission/ray-client.html
|
||||
type: page
|
||||
- from: /handling-dependencies.html#remote-uris
|
||||
to: /ray-core/handling-dependencies.html#remote-uris
|
||||
type: page
|
||||
- from: /handling-dependencies.html
|
||||
to: /ray-core/handling-dependencies.html
|
||||
type: page
|
||||
- from: /ray-core/examples/tips-for-first-time.html
|
||||
to: /ray-core/tips-for-first-time.html
|
||||
type: page
|
||||
- from: /ray-core/examples/placement-group.html
|
||||
to: /ray-core/scheduling/placement-group.html
|
||||
type: page
|
||||
- from: /ray-core/using-ray-with-pytorch.html
|
||||
to: /train/examples.html
|
||||
type: page
|
||||
- from: /ray-core/using-ray-with-tensorflow.html
|
||||
to: /train/examples.html
|
||||
type: page
|
||||
- from: /ray-core/examples/plot_newsreader.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /ray-core/examples/plot_streaming.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /ray-core/examples/progress_bar.html
|
||||
to: /ray-core/walkthrough.html
|
||||
type: page
|
||||
- from: /serve/tutorial.html
|
||||
to: /serve/getting_started.html
|
||||
type: page
|
||||
- from: /ray-design-patterns/map-reduce.html
|
||||
to: /ray-core/patterns/index.html
|
||||
type: page
|
||||
- from: /cluster/launcher.html
|
||||
to: /cluster/cli.html
|
||||
type: page
|
||||
- from: /train/user_guide.html
|
||||
to: /train/user-guides.html
|
||||
type: page
|
||||
- from: /cluster/user-guide.html
|
||||
to: /cluster/running-applications/index.html
|
||||
type: page
|
||||
- from: /serve/core-apis.html
|
||||
to: /serve/api/index.html
|
||||
type: page
|
||||
- from: /cluster/ray-client.html
|
||||
to: /cluster/running-applications/job-submission/ray-client.html
|
||||
type: page
|
||||
- from: /cluster/job-submission.html
|
||||
to: /cluster/running-applications/job-submission/index.html
|
||||
type: page
|
||||
- from: /cluster/quickstart.html
|
||||
to: /cluster/getting-started.html
|
||||
type: page
|
||||
- from: /ray-core/strftime.org
|
||||
to: https://strftime.org
|
||||
type: page
|
||||
- from: /data/package-ref.html
|
||||
to: /data/api/api.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/tree-of-tasks.html
|
||||
to: /ray-core/patterns/nested-tasks.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/limit-tasks.html
|
||||
to: /ray-core/patterns/limit-pending-tasks.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/closure-capture.html
|
||||
to: /ray-core/patterns/closure-capture-large-objects.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/fine-grained-tasks.html
|
||||
to: /ray-core/patterns/too-fine-grained-tasks.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/submission-order.html
|
||||
to: /ray-core/patterns/ray-get-submission-order.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/too-many-results.html
|
||||
to: /ray-core/patterns/ray-get-too-many-objects.html
|
||||
type: page
|
||||
- from: /ray-core/more-topics.html
|
||||
to: /ray-core/advanced-topics.html
|
||||
type: page
|
||||
- from: /ray-core/placement-group.html
|
||||
to: /ray-core/scheduling/placement-group.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/resources.html
|
||||
to: /ray-core/scheduling/resources.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/scheduling.html
|
||||
to: /ray-core/scheduling/index.html
|
||||
type: page
|
||||
- from: /ray-core/actors/scheduling.html
|
||||
to: /ray-core/scheduling/index.html
|
||||
type: page
|
||||
- from: /ray-core/tasks/patterns/*
|
||||
to: /ray-core/patterns/:splat
|
||||
type: page
|
||||
- from: /auto_examples/*
|
||||
to: /ray-core/examples/:splat
|
||||
type: page
|
||||
- from: /rllib/rllib-torch2x.html
|
||||
to: /rllib/new-api-stack-migration-guide.html
|
||||
type: page
|
||||
- from: /rllib/rllib-rlmodule.html
|
||||
to: /rllib/key-concepts.html
|
||||
type: page
|
||||
- from: /rllib/rllib-models.html
|
||||
to: /rllib/rl-modules.html
|
||||
type: page
|
||||
- from: /rllib/rllib-*
|
||||
to: /rllib/index.html
|
||||
type: page
|
||||
- from: /rllib/package_ref/*
|
||||
to: /rllib/package_ref/index.html
|
||||
type: page
|
||||
- from: /serve/api/*
|
||||
to: /serve/api/index.html
|
||||
type: page
|
||||
- from: /data/api/*
|
||||
to: /data/api/api.html
|
||||
type: page
|
||||
- from: /ray-air/examples/*
|
||||
to: /ray-overview/examples.html
|
||||
type: page
|
||||
- from: /ray-air/*
|
||||
to: /ray-air/getting-started.html
|
||||
type: page
|
||||
- from: /tune/examples/*
|
||||
to: /ray-overview/examples.html
|
||||
type: page
|
||||
- from: /tune/api/*
|
||||
to: /tune/api/api.html
|
||||
type: page
|
||||
- from: /train/api/*
|
||||
to: /train/api/api.html
|
||||
type: page
|
||||
- from: /train/examples/*
|
||||
to: /ray-overview/examples.html
|
||||
type: page
|
||||
- from: /train/faq.html
|
||||
to: /train/train.html
|
||||
type: page
|
||||
- from: /workflows/*
|
||||
to: /ray-overview/index.html
|
||||
type: page
|
||||
- from: /_modules/*
|
||||
to: https://github.com/ray-project/ray
|
||||
type: page
|
||||
description: Sphinx source-view pages have no docs equivalent so they land on the Ray source code
|
||||
- from: /cluster/kubernetes/*
|
||||
to: /cluster/kubernetes/index.html
|
||||
type: page
|
||||
- from: /serve/tutorials/*
|
||||
to: /serve/index.html
|
||||
type: page
|
||||
- from: /ray-core/examples/*
|
||||
to: /ray-overview/examples.html
|
||||
type: page
|
||||
- from: /ray-core/api/*
|
||||
to: /ray-core/api/index.html
|
||||
type: page
|
||||
- from: /ray-observability/*
|
||||
to: /ray-observability/index.html
|
||||
type: page
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
../python/deplocks/docs/docbuild_depset_py3.11.lock
|
||||
@@ -0,0 +1,47 @@
|
||||
# Production requirements. This is what readthedocs.com picks up
|
||||
|
||||
# Required to build the docs on 3.12 due to pkg_resources deprecation
|
||||
setuptools==80.9.0
|
||||
|
||||
# Syntax highlighting
|
||||
Pygments==2.18.0
|
||||
|
||||
# Sphinx
|
||||
sphinx==8.2.3
|
||||
sphinx-click==5.1.0
|
||||
sphinx-copybutton==0.5.2
|
||||
sphinxemoji>=0.3.2
|
||||
sphinx-jsonschema==1.19.2
|
||||
sphinx-llms-txt==0.7.1
|
||||
sphinx-sitemap==2.9.0
|
||||
sphinxcontrib-redoc==1.6.0
|
||||
sphinx-remove-toctrees==0.0.3
|
||||
sphinx_design==0.7.0
|
||||
sphinx-autobuild==2024.4.16
|
||||
sphinxext-opengraph==0.13.0
|
||||
pydata-sphinx-theme==0.18.0
|
||||
autodoc_pydantic==2.2.0
|
||||
appnope
|
||||
sphinx-docsearch==0.3.0
|
||||
sphinx-collections==0.3.1
|
||||
|
||||
pydantic==2.12.4
|
||||
|
||||
# MyST
|
||||
myst-parser==5.1.0 # Needed to parse markdown
|
||||
myst-nb==1.4.0
|
||||
# Jupyter conversion
|
||||
jupytext==1.15.2
|
||||
|
||||
# Pin urllib to avoid downstream ssl incompatibility issues
|
||||
urllib3 < 1.27
|
||||
|
||||
# External dependencies such as ML libraries should be mocked out, not added here.
|
||||
# See doc/source/conf.py for examples of how to mock out external dependencies.
|
||||
click==8.1.7
|
||||
boto3==1.34.69
|
||||
requests==2.32.5
|
||||
tensorflow==2.20.0
|
||||
tf-keras==2.20.1
|
||||
xmltodict==0.13.0
|
||||
pyarrow==19.0.1
|
||||
@@ -0,0 +1,276 @@
|
||||
#!/usr/bin/env python
|
||||
"""Preflight check that the local environment matches the Read the Docs build.
|
||||
|
||||
Ray's docs build on Read the Docs with ``fail_on_warning: true``, so a local
|
||||
build is only trustworthy when it reproduces RtD's environment. This doctor
|
||||
compares the active interpreter and installed packages against what RtD uses
|
||||
and reports any drift before ``make rtd`` spends minutes on a build that
|
||||
wouldn't be faithful anyway.
|
||||
|
||||
The expected values are read from the repo, never hardcoded, so this keeps
|
||||
working as versions are tweaked:
|
||||
|
||||
* Python version <- ``.readthedocs.yaml`` ``build.tools.python``
|
||||
* pinned deps <- ``doc/requirements-doc.lock.txt`` (the docs lock)
|
||||
|
||||
This script is intentionally stdlib-only: it diagnoses a possibly-broken or
|
||||
mismatched environment, so it must not depend on the very packages it checks
|
||||
(it uses PyYAML only if it happens to be importable, with a regex fallback).
|
||||
|
||||
Exit status: 0 when faithful (or under ``--warn-only``), 1 on critical drift,
|
||||
2 when the expected values can't be determined (a repo/config problem).
|
||||
"""
|
||||
|
||||
# Annotations are lazy strings so this runs under any interpreter a contributor
|
||||
# might have active, even one older than the docs build's Python.
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import platform
|
||||
import re
|
||||
import sys
|
||||
from importlib import metadata
|
||||
from pathlib import Path
|
||||
|
||||
# Optional: use PyYAML for a structural parse when present, but never depend on
|
||||
# it — this script must run in an environment that's missing the docs deps.
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
yaml = None
|
||||
|
||||
# doc/rtd_doctor.py -> doc/ -> repo root
|
||||
DOC_DIR = Path(__file__).resolve().parent
|
||||
RAY_ROOT = DOC_DIR.parent
|
||||
RTD_YAML = RAY_ROOT / ".readthedocs.yaml"
|
||||
DOC_LOCK = DOC_DIR / "requirements-doc.lock.txt"
|
||||
|
||||
# RtD runs on this OS (informational only; see .readthedocs.yaml build.os).
|
||||
RTD_OS = "ubuntu-24.04"
|
||||
|
||||
# Packages whose pinned version most directly determines build fidelity. The
|
||||
# names are stable; their expected versions come from the lock, not from here.
|
||||
KEY_PACKAGES = ["sphinx", "myst-parser", "myst-nb", "pydata-sphinx-theme"]
|
||||
|
||||
# Status tiers. CRITICAL fails the check (and blocks `make rtd`); WARN and INFO
|
||||
# never do.
|
||||
CRITICAL, WARN, INFO, OK = "critical", "warn", "info", "ok"
|
||||
|
||||
|
||||
class DoctorError(Exception):
|
||||
"""The doctor couldn't determine what to check (a repo/config problem)."""
|
||||
|
||||
|
||||
def _normalize(name: str) -> str:
|
||||
"""Normalize a distribution name per PEP 503 for comparison."""
|
||||
return re.sub(r"[-_.]+", "-", name).lower()
|
||||
|
||||
|
||||
def read_expected_python() -> str:
|
||||
"""Return the Python version RtD builds with, from .readthedocs.yaml."""
|
||||
if not RTD_YAML.is_file():
|
||||
raise DoctorError(f"{RTD_YAML} not found")
|
||||
text = RTD_YAML.read_text()
|
||||
|
||||
# Prefer a structural parse when PyYAML is available; fall back to a
|
||||
# targeted scan so the doctor still runs in a stripped-down environment.
|
||||
if yaml is not None:
|
||||
try:
|
||||
data = yaml.safe_load(text)
|
||||
value = data["build"]["tools"]["python"]
|
||||
return str(value).strip()
|
||||
except (KeyError, TypeError, AttributeError):
|
||||
pass # fall through to the regex scan
|
||||
|
||||
# The first `python:` after `tools:` is build.tools.python; the top-level
|
||||
# `python:` install block appears later in the file.
|
||||
match = re.search(r"tools:.*?\bpython:\s*[\"']?(\d+(?:\.\d+)?)", text, re.DOTALL)
|
||||
if not match:
|
||||
raise DoctorError(f"could not find build.tools.python in {RTD_YAML.name}")
|
||||
return match.group(1)
|
||||
|
||||
|
||||
def read_lock_versions() -> dict[str, str]:
|
||||
"""Return {normalized name: version} parsed from the docs lock."""
|
||||
if not DOC_LOCK.is_file():
|
||||
raise DoctorError(f"{DOC_LOCK} not found")
|
||||
versions = {}
|
||||
# Lock lines look like `sphinx==8.2.3 \`; --hash/# via/--index-url lines
|
||||
# don't start with a package-name character, so anchoring excludes them.
|
||||
pin = re.compile(r"^([A-Za-z0-9][A-Za-z0-9._-]*)==([^\s\\]+)")
|
||||
for line in DOC_LOCK.read_text().splitlines():
|
||||
match = pin.match(line)
|
||||
if match:
|
||||
versions[_normalize(match.group(1))] = match.group(2)
|
||||
if not versions:
|
||||
raise DoctorError(f"no pinned versions parsed from {DOC_LOCK.name}")
|
||||
return versions
|
||||
|
||||
|
||||
def installed_version(name: str) -> str | None:
|
||||
"""Return the installed version of ``name``, or None if absent."""
|
||||
try:
|
||||
return metadata.version(name)
|
||||
except metadata.PackageNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def lock_target_python() -> str | None:
|
||||
"""Best-effort Python tag from the resolved lock filename (e.g. '3.11')."""
|
||||
try:
|
||||
name = DOC_LOCK.resolve().name
|
||||
except OSError:
|
||||
return None
|
||||
match = re.search(r"py(\d+)\.?(\d+)", name)
|
||||
return f"{match.group(1)}.{match.group(2)}" if match else None
|
||||
|
||||
|
||||
def build_findings() -> tuple[list[tuple[str, str, str, str]], str]:
|
||||
"""Run every check, returning (rows, expected_python).
|
||||
|
||||
Each row is (axis, expected, actual, status). Raises DoctorError if the
|
||||
expected values themselves can't be read.
|
||||
"""
|
||||
expected_python = read_expected_python()
|
||||
lock = read_lock_versions()
|
||||
|
||||
rows = []
|
||||
|
||||
actual_python = f"{sys.version_info.major}.{sys.version_info.minor}"
|
||||
rows.append(
|
||||
(
|
||||
"Python",
|
||||
expected_python,
|
||||
actual_python,
|
||||
OK if actual_python == expected_python else CRITICAL,
|
||||
)
|
||||
)
|
||||
|
||||
for pkg in KEY_PACKAGES:
|
||||
expected = lock.get(_normalize(pkg))
|
||||
if expected is None:
|
||||
rows.append((pkg, "(not in lock)", "-", WARN))
|
||||
continue
|
||||
actual = installed_version(pkg)
|
||||
if actual is None:
|
||||
rows.append((pkg, expected, "(not installed)", CRITICAL))
|
||||
else:
|
||||
rows.append((pkg, expected, actual, OK if actual == expected else CRITICAL))
|
||||
|
||||
# Informational / repo-consistency axes — never block.
|
||||
rows.append(("operating system", RTD_OS, platform.platform(), INFO))
|
||||
lock_py = lock_target_python()
|
||||
if lock_py is not None and lock_py != expected_python:
|
||||
rows.append(("docs lock target", f"py{expected_python}", f"py{lock_py}", WARN))
|
||||
|
||||
return rows, expected_python
|
||||
|
||||
|
||||
def render(rows: list[tuple[str, str, str, str]], warn_only: bool, quiet: bool) -> int:
|
||||
"""Print the report and return the process exit code."""
|
||||
criticals = [r for r in rows if r[3] == CRITICAL]
|
||||
warnings = [r for r in rows if r[3] == WARN]
|
||||
|
||||
status_label = {
|
||||
OK: "ok",
|
||||
CRITICAL: "MISMATCH" if warn_only else "MISMATCH (critical)",
|
||||
WARN: "check",
|
||||
INFO: "differs (low risk)",
|
||||
}
|
||||
|
||||
if not quiet:
|
||||
print("Read the Docs build fidelity check")
|
||||
print(f" expected Python from {RTD_YAML.name} (build.tools.python)")
|
||||
print(f" expected deps from {DOC_LOCK.name} (the docs lock)")
|
||||
print()
|
||||
axis_w = max(len(r[0]) for r in rows)
|
||||
exp_w = max(len(str(r[1])) for r in rows)
|
||||
act_w = max(len(str(r[2])) for r in rows)
|
||||
for axis, expected, actual, status in rows:
|
||||
print(
|
||||
f" {axis:<{axis_w}} {str(expected):<{exp_w}} "
|
||||
f"{str(actual):<{act_w}} {status_label[status]}"
|
||||
)
|
||||
print()
|
||||
|
||||
if criticals and not warn_only:
|
||||
print(
|
||||
f"✗ Environment does not match the Read the Docs build "
|
||||
f"({len(criticals)} critical issue(s))."
|
||||
)
|
||||
_print_fixes(criticals)
|
||||
return 1
|
||||
|
||||
if criticals: # warn_only: surface them but don't block
|
||||
print(
|
||||
f"! {len(criticals)} environment mismatch(es) (continuing anyway "
|
||||
f"because --warn-only). The local build may not match RtD."
|
||||
)
|
||||
_print_fixes(criticals)
|
||||
return 0
|
||||
|
||||
if warnings:
|
||||
print(
|
||||
"✓ Environment matches the Read the Docs build "
|
||||
f"({len(warnings)} non-blocking warning(s) above)."
|
||||
)
|
||||
return 0
|
||||
|
||||
print("✓ Environment matches the Read the Docs build.")
|
||||
return 0
|
||||
|
||||
|
||||
def _print_fixes(criticals: list[tuple[str, str, str, str]]) -> None:
|
||||
"""Print actionable remediation for the critical findings."""
|
||||
axes = {r[0] for r in criticals}
|
||||
print(" Fix:")
|
||||
if "Python" in axes:
|
||||
expected = next(r[1] for r in criticals if r[0] == "Python")
|
||||
print(
|
||||
f" - Use Python {expected} (RtD's version). Recreate your env, "
|
||||
f"e.g. `conda create -n docs python={expected}`."
|
||||
)
|
||||
if axes - {"Python"}: # any dependency drift
|
||||
print(
|
||||
" - Reinstall the pinned docs deps without -U, from the doc/ "
|
||||
"directory:"
|
||||
)
|
||||
print(" pip install -r requirements-doc.lock.txt")
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description=(
|
||||
"Check that this environment matches the Read the Docs build "
|
||||
"(Python version and pinned docs dependencies)."
|
||||
)
|
||||
)
|
||||
parser.add_argument(
|
||||
"--warn-only",
|
||||
action="store_true",
|
||||
help="Report drift as warnings and exit 0 instead of failing.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--quiet",
|
||||
action="store_true",
|
||||
help="Print only the verdict and any fixes, not the full table.",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
rows, _ = build_findings()
|
||||
except DoctorError as exc:
|
||||
if args.warn_only:
|
||||
print(
|
||||
f"! Could not run the RtD fidelity check: {exc} "
|
||||
f"(continuing because --warn-only)."
|
||||
)
|
||||
return 0
|
||||
print(f"✗ Could not run the RtD fidelity check: {exc}")
|
||||
return 2
|
||||
|
||||
return render(rows, warn_only=args.warn_only, quiet=args.quiet)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,147 @@
|
||||
"""APIs-tab sidebar (Pattern B): a single shared API navigation, loaded client-side.
|
||||
|
||||
Ray's global sidebar server-renders the whole toctree into *every* page. The symbol-
|
||||
level API nav has ~3k stub pages; rendering those into every page would bloat each one
|
||||
(~250 KB of sidebar) and OOM the build. Instead:
|
||||
|
||||
1. Capture the full ``apis/`` toctree once at ``env-updated``, *before*
|
||||
``sphinx_remove_toctrees`` (priority 500) prunes the stub pages, and render it to
|
||||
a single fragment written to ``_static/api-nav.html`` at ``build-finished`` (the
|
||||
HTML writer isn't ready until then).
|
||||
2. The per-symbol stubs are kept OUT of the global toctree by ``remove_from_toctrees``
|
||||
in conf.py (Ray already does this), so the global ``main-sidebar`` stays small.
|
||||
3. On API pages, swap in a tiny container that loads the shared fragment via
|
||||
``_static/api-nav-loader.js`` and highlights the current page client-side.
|
||||
|
||||
The API reference pages keep their *original* locations (``data/api/``, ``train/api/``,
|
||||
``rllib/package_ref/``, ...); they are pulled into the APIs tab purely by
|
||||
``apis/index``'s toctree, with no URL change. "Is this an API page?" is therefore
|
||||
answered by membership in those known source directories (``API_PATH_PREFIXES``) -- a
|
||||
stateless check, so it works under parallel writing (no reliance on cross-process
|
||||
state).
|
||||
|
||||
So each API page embeds ~no navigation HTML; the nav is one browser-cached file.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
|
||||
import bs4
|
||||
from sphinx.environment.adapters.toctree import global_toctree_for_doc
|
||||
from sphinx.util import logging as sphinx_logging
|
||||
|
||||
# Reuse the in-repo copy rather than importing from ``pydata_sphinx_theme.toctree``:
|
||||
# that symbol isn't part of the theme's public API, so importing it directly makes
|
||||
# the docs build fragile across theme upgrades. See the docstring on the vendored copy.
|
||||
from custom_directives import add_collapse_checkboxes
|
||||
|
||||
logger = sphinx_logging.getLogger(__name__)
|
||||
|
||||
APIS_PREFIX = "apis"
|
||||
NAV_DOCNAME = APIS_PREFIX + "/index"
|
||||
NAV_FILENAME = "api-nav.html"
|
||||
|
||||
# Source directories whose pages make up the APIs tab. These are aggregated by the
|
||||
# toctree in apis/index.md; pages under them get the shared API sidebar. Kept in sync
|
||||
# with that toctree. A page's docname starting with one of these (or being the APIs
|
||||
# landing itself) marks it as API content. Stateless on purpose -- html-page-context
|
||||
# can fire in worker processes under `-j`, where main-process state isn't shared.
|
||||
API_PATH_PREFIXES = (
|
||||
"apis/", # the APIs landing page (apis/index)
|
||||
"data/api/",
|
||||
"train/api/",
|
||||
"tune/api/",
|
||||
"serve/api/",
|
||||
"ray-core/api/",
|
||||
"rllib/package_ref/",
|
||||
)
|
||||
|
||||
# Captured in the main process at env-updated and consumed in the main process at
|
||||
# build-finished (same process), so it is safe under parallel read/write.
|
||||
_state = {}
|
||||
|
||||
|
||||
def _capture_apis_toctree(app, env):
|
||||
"""env-updated @ priority < 500: resolve the full apis/ toctree while the stub
|
||||
pages are still present (sphinx_remove_toctrees prunes them at priority 500)."""
|
||||
try:
|
||||
node = global_toctree_for_doc(
|
||||
env, NAV_DOCNAME, app.builder,
|
||||
collapse=False, maxdepth=6, includehidden=True, titles_only=True,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.warning("[api_sidebar] could not capture apis toctree: %s", exc)
|
||||
return
|
||||
if node is None:
|
||||
logger.warning("[api_sidebar] apis toctree resolved empty (is %s present?)", NAV_DOCNAME)
|
||||
return
|
||||
_state["node"] = node
|
||||
|
||||
|
||||
def _root_relative(html):
|
||||
"""global_toctree_for_doc resolves links relative to apis/index, which lives in the
|
||||
apis/ directory (depth 1). The API docs themselves live at their original locations
|
||||
(data/api/, train/api/, ...), so each link is ``../<path>``. Strip the single
|
||||
leading ``../`` to make hrefs root-relative; the loader then resolves them against
|
||||
each page's URL root."""
|
||||
def repl(m):
|
||||
href = m.group(1)
|
||||
if re.match(r"^(https?:|/|#|mailto:)", href):
|
||||
return m.group(0)
|
||||
if href.startswith("../"):
|
||||
href = href[3:]
|
||||
return 'href="%s"' % href
|
||||
return re.sub(r'href="([^"]*)"', repl, html)
|
||||
|
||||
|
||||
def _render_and_write(app, exc):
|
||||
"""build-finished (main process, writer ready): render the captured global toc,
|
||||
keep ONLY the APIs section, and write it as the shared fragment.
|
||||
|
||||
global_toctree_for_doc returns the whole-site nav (all top-level sections), so we
|
||||
isolate the API subtree here: because the toc was resolved for apis/index, the
|
||||
"APIs" top-level entry is the one marked ``current`` -- we keep just its child
|
||||
list (the libraries and their symbols). Leaving the API section is the top nav's
|
||||
job, so nothing else belongs in this sidebar."""
|
||||
if exc is not None or _state.get("node") is None:
|
||||
return
|
||||
try:
|
||||
html = app.builder.render_partial(_state["node"])["fragment"]
|
||||
except Exception as e:
|
||||
logger.warning("[api_sidebar] could not render apis nav fragment: %s", e)
|
||||
return
|
||||
soup = bs4.BeautifulSoup(html, "html.parser")
|
||||
apis_li = next(
|
||||
(li for li in soup.select("li.toctree-l1") if "current" in (li.get("class") or [])),
|
||||
None,
|
||||
)
|
||||
libs = apis_li.find("ul") if apis_li else None
|
||||
if libs is None:
|
||||
logger.warning("[api_sidebar] could not isolate the APIs section in the toc; "
|
||||
"fragment not written")
|
||||
return
|
||||
libs["class"] = "nav bd-sidenav"
|
||||
# render_partial emits plain nested <ul>; reuse pydata's helper to add the
|
||||
# collapsible <details>/<summary> structure (closed by default) so the loader can
|
||||
# expand the current path and the theme's CSS styles the chevrons natively.
|
||||
frag_soup = bs4.BeautifulSoup(str(libs), "html.parser")
|
||||
add_collapse_checkboxes(frag_soup)
|
||||
html = _root_relative(str(frag_soup))
|
||||
static_dir = os.path.join(app.outdir, "_static")
|
||||
os.makedirs(static_dir, exist_ok=True)
|
||||
with open(os.path.join(static_dir, NAV_FILENAME), "w", encoding="utf-8") as fh:
|
||||
fh.write(html)
|
||||
logger.info("[api_sidebar] wrote shared apis nav fragment (%d KB, API subtree only) "
|
||||
"to _static/%s", len(html) // 1024, NAV_FILENAME)
|
||||
|
||||
|
||||
def _on_html_page_context(app, pagename, templatename, context, doctree):
|
||||
if pagename.startswith(API_PATH_PREFIXES):
|
||||
context["sidebars"] = ["api-sidebar.html"]
|
||||
|
||||
|
||||
def setup(app):
|
||||
# priority < 500 so we capture the toctree BEFORE sphinx_remove_toctrees prunes it.
|
||||
app.connect("env-updated", _capture_apis_toctree, priority=1)
|
||||
app.connect("build-finished", _render_and_write)
|
||||
app.connect("html-page-context", _on_html_page_context, priority=900)
|
||||
return {"version": "1.0", "parallel_read_safe": True, "parallel_write_safe": True}
|
||||
@@ -0,0 +1,199 @@
|
||||
from docutils import nodes
|
||||
|
||||
from sphinx.util.docutils import SphinxDirective
|
||||
from sphinx.transforms import SphinxTransform
|
||||
from docutils.nodes import Node
|
||||
|
||||
# BASE_NUM = 2775 # black circles, white numbers
|
||||
BASE_NUM = 2459 # white circle, black numbers
|
||||
|
||||
|
||||
class CalloutIncludePostTransform(SphinxTransform):
|
||||
"""Code block post-processor for `literalinclude` blocks used in callouts."""
|
||||
|
||||
default_priority = 400
|
||||
|
||||
def apply(self, **kwargs) -> None:
|
||||
visitor = LiteralIncludeVisitor(self.document)
|
||||
self.document.walkabout(visitor)
|
||||
|
||||
|
||||
class LiteralIncludeVisitor(nodes.NodeVisitor):
|
||||
"""Change a literal block upon visiting it."""
|
||||
|
||||
def __init__(self, document: nodes.document) -> None:
|
||||
super().__init__(document)
|
||||
|
||||
def unknown_visit(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def unknown_departure(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def visit_document(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def depart_document(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def visit_start_of_file(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def depart_start_of_file(self, node: Node) -> None:
|
||||
pass
|
||||
|
||||
def visit_literal_block(self, node: nodes.literal_block) -> None:
|
||||
if "<1>" in node.rawsource:
|
||||
source = str(node.rawsource)
|
||||
for i in range(1, 20):
|
||||
source = source.replace(
|
||||
f"<{i}>", chr(int(f"0x{BASE_NUM + i}", base=16))
|
||||
)
|
||||
node.rawsource = source
|
||||
node[:] = [nodes.Text(source)]
|
||||
|
||||
|
||||
class callout(nodes.General, nodes.Element):
|
||||
"""Sphinx callout node."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def visit_callout_node(self, node):
|
||||
"""We pass on node visit to prevent the
|
||||
callout being treated as admonition."""
|
||||
pass
|
||||
|
||||
|
||||
def depart_callout_node(self, node):
|
||||
"""Departing a callout node is a no-op, too."""
|
||||
pass
|
||||
|
||||
|
||||
class annotations(nodes.Element):
|
||||
"""Sphinx annotations node."""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def _replace_numbers(content: str):
|
||||
"""
|
||||
Replaces strings of the form <x> with circled unicode numbers (e.g. ①) as text.
|
||||
|
||||
Args:
|
||||
content: Python str from a callout or annotations directive.
|
||||
|
||||
Returns: The formatted content string.
|
||||
"""
|
||||
for i in range(1, 20):
|
||||
content.replace(f"<{i}>", chr(int(f"0x{BASE_NUM + i}", base=16)))
|
||||
return content
|
||||
|
||||
|
||||
def _parse_recursively(self, node):
|
||||
"""Utility to recursively parse a node from the Sphinx AST."""
|
||||
self.state.nested_parse(self.content, self.content_offset, node)
|
||||
|
||||
|
||||
class CalloutDirective(SphinxDirective):
|
||||
"""Code callout directive with annotations for Sphinx.
|
||||
|
||||
Use this `callout` directive by wrapping either `code-block` or `literalinclude`
|
||||
directives. Each line that's supposed to be equipped with an annotation should
|
||||
have an inline comment of the form "# <x>" where x is an integer.
|
||||
|
||||
Afterwards use the `annotations` directive to add annotations to the previously
|
||||
defined code labels ("<x>") by using the syntax "<x> my annotation" to produce an
|
||||
annotation "my annotation" for x.
|
||||
Note that annotation lines have to be separated by a new line, i.e.
|
||||
|
||||
.. annotations::
|
||||
|
||||
<1> First comment followed by a newline,
|
||||
|
||||
<2> second comment after the newline.
|
||||
|
||||
|
||||
Usage example:
|
||||
-------------
|
||||
|
||||
.. callout::
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from ray import tune
|
||||
from ray.tune.search.hyperopt import HyperOptSearch
|
||||
import keras
|
||||
|
||||
def objective(config): # <1>
|
||||
...
|
||||
|
||||
search_space = {"activation": tune.choice(["relu", "tanh"])} # <2>
|
||||
algo = HyperOptSearch()
|
||||
|
||||
tuner = tune.Tuner( # <3>
|
||||
...
|
||||
)
|
||||
results = tuner.fit()
|
||||
|
||||
.. annotations::
|
||||
|
||||
<1> Wrap a Keras model in an objective function.
|
||||
|
||||
<2> Define a search space and initialize the search algorithm.
|
||||
|
||||
<3> Start a Tune run that maximizes accuracy.
|
||||
"""
|
||||
|
||||
has_content = True
|
||||
|
||||
def run(self):
|
||||
self.assert_has_content()
|
||||
|
||||
content = self.content
|
||||
content = _replace_numbers(content)
|
||||
|
||||
callout_node = callout("\n".join(content))
|
||||
_parse_recursively(self, callout_node)
|
||||
|
||||
return [callout_node]
|
||||
|
||||
|
||||
class AnnotationsDirective(SphinxDirective):
|
||||
"""Annotations directive, which is only used nested within a Callout directive."""
|
||||
|
||||
has_content = True
|
||||
|
||||
def run(self):
|
||||
content = self.content
|
||||
content = _replace_numbers(content)
|
||||
|
||||
joined_content = "\n".join(content)
|
||||
annotations_node = callout(joined_content)
|
||||
_parse_recursively(self, annotations_node)
|
||||
|
||||
return [annotations_node]
|
||||
|
||||
|
||||
def setup(app):
|
||||
# Add new node types
|
||||
app.add_node(
|
||||
callout,
|
||||
html=(visit_callout_node, depart_callout_node),
|
||||
latex=(visit_callout_node, depart_callout_node),
|
||||
text=(visit_callout_node, depart_callout_node),
|
||||
)
|
||||
app.add_node(annotations)
|
||||
|
||||
# Add new directives
|
||||
app.add_directive("callout", CalloutDirective)
|
||||
app.add_directive("annotations", AnnotationsDirective)
|
||||
|
||||
# Add post-processor
|
||||
app.add_post_transform(CalloutIncludePostTransform)
|
||||
|
||||
return {
|
||||
"version": "0.1",
|
||||
"parallel_read_safe": True,
|
||||
"parallel_write_safe": True,
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
from docutils import nodes, utils, frontend
|
||||
from docutils.parsers.rst import directives, Parser
|
||||
from sphinx.util.docutils import SphinxDirective
|
||||
|
||||
|
||||
class URLQueryParamRefNode(nodes.General, nodes.Element):
|
||||
"""Node type for references that need URL query parameters."""
|
||||
|
||||
|
||||
class WrapperNode(nodes.paragraph):
|
||||
"""A wrapper node for URL query param references.
|
||||
|
||||
Sphinx will not build if you don't wrap a reference in a paragraph. And a <div>
|
||||
is needed anyway for styling purposes.
|
||||
"""
|
||||
|
||||
def visit(self, node):
|
||||
if "class" not in node:
|
||||
raise ValueError(
|
||||
"'class' attribute must be set on the WrapperNode for a "
|
||||
"URL query parameter reference."
|
||||
)
|
||||
self.body.append(self.starttag(node, "div", CLASS=node["class"]))
|
||||
|
||||
def depart(self, node):
|
||||
self.body.append("</div>")
|
||||
|
||||
|
||||
class URLQueryParamRefDirective(SphinxDirective):
|
||||
"""Sphinx directive to insert a reference with query parameters."""
|
||||
|
||||
required_arguments = 1
|
||||
optional_arguments = 0
|
||||
final_argument_whitespace = True
|
||||
has_content = True
|
||||
option_spec = {
|
||||
"parameters": directives.unchanged_required,
|
||||
"classes": directives.class_option,
|
||||
"ref-type": (
|
||||
lambda choice: directives.choice(choice, ["any", "ref", "doc", "myst"])
|
||||
),
|
||||
}
|
||||
|
||||
def run(self):
|
||||
ref_type = self.options.get("ref-type", "any")
|
||||
content = self.content.data
|
||||
reftarget = directives.uri(self.arguments[0])
|
||||
return [
|
||||
URLQueryParamRefNode(
|
||||
{
|
||||
"docname": self.env.docname,
|
||||
"parameters": self.options.get("parameters", None),
|
||||
"classes": self.options.get("classes", []),
|
||||
"reftarget": reftarget,
|
||||
"refdocname": self.env.docname,
|
||||
"refdomain": "std" if ref_type in {"ref", "doc"} else "",
|
||||
"reftype": ref_type,
|
||||
"refexplicit": content if content else reftarget,
|
||||
"refwarn": True,
|
||||
}
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def on_doctree_resolved(app, doctree, docname):
|
||||
"""Replace URLQueryParamRefNode instances with real references.
|
||||
|
||||
Any text that lives inside a URLQueryParamRefNode is parsed as usual.
|
||||
|
||||
Args:
|
||||
app: Sphinx application
|
||||
doctree: Doctree which has just been resolved
|
||||
docname: Name of the document containing the reference nodes
|
||||
"""
|
||||
parser = Parser()
|
||||
for node in doctree.traverse(URLQueryParamRefNode):
|
||||
tmp_node = utils.new_document(
|
||||
"Content nested under URLQueryParamRefNode",
|
||||
frontend.OptionParser(components=[Parser]).get_default_values(),
|
||||
)
|
||||
text = "\n".join(node.rawsource["refexplicit"])
|
||||
|
||||
# Parse all child RST as usual, then append any parsed nodes to the
|
||||
# reference node.
|
||||
parser.parse(text, tmp_node)
|
||||
ref_node = nodes.reference(
|
||||
rawsource=text,
|
||||
text="",
|
||||
)
|
||||
for child in tmp_node.children:
|
||||
ref_node.append(child)
|
||||
|
||||
# Pass all URLQueryParamRefNode attributes to ref node;
|
||||
# possibly not necessary
|
||||
for key, value in node.rawsource.items():
|
||||
ref_node[key] = value
|
||||
|
||||
# Need to update refuri of ref node to include URL query parameters
|
||||
ref_node["refuri"] = (
|
||||
app.builder.get_relative_uri(
|
||||
docname,
|
||||
node.rawsource["reftarget"],
|
||||
)
|
||||
+ node.rawsource["parameters"]
|
||||
)
|
||||
|
||||
# Need to wrap the node in a paragraph for Sphinx to build
|
||||
wrapper = WrapperNode()
|
||||
wrapper["class"] = "query-param-ref-wrapper"
|
||||
wrapper += ref_node
|
||||
node.replace_self([wrapper])
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_directive("query-param-ref", URLQueryParamRefDirective)
|
||||
app.connect("doctree-resolved", on_doctree_resolved)
|
||||
app.add_node(WrapperNode, html=(WrapperNode.visit, WrapperNode.depart))
|
||||
return {
|
||||
"version": "0.1",
|
||||
"parallel_read_safe": True,
|
||||
"parallel_write_safe": True,
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
You can post questions or issues or feedback through the following channels:
|
||||
|
||||
1. `Discussion Board`_: For **questions about Ray usage** or **feature requests**.
|
||||
2. `GitHub Issues`_: For **bug reports**.
|
||||
3. `Ray Slack`_: For **getting in touch** with Ray maintainers.
|
||||
4. `StackOverflow`_: Use the [ray] tag for **questions about Ray**.
|
||||
|
||||
.. _`Discussion Board`: https://discuss.ray.io/
|
||||
.. _`GitHub Issues`: https://github.com/ray-project/ray/issues
|
||||
.. _`Ray Slack`: https://www.ray.io/join-slack
|
||||
.. _`StackOverflow`: https://stackoverflow.com/questions/tagged/ray
|
||||
@@ -0,0 +1,6 @@
|
||||
.. admonition:: Check your version!
|
||||
|
||||
Things can change quickly, and so does this contributor guide.
|
||||
To make sure you've got the most cutting edge version of this guide,
|
||||
go check out the
|
||||
`latest version <https://docs.ray.io/en/master/ray-contribute/getting-involved.html>`__.
|
||||
@@ -0,0 +1,8 @@
|
||||
..
|
||||
.. note::
|
||||
|
||||
Ray 2.40 uses RLlib's new API stack by default.
|
||||
The Ray team has mostly completed transitioning algorithms, example scripts, and
|
||||
documentation to the new code base.
|
||||
|
||||
If you're still using the old API stack, see :doc:`New API stack migration guide </rllib/new-api-stack-migration-guide>` for details on how to migrate.
|
||||
@@ -0,0 +1,79 @@
|
||||
/* APIs-tab sidebar loader (Pattern B).
|
||||
*
|
||||
* Fetches the single shared API-nav fragment (_static/api-nav.html), injects it into
|
||||
* the #api-nav-mount container, highlights the current page, collapses non-current
|
||||
* sections, and resolves the fragment's root-relative links against this page.
|
||||
* The fragment is fetched once and reused from the browser cache across pages.
|
||||
*/
|
||||
(function () {
|
||||
"use strict";
|
||||
|
||||
function init() {
|
||||
var mount = document.getElementById("api-nav-mount");
|
||||
if (!mount) return;
|
||||
|
||||
var navUrl = mount.getAttribute("data-api-nav-url");
|
||||
var pagename = mount.getAttribute("data-pagename") || "";
|
||||
// Path from this page back to the doc root (e.g. "../../"), derived from the
|
||||
// fetch URL so we don't depend on other globals.
|
||||
var root = navUrl.replace(/_static\/api-nav\.html(\?.*)?$/, "");
|
||||
|
||||
fetch(navUrl)
|
||||
.then(function (resp) {
|
||||
if (!resp.ok) throw new Error("HTTP " + resp.status);
|
||||
return resp.text();
|
||||
})
|
||||
.then(function (html) {
|
||||
mount.innerHTML = html;
|
||||
|
||||
var links = mount.querySelectorAll("a[href]");
|
||||
|
||||
// 1) Find + mark the current page (fragment hrefs are root-relative).
|
||||
var currentHref = pagename + ".html";
|
||||
var currentLink = null;
|
||||
links.forEach(function (a) {
|
||||
if (a.getAttribute("href") === currentHref) currentLink = a;
|
||||
});
|
||||
|
||||
// 2) Collapse every section, then re-open only the current page's ancestors.
|
||||
mount.querySelectorAll("details").forEach(function (d) {
|
||||
d.removeAttribute("open");
|
||||
});
|
||||
if (currentLink) {
|
||||
currentLink.classList.add("current");
|
||||
var li = currentLink.closest("li");
|
||||
if (li) li.classList.add("current", "active");
|
||||
var el = currentLink.parentElement;
|
||||
while (el && el !== mount) {
|
||||
if (el.tagName === "DETAILS") el.setAttribute("open", "");
|
||||
el = el.parentElement;
|
||||
}
|
||||
if (li) {
|
||||
var own = li.querySelector(":scope > details");
|
||||
if (own) own.setAttribute("open", "");
|
||||
}
|
||||
if (li && li.scrollIntoView) {
|
||||
li.scrollIntoView({ block: "nearest" });
|
||||
}
|
||||
}
|
||||
|
||||
// 3) Resolve root-relative hrefs against this page.
|
||||
links.forEach(function (a) {
|
||||
var h = a.getAttribute("href");
|
||||
if (h && !/^(https?:|\/|#|mailto:)/.test(h)) {
|
||||
a.setAttribute("href", root + h);
|
||||
}
|
||||
});
|
||||
})
|
||||
.catch(function () {
|
||||
mount.innerHTML =
|
||||
'<p class="api-nav-status">API navigation failed to load.</p>';
|
||||
});
|
||||
}
|
||||
|
||||
if (document.readyState === "loading") {
|
||||
document.addEventListener("DOMContentLoaded", init);
|
||||
} else {
|
||||
init();
|
||||
}
|
||||
})();
|
||||
@@ -0,0 +1,177 @@
|
||||
/* Kapa Ask AI button */
|
||||
#kapa-widget-container figure {
|
||||
padding: 0 !important;
|
||||
}
|
||||
|
||||
.mantine-Modal-root figure {
|
||||
padding: 0 !important;
|
||||
}
|
||||
|
||||
.assistant-title {
|
||||
margin-top: 1em;
|
||||
}
|
||||
|
||||
.container-xl.blurred {
|
||||
filter: blur(5px);
|
||||
}
|
||||
|
||||
.chat-widget {
|
||||
position: fixed;
|
||||
bottom: 10px;
|
||||
right: 10px;
|
||||
z-index: 1000;
|
||||
}
|
||||
|
||||
@keyframes jump {
|
||||
0% {
|
||||
transform: scale(1);
|
||||
box-shadow: 0 1px 2px rgba(0,0,0,.15);
|
||||
}
|
||||
100% {
|
||||
transform: scale(1.05);
|
||||
box-shadow: 0 4px 20px rgba(0,0,0,.1);
|
||||
}
|
||||
}
|
||||
.search-button__wrapper.show ~ .chat-widget {
|
||||
z-index: 1050;
|
||||
animation: .4s jump ease infinite alternate;
|
||||
filter: brightness(1.5);
|
||||
}
|
||||
|
||||
.chat-popup {
|
||||
display: none;
|
||||
position: fixed;
|
||||
top: 20%;
|
||||
left: 50%;
|
||||
transform: translate(-50%, -20%);
|
||||
width: 55%;
|
||||
height: 75%;
|
||||
background-color: var(--pst-color-background);
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 10px;
|
||||
box-shadow: 0 5px 10px var(--pst-color-shadow);
|
||||
z-index: 1032;
|
||||
max-height: 1000px;
|
||||
overflow: hidden;
|
||||
padding-bottom: 40px;
|
||||
}
|
||||
|
||||
.chatFooter {
|
||||
position: absolute;
|
||||
bottom: 0;
|
||||
right: 0;
|
||||
width: 100%;
|
||||
background-color: var(--pst-color-surface);
|
||||
}
|
||||
|
||||
#openChatBtn {
|
||||
border: 1px solid var(--pst-color-border);
|
||||
background-color: var(--pst-color-surface);
|
||||
color: var(--pst-color-text-base);
|
||||
width: 70px;
|
||||
height: 70px;
|
||||
border-radius: 10px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
#closeChatBtn {
|
||||
border: none;
|
||||
background-color: transparent;
|
||||
color: var(--pst-color-text-base);
|
||||
font-size: 1.2em;
|
||||
}
|
||||
|
||||
#closeChatBtn:hover {
|
||||
color: var(--pst-color-link-hover);
|
||||
}
|
||||
|
||||
#searchBar {
|
||||
border: 1px solid var(--pst-color-border);
|
||||
background-color: var(--pst-color-surface);
|
||||
margin: 0em 1em 0em 0em;
|
||||
border-radius: 4px;
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
.chatHeader {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: .5rem;
|
||||
}
|
||||
|
||||
.chatHeader > .header-wrapper {
|
||||
text-align: center;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.chatContentContainer {
|
||||
padding: 15px;
|
||||
max-height: calc(100% - 100px);
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.chatContentContainer input {
|
||||
margin-top: 10px;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
hr {
|
||||
border: none;
|
||||
height: 1px;
|
||||
/* Set the hr color */
|
||||
color: #333; /* old IE */
|
||||
background-color: #333; /* Modern Browsers */
|
||||
}
|
||||
|
||||
#result * {
|
||||
overflow-anchor: none;
|
||||
}
|
||||
|
||||
#anchor {
|
||||
overflow-anchor: auto;
|
||||
height: 1px;
|
||||
}
|
||||
|
||||
#result {
|
||||
padding: 15px;
|
||||
border-radius: 10px;
|
||||
margin-top: 10px;
|
||||
margin-bottom: 10px;
|
||||
background-color: var(--pst-color-surface);
|
||||
max-height: 50vh; /* Ensure the result area does not take too much vertical space */
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.chatContentContainer textarea {
|
||||
flex-grow: 1;
|
||||
min-width: 50px;
|
||||
max-height: 40px;
|
||||
resize: none;
|
||||
}
|
||||
|
||||
#searchBtn {
|
||||
white-space: nowrap;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.input-group {
|
||||
display: flex;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
#blurDiv {
|
||||
width: 100vw;
|
||||
height: 100vh;
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
backdrop-filter: blur(5px);
|
||||
z-index: 1031;
|
||||
}
|
||||
|
||||
#blurDiv.blurDiv-hidden {
|
||||
display: none !important;
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
/* CSAT widgets */
|
||||
#csat-inputs {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.csat-hidden {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
#csat-feedback-label {
|
||||
color: var(--pst-color-text-base);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.csat-button {
|
||||
margin-left: 16px;
|
||||
padding: 8px 16px 8px 16px;
|
||||
border-radius: 4px;
|
||||
border: 1px solid var(--pst-color-border);
|
||||
background: var(--pst-color-background);
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
cursor: pointer;
|
||||
width: 85px;
|
||||
}
|
||||
|
||||
#csat-textarea-group {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
#csat-submit {
|
||||
margin-left: auto;
|
||||
font-weight: 700;
|
||||
border: none;
|
||||
margin-top: 12px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#csat-feedback-received {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.csat-button-active {
|
||||
border: 1px solid var(--pst-color-border);
|
||||
}
|
||||
|
||||
.csat-icon {
|
||||
margin-right: 4px;
|
||||
}
|
||||
|
||||
#csat {
|
||||
padding: 1em;
|
||||
min-width: 60%;
|
||||
}
|
||||
|
||||
#csat-textarea {
|
||||
resize: none;
|
||||
background-color: var(--pst-color-on-background);
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 4px;
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
#csat-textarea::placeholder {
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
.csat-icon > path {
|
||||
fill: var(--pst-color-text-base);
|
||||
}
|
||||
@@ -0,0 +1,451 @@
|
||||
/* Override default colors used in the Sphinx theme. See
|
||||
* https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/styling.html#css-theme-variables
|
||||
* for more information. `important!` is needed below to override
|
||||
* dark/light theme specific values, which normally take precedence over the PST defaults.
|
||||
* */
|
||||
html {
|
||||
--anyscale-blue: #0066FF;
|
||||
--ray-blue: #02A0CF; /* Ray blue color - use this for all ray branding */
|
||||
--pst-color-primary: var(--ray-blue) !important;
|
||||
--pst-color-inline-code-links: var(--ray-blue) !important;
|
||||
|
||||
/* Transparent highlight color; default yellow is hard on the eyes */
|
||||
--pst-color-target: #ffffff00 !important;
|
||||
--color-diff-delete-bg: rgba(212, 118, 22, 0.3);
|
||||
--color-diff-insert-bg: rgba(56, 139, 253, 0.3);
|
||||
--color-diff-nochange-bg: rgba(0, 0, 0, 0);
|
||||
|
||||
--pst-font-family-base: 'Inter', sans-serif;
|
||||
--stata-dark-background: #232629;
|
||||
}
|
||||
|
||||
html[data-theme='dark'] {
|
||||
--pst-color-background: #161a1d;
|
||||
--pst-color-on-background: #1d2125;
|
||||
--pst-color-text-base: #f1f2f4;
|
||||
--pst-color-text-muted: #b3b9c4;
|
||||
--pst-color-border: #2c333a;
|
||||
--bs-body-color: #f1f2f4;
|
||||
--heading-color: #ffffff;
|
||||
--base-pygments-code-color: #cccccc;
|
||||
--pst-color-link-hover: #cce0ff;
|
||||
--anyscale-border-color: #f1f2f4;
|
||||
}
|
||||
|
||||
html[data-theme='light'] {
|
||||
--pst-color-background: #ffffff;
|
||||
--pst-color-on-background: #ffffff;
|
||||
--pst-color-text-base: #22272b;
|
||||
--pst-color-text-muted: #454f59;
|
||||
--pst-color-border: #dcdfe4;
|
||||
--heading-color: #161a1d;
|
||||
--base-pygments-code-color: #cccccc;
|
||||
--pst-color-link-hover: #09326c;
|
||||
--anyscale-border-color: #161a1d;
|
||||
}
|
||||
|
||||
nav.bd-links li > a:hover {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
a:hover {
|
||||
text-decoration-thickness: unset;
|
||||
}
|
||||
|
||||
h1,
|
||||
h2,
|
||||
h3,
|
||||
h4,
|
||||
h5,
|
||||
h6 {
|
||||
color: var(--heading-color);
|
||||
}
|
||||
|
||||
/* Gradient ellipse background */
|
||||
.bd-sidebar-secondary {
|
||||
background-color: transparent;
|
||||
}
|
||||
.bd-content:after {
|
||||
/* Commenting the code below to make more modifications after 2.10 release */
|
||||
/* background: linear-gradient(
|
||||
60deg,
|
||||
rgba(0, 85, 204, 0.18) 14%,
|
||||
rgba(110, 93, 198, 0.18) 49.2%,
|
||||
rgba(174, 71, 135, 0.18) 81.54%
|
||||
); */
|
||||
background-size: 746px 746px;
|
||||
background-repeat: no-repeat;
|
||||
background-position: center;
|
||||
border-radius: 373px;
|
||||
background-origin: 50%;
|
||||
background-attachment: scroll;
|
||||
content: '';
|
||||
transform: translate(50%, 0%);
|
||||
width: 746px;
|
||||
height: 746px;
|
||||
position: absolute;
|
||||
filter: blur(100px);
|
||||
z-index: -1;
|
||||
}
|
||||
|
||||
/* Pygments diff code cell line colors; match github colorblind theme */
|
||||
div.highlight > pre > span.gd {
|
||||
background-color: var(--color-diff-delete-bg);
|
||||
}
|
||||
div.highlight > pre > span.gi {
|
||||
background-color: var(--color-diff-insert-bg);
|
||||
}
|
||||
div.highlight > pre > span.w {
|
||||
background-color: var(--color-diff-nochange-bg);
|
||||
}
|
||||
/* Fix some pygments styles that inadvertently get overridden by PST */
|
||||
.highlight pre {
|
||||
background-color: var(--stata-dark-background);
|
||||
color: var(--base-pygments-code-color);
|
||||
}
|
||||
|
||||
/* Make the article content take up all available space */
|
||||
.bd-main .bd-content .bd-article-container {
|
||||
max-width: 100%; /* default is 60em */
|
||||
}
|
||||
.bd-page-width {
|
||||
max-width: 100%; /* default is 88rem */
|
||||
}
|
||||
|
||||
/* Hide the "Hide Search Matches" button (we aren't highlighting search terms anyway) */
|
||||
#searchbox {
|
||||
display: none;
|
||||
}
|
||||
|
||||
/* Top navbar styling */
|
||||
.navbar-toplevel p {
|
||||
margin: 0;
|
||||
padding-inline-start: 0;
|
||||
}
|
||||
.ref-container > p {
|
||||
height: 100%;
|
||||
}
|
||||
div.navbar-dropdown {
|
||||
display: none;
|
||||
position: relative;
|
||||
left: -50%;
|
||||
color: var(--pst-color-text-muted);
|
||||
}
|
||||
span.navbar-link-title {
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
.navbar-sublevel p a.reference {
|
||||
text-decoration: none;
|
||||
color: var(--pst-color-text-muted);
|
||||
}
|
||||
.navbar-sublevel p a.reference:hover > span.navbar-link-title {
|
||||
text-decoration: underline;
|
||||
color: var(--pst-color-link-hover);
|
||||
}
|
||||
.navbar-toplevel li {
|
||||
display: inline-flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
height: 100%;
|
||||
padding: 0em 1em;
|
||||
}
|
||||
ul.navbar-toplevel li:hover > div.navbar-dropdown {
|
||||
display: block;
|
||||
}
|
||||
ul.navbar-toplevel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
margin: 0;
|
||||
height: 100%;
|
||||
padding-left: 0;
|
||||
}
|
||||
.navbar-content ul.navbar-sublevel {
|
||||
position: absolute;
|
||||
background: var(--pst-color-on-background);
|
||||
white-space: pre;
|
||||
padding: 0em 1em;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: baseline;
|
||||
box-shadow: 0 5px 15px 0 rgb(0 0 0 / 10%);
|
||||
}
|
||||
div.navbar-content a {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: start;
|
||||
white-space: pre;
|
||||
justify-content: center;
|
||||
/* pydata-sphinx-theme 0.14's `.navbar-nav li a { height: 100% }` was
|
||||
scoped to `ul.navbar-nav li a` in 0.17, which no longer matches our
|
||||
`<nav class="navbar-nav">`. Without height: 100% on <a>, the <a>
|
||||
shrinks to text height (~26px) inside its 64px-tall <p> parent,
|
||||
leaving the link text top-aligned in the navbar and the chevron
|
||||
(which is centered on the .ref-container) visually below the text.
|
||||
Restoring height: 100% lets justify-content: center vertically
|
||||
center the text in the full-height anchor. */
|
||||
height: 100%;
|
||||
}
|
||||
div.navbar-content {
|
||||
height: 100%;
|
||||
}
|
||||
nav.navbar-nav {
|
||||
height: 100%;
|
||||
}
|
||||
.ref-container {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
gap: 0.5em;
|
||||
height: 100%;
|
||||
}
|
||||
/* Highlight active nav bar link */
|
||||
li.active-link {
|
||||
font-weight: bold;
|
||||
}
|
||||
.navbar-header-items__end {
|
||||
/* Prevent the anyscale button from wrapping */
|
||||
flex-flow: nowrap !important;
|
||||
}
|
||||
.navbar {
|
||||
box-shadow: 0px 4px 10px 0px rgba(0, 0, 0, 0.08);
|
||||
}
|
||||
/* Set the first .navbar-persistent--mobile element to have auto left margin */
|
||||
.navbar-persistent--mobile {
|
||||
margin-left: auto;
|
||||
}
|
||||
/* Set any .navbar-persistent--mobile preceeded by a .navbar-persistent--mobile to have */
|
||||
/* a 1em left margin */
|
||||
.navbar-persistent--mobile ~ .navbar-persistent--mobile {
|
||||
margin-left: 1em;
|
||||
}
|
||||
/* Disable underline for hovered links in the nav bar */
|
||||
.navbar-nav li a:hover {
|
||||
text-decoration: none;
|
||||
}
|
||||
/* pydata-sphinx-theme 0.17 stopped removing text-decoration on
|
||||
<a class="reference internal"> inside the top navbar (the rule it
|
||||
ships now only matches <a class="nav-link">). Re-add it for our
|
||||
reference-internal anchors. */
|
||||
.navbar-toplevel a.reference {
|
||||
text-decoration: none;
|
||||
}
|
||||
/* pydata-sphinx-theme 0.17 stopped hiding the sidebar-header-items__title
|
||||
element (Ray's navbar-links.html template emits "Site Navigation" as
|
||||
a screen-reader heading). Hide it in the header context — it still
|
||||
renders in the actual sidebar drawer where the heading is useful. */
|
||||
.navbar-header-items .sidebar-header-items__title {
|
||||
display: none;
|
||||
}
|
||||
.navbar-header-items {
|
||||
padding-left: 0;
|
||||
}
|
||||
|
||||
/* Ray logo */
|
||||
.navbar-brand.logo > svg {
|
||||
width: 120px;
|
||||
}
|
||||
.navbar-brand.logo > svg path#ray-text {
|
||||
fill: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
/* Anyscale branding */
|
||||
#try-anyscale-text {
|
||||
color: var(--pst-color-text-base);
|
||||
border-radius: 2px;
|
||||
white-space: nowrap;
|
||||
padding: 0px 12px;
|
||||
height: 40px;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
gap: 1em;
|
||||
border: 1px solid var(--anyscale-border-color);
|
||||
}
|
||||
#try-anyscale-href {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
/* Remove margin for the release header in the sidebar, otherwise it's too much space */
|
||||
#release-header {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* Center the Ray release header in the sidebar */
|
||||
div.sidebar-primary-item:nth-child(1) {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
/* Center the search button in the sidebar */
|
||||
div.sidebar-primary-item:nth-child(2) {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
/* Disable the "Back to top" button that appears if you scroll down */
|
||||
button#pst-back-to-top {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
.bottom-right-promo-banner {
|
||||
position: fixed;
|
||||
bottom: 100px;
|
||||
right: 20px;
|
||||
width: 270px;
|
||||
}
|
||||
|
||||
@media (max-width: 1500px) {
|
||||
.bottom-right-promo-banner {
|
||||
display: none;
|
||||
}
|
||||
}
|
||||
|
||||
/* Nav sidebar styles */
|
||||
.bd-sidebar-primary {
|
||||
width: 280px;
|
||||
padding: 2em 2em 0em 2em;
|
||||
}
|
||||
/* Make sidebar take up full primary sidebar gutter, but don't wrap content */
|
||||
#main-sidebar {
|
||||
width: 100%;
|
||||
}
|
||||
nav.bd-links li > a {
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
/* Sidebar checkboxes are toggled by clicking on the label; hide actual checkboxes */
|
||||
.toctree-checkbox[type='checkbox']:checked ~ ul > li.current-page:before {
|
||||
background-color: var(--ray-blue);
|
||||
border-radius: 0.5px;
|
||||
}
|
||||
.toctree-checkbox[type='checkbox']:checked ~ ul > li:before {
|
||||
content: '';
|
||||
width: 1px;
|
||||
height: 100%;
|
||||
position: absolute;
|
||||
background-color: var(--pst-color-border);
|
||||
}
|
||||
/* Highlight and bold the primary sidebar entry for the current page */
|
||||
#main-sidebar li.current-page > a {
|
||||
color: var(--ray-blue) !important;
|
||||
font-weight: 600;
|
||||
}
|
||||
/* Bold the top level primary sidebar links */
|
||||
#main-sidebar > .navbar-nav > .nav.bd-sidenav > li > a {
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
/* Fix some spacing issues associated with competition with PST styles */
|
||||
.sidebar-content dl {
|
||||
margin-bottom: 0;
|
||||
}
|
||||
.sidebar-content ol li > p:first-child,
|
||||
ul li > p:first-child {
|
||||
margin-top: 0 !important;
|
||||
}
|
||||
|
||||
/* Set autosummary API docs to have fixed two-col format, with alternating different background
|
||||
* on rows */
|
||||
table.autosummary {
|
||||
table-layout: fixed;
|
||||
}
|
||||
table.autosummary .row-odd {
|
||||
background-color: var(--pst-color-surface);
|
||||
}
|
||||
|
||||
/* Ensure that long function names get elided and show ellipses to not overflow their bounding boxes */
|
||||
table.autosummary tr > td:first-child > p > a > code {
|
||||
max-width: 100%;
|
||||
width: fit-content;
|
||||
display: block;
|
||||
}
|
||||
table.autosummary tr > td:first-child > p > a > code > span {
|
||||
display: block;
|
||||
overflow: clip;
|
||||
text-overflow: ellipsis;
|
||||
}
|
||||
|
||||
/* RTD footer container makes the parent */
|
||||
/* #main-sidebar always scrollable if you don't remove negative margin. */
|
||||
/* Restrict width to 30% of the window */
|
||||
.bd-sidebar-primary div#rtd-footer-container {
|
||||
margin: unset;
|
||||
max-width: 30vw;
|
||||
}
|
||||
|
||||
.query-param-ref-wrapper {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 8px;
|
||||
}
|
||||
|
||||
.query-param-ref-wrapper p {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
/* Styles for tables in example pages */
|
||||
.table.example-table {
|
||||
table-layout: fixed;
|
||||
}
|
||||
.table.example-table th:first-child {
|
||||
width: 30%;
|
||||
}
|
||||
|
||||
/* pydata-sphinx-theme 0.14 painted the announcement banner background via
|
||||
an absolutely-positioned `::after` pseudo-element. 0.17 replaced that
|
||||
with `background-color: var(--pst-color-secondary-bg)` directly on
|
||||
`.bd-header-announcement`, so the `::after` rule is now a no-op and
|
||||
pydata's default secondary-bg paints the banner lavender. Set the
|
||||
background directly on the element to keep Ray's teal. */
|
||||
.bd-header-announcement {
|
||||
color: var(--pst-color-light);
|
||||
background-color: var(--ray-blue);
|
||||
}
|
||||
|
||||
.bd-header-announcement a {
|
||||
color: var(--pst-color-light);
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Prevent the PyData theme Version Switcher from getting too large */
|
||||
.version-switcher__menu {
|
||||
max-height: 40rem;
|
||||
overflow-y: scroll;
|
||||
}
|
||||
|
||||
/* Right align the version switcher dropdown menu to prevent it from going off screen */
|
||||
.version-switcher__menu[data-bs-popper] {
|
||||
right: 0;
|
||||
left: unset;
|
||||
}
|
||||
|
||||
/* Hide the RTD version switcher since we are using PyData theme one */
|
||||
readthedocs-flyout {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
/* Styling the experimental Anyscale upsell CTA */
|
||||
.anyscale-cta {
|
||||
margin-bottom: 16px;
|
||||
}
|
||||
|
||||
|
||||
/* Prevent text wrapping around left-aligned images on ultra-wide screens */
|
||||
@media (min-width: 1600px) {
|
||||
.bd-content .align-left,
|
||||
.bd-content .figure.align-left,
|
||||
.bd-content img.align-left {
|
||||
float: none !important;
|
||||
display: block;
|
||||
clear: both;
|
||||
margin-left: 0 !important;
|
||||
margin-right: 0 !important;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
#close-banner {
|
||||
background: none;
|
||||
border: none;
|
||||
color: inherit;
|
||||
font-size: 1.2em;
|
||||
cursor: pointer;
|
||||
padding: 0 5px;
|
||||
border-radius: 3px;
|
||||
transition: background-color 0.2s ease;
|
||||
position: absolute;
|
||||
top: 8px;
|
||||
right: 12px;
|
||||
}
|
||||
@@ -0,0 +1,259 @@
|
||||
html {
|
||||
--remix-icon-color: #ffffff;
|
||||
}
|
||||
|
||||
html[data-theme='dark'] {
|
||||
--example-color: color-mix(in srgb, #ffffff, transparent 90%);
|
||||
}
|
||||
|
||||
html[data-theme='light'] {
|
||||
--example-color: #ffffff;
|
||||
}
|
||||
|
||||
#examples-search-input-label {
|
||||
height: 100%;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
#examples-search-icon {
|
||||
fill: var(--pst-color-text-base);
|
||||
margin: 0em 1em;
|
||||
}
|
||||
|
||||
#examples-search-input {
|
||||
width: 100%;
|
||||
color: var(--pst-color-text-base);
|
||||
border: none;
|
||||
background-color: transparent;
|
||||
}
|
||||
|
||||
#examples-search-input:focus-visible {
|
||||
outline: none;
|
||||
}
|
||||
|
||||
#examples-search-input::placeholder {
|
||||
color: var(--pst-color-text-muted);
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
#no-matches {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
#no-matches-inner-content {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
#no-matches.hidden,
|
||||
.gallery-item.hidden {
|
||||
display: none !important;
|
||||
}
|
||||
|
||||
.examples-search-area {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
padding: 0.75em 0em;
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 4px;
|
||||
background: var(--pst-color-background);
|
||||
}
|
||||
|
||||
.content {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
width: 70%;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.content-wrapper {
|
||||
width: 100%;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.example-tags {
|
||||
color: var(--pst-color-text-muted);
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.example-list-area {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr;
|
||||
gap: 2em;
|
||||
}
|
||||
|
||||
@media (min-width: 960px) {
|
||||
.example-list-area {
|
||||
grid-template-columns: 1fr 1fr;
|
||||
}
|
||||
}
|
||||
|
||||
.example-text-area {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
#dropdown-area {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 2em;
|
||||
margin: 1.5em 0em 2.5em 0em;
|
||||
}
|
||||
|
||||
.filter-dropdown {
|
||||
border: 1px solid var(--dropdown-border-color);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.dropdown-checkbox {
|
||||
display: none;
|
||||
padding: 0.5em 1.5em;
|
||||
}
|
||||
|
||||
.dropdown-label {
|
||||
user-select: none;
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 4px;
|
||||
padding: 0.5em 1em;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
gap: 1em;
|
||||
}
|
||||
|
||||
/* When the dropdown is exposed, highlight the background of the parent div label */
|
||||
.dropdown-checkbox[type='checkbox']:checked ~ .dropdown-label {
|
||||
background-color: var(--pst-color-on-background);
|
||||
}
|
||||
|
||||
.dropdown-content {
|
||||
display: none;
|
||||
position: absolute;
|
||||
background: var(--pst-color-on-background);
|
||||
padding: 0.5em 1em;
|
||||
border: 1px solid var(--pst-color-border);
|
||||
border-radius: 4px;
|
||||
flex-direction: column;
|
||||
gap: 1em;
|
||||
z-index: 1;
|
||||
}
|
||||
|
||||
/* Show dropdown when the checkbox is clicked */
|
||||
.dropdown-checkbox[type='checkbox']:checked ~ .dropdown-content {
|
||||
display: flex !important;
|
||||
}
|
||||
|
||||
/* Checkboxes */
|
||||
.checkbox-container {
|
||||
display: block;
|
||||
position: relative;
|
||||
padding-left: 35px;
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.checkbox-container input {
|
||||
position: absolute;
|
||||
opacity: 0;
|
||||
cursor: pointer;
|
||||
height: 0;
|
||||
width: 0;
|
||||
}
|
||||
.checkmark {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
height: 25px;
|
||||
width: 25px;
|
||||
/* Color of an empty checkbox square */
|
||||
background-color: var(--pst-color-surface);
|
||||
}
|
||||
.checkbox-container input:checked ~ .checkmark {
|
||||
/* Color of checkbox when checked */
|
||||
background-color: var(--pst-color-primary-highlight);
|
||||
}
|
||||
.checkbox-container input:checked ~ .checkmark:after {
|
||||
display: block;
|
||||
}
|
||||
.checkmark:after {
|
||||
content: '';
|
||||
position: absolute;
|
||||
display: none;
|
||||
}
|
||||
.checkbox-container .checkmark:after {
|
||||
left: 10px;
|
||||
top: 6px;
|
||||
width: 6px;
|
||||
height: 12px;
|
||||
border: solid var(--pst-color-border);
|
||||
border-width: 0 3px 3px 0;
|
||||
transform: rotate(45deg);
|
||||
}
|
||||
|
||||
.example.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.example-icon-area {
|
||||
width: 70px;
|
||||
height: 70px;
|
||||
position: relative;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.example-icon {
|
||||
border-radius: 10px !important; /* Competes against pydata-sphinx-theme img styles */
|
||||
background-color: transparent !important; /* Competes against pydata-sphinx-theme img styles */
|
||||
min-width: 75px;
|
||||
min-height: 75px;
|
||||
}
|
||||
|
||||
.remix-icon {
|
||||
position: absolute;
|
||||
font-size: 28px;
|
||||
color: var(--remix-icon-color);
|
||||
}
|
||||
|
||||
.example-link {
|
||||
color: var(--pst-color-text-base) !important;
|
||||
text-decoration: none;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
align-items: center;
|
||||
padding: 1em;
|
||||
}
|
||||
|
||||
.example {
|
||||
border-radius: 4px;
|
||||
background-color: color-mix(
|
||||
in srgb,
|
||||
var(--pst-color-on-background) 70%,
|
||||
transparent
|
||||
);
|
||||
box-shadow: 0px 4px 15px color-mix(in srgb, black 5%, transparent);
|
||||
}
|
||||
|
||||
.example-other-keywords {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.community-text {
|
||||
color: var(--pst-color-text-muted);
|
||||
}
|
||||
|
||||
.community-emojis {
|
||||
padding-left: 0.5em;
|
||||
}
|
||||
@@ -0,0 +1,292 @@
|
||||
html[data-theme='dark'] {
|
||||
--community-box-color: var(--pst-color-surface);
|
||||
}
|
||||
html[data-theme='light'] {
|
||||
--community-box-color: var(--pst-color-background);
|
||||
}
|
||||
|
||||
.main-content {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 0em 5em;
|
||||
}
|
||||
|
||||
.centered-heading {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-items: center;
|
||||
padding-bottom: 2em;
|
||||
}
|
||||
|
||||
.centered-heading > p {
|
||||
font-weight: 400;
|
||||
}
|
||||
|
||||
.clicky-tab-side-by-side {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
}
|
||||
|
||||
/* Area which contains all the tab selector labels */
|
||||
.tab-selector {
|
||||
width: 30%;
|
||||
min-width: 30%;
|
||||
}
|
||||
|
||||
/* Area which contains all the tab panes */
|
||||
.tab-area {
|
||||
flex-grow: 1;
|
||||
overflow-x: auto;
|
||||
box-shadow: 0px 6px 30px 5px var(--pst-color-shadow);
|
||||
}
|
||||
|
||||
.tab-pane {
|
||||
border-radius: 4px;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.tab-pane-links {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
padding: 1em;
|
||||
background: var(--pst-color-surface);
|
||||
border-radius: 0px 0px 4px 4px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.tab-pane-links > a:not(:first-child) {
|
||||
border-left: 1px solid var(--pst-color-border);
|
||||
padding-left: 1em;
|
||||
}
|
||||
.tab-pane pre {
|
||||
margin: 0;
|
||||
padding: 0.5em 1em;
|
||||
overflow-y: auto;
|
||||
animation: fadeEffect 1s; /* Fading effect takes 1 second */
|
||||
border: none;
|
||||
}
|
||||
|
||||
.tab-pane .highlight {
|
||||
border-radius: 4px 4px 0px 0px;
|
||||
height: 14em;
|
||||
}
|
||||
|
||||
/* Go from zero to full opacity */
|
||||
@keyframes fadeEffect {
|
||||
from {
|
||||
opacity: 0;
|
||||
}
|
||||
to {
|
||||
opacity: 1;
|
||||
}
|
||||
}
|
||||
|
||||
.header-button-icon {
|
||||
fill: var(--pst-color-text-base);
|
||||
width: 17px;
|
||||
height: 20px;
|
||||
min-width: 17px;
|
||||
min-height: 20px;
|
||||
}
|
||||
|
||||
.card-row {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
}
|
||||
|
||||
.link-card {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.5em;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.link-card-icon-label {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
gap: 0.5em;
|
||||
}
|
||||
|
||||
.link-card h4 {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.card-text-area {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding-left: 2.5em;
|
||||
}
|
||||
|
||||
.card-icon {
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
}
|
||||
|
||||
.community-box {
|
||||
border-radius: 4px;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
box-shadow: 0px 4px 10px 0px var(--pst-color-shadow);
|
||||
color: var(--pst-color-text-base) !important;
|
||||
align-items: center;
|
||||
background: var(--community-box-color);
|
||||
padding-left: 2em;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* Make remix icons larger */
|
||||
.community-box i {
|
||||
font-size: 25px;
|
||||
}
|
||||
|
||||
.community-box p {
|
||||
margin-top: 1rem;
|
||||
text-wrap: nowrap;
|
||||
text-overflow: ellipsis;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.community-box > img {
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
}
|
||||
|
||||
.nav-pills {
|
||||
background-color: var(--pst-color-surface);
|
||||
padding: 1em;
|
||||
border-radius: 4px;
|
||||
justify-content: center;
|
||||
width: 30%;
|
||||
gap: 0.5em;
|
||||
}
|
||||
|
||||
.nav-pills .nav-link.active {
|
||||
background-color: var(--pst-color-background) !important;
|
||||
box-shadow: 0px 3px 14px 2px var(--pst-color-shadow);
|
||||
border-radius: 4px;
|
||||
color: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
#v-pills-tab > .nav-link:hover {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
#v-pills-tab > a {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#v-pills-tab > .nav-link {
|
||||
color: var(--pst-color-text-base);
|
||||
white-space: nowrap;
|
||||
overflow: hidden;
|
||||
text-overflow: ellipsis;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
#v-pills-tabContent .row {
|
||||
background-color: var(--pst-color-background);
|
||||
}
|
||||
|
||||
#v-pills-tabContent {
|
||||
box-shadow: 0px 6px 30px 5px var(--pst-color-shadow);
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.links-grid-wrapper {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.links-grid {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr 1fr;
|
||||
gap: 2em;
|
||||
}
|
||||
|
||||
.links-grid > h4 {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.bd-article {
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.main-content {
|
||||
padding: 0em 5em;
|
||||
}
|
||||
|
||||
.heading-buttons {
|
||||
display: grid;
|
||||
grid-template-columns: 1fr 1fr 1fr;
|
||||
gap: 2em;
|
||||
}
|
||||
|
||||
.heading-buttons > a {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
align-items: center;
|
||||
padding: 1em 1em 1em 2em;
|
||||
border-radius: 4px;
|
||||
background: var(--community-box-color);
|
||||
color: var(--pst-color-text-base);
|
||||
font-weight: 500;
|
||||
text-wrap: nowrap;
|
||||
box-shadow: 0px 4px 10px 0px var(--pst-color-shadow);
|
||||
}
|
||||
|
||||
.header-button {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
gap: 1em;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.header-button i {
|
||||
font-size: 25px;
|
||||
}
|
||||
|
||||
a {
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.card-icon svg {
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
}
|
||||
|
||||
.card-icon path {
|
||||
stroke: var(--pst-color-link);
|
||||
}
|
||||
|
||||
#link-card-icon-filled {
|
||||
fill: var(--pst-color-link);
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
h3 {
|
||||
margin-bottom: 1em;
|
||||
margin-top: 4em;
|
||||
}
|
||||
|
||||
h4 {
|
||||
font-size: 16px;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.links-grid b {
|
||||
font-size: 18px;
|
||||
font-weight: 600;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
.sd-card.body {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
.sd-card-body > figure {
|
||||
height: 10em;
|
||||
}
|
||||
.card-figure {
|
||||
object-fit: contain;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
#train-logo > #train-logo-icon > path, #train-logo > #train-logo-icon > circle {
|
||||
stroke: var(--ray-blue);
|
||||
stroke-width: 10;
|
||||
fill: transparent;
|
||||
}
|
||||
|
||||
#train-logo > #train-logo-text {
|
||||
stroke: var(--pst-color-text-base);
|
||||
fill: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
#train-logo {
|
||||
margin: 3em 0em;
|
||||
}
|
||||
@@ -0,0 +1,108 @@
|
||||
/**
|
||||
* termynal.js
|
||||
*
|
||||
* @author Ines Montani <ines@ines.io>
|
||||
* @version 0.0.1
|
||||
* @license MIT
|
||||
*/
|
||||
|
||||
:root {
|
||||
--color-bg: #252a33;
|
||||
--color-text: #eee;
|
||||
--color-text-subtle: #a2a2a2;
|
||||
}
|
||||
|
||||
[data-termynal] {
|
||||
width: auto;
|
||||
max-width: 100%;
|
||||
background: var(--color-bg);
|
||||
color: var(--color-text);
|
||||
font-size: 18px;
|
||||
font-family: 'Fira Mono', Consolas, Menlo, Monaco, 'Courier New', Courier, monospace;
|
||||
border-radius: 4px;
|
||||
padding: 75px 45px 35px;
|
||||
position: relative;
|
||||
-webkit-box-sizing: border-box;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
[data-termynal]:before {
|
||||
content: '';
|
||||
position: absolute;
|
||||
top: 15px;
|
||||
left: 15px;
|
||||
display: inline-block;
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
border-radius: 50%;
|
||||
/* A little hack to display the window buttons in one pseudo element. */
|
||||
background: #d9515d;
|
||||
-webkit-box-shadow: 25px 0 0 #f4c025, 50px 0 0 #3ec930;
|
||||
box-shadow: 25px 0 0 #f4c025, 50px 0 0 #3ec930;
|
||||
}
|
||||
|
||||
[data-termynal]:after {
|
||||
content: 'bash';
|
||||
position: absolute;
|
||||
color: var(--color-text-subtle);
|
||||
top: 5px;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
[data-ty] {
|
||||
display: block;
|
||||
line-height: 2;
|
||||
}
|
||||
|
||||
[data-ty]:before {
|
||||
/* Set up defaults and ensure empty lines are displayed. */
|
||||
content: '';
|
||||
display: inline-block;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
[data-ty="input"]:before,
|
||||
[data-ty-prompt]:before {
|
||||
margin-right: 0.75em;
|
||||
color: var(--color-text-subtle);
|
||||
}
|
||||
|
||||
[data-ty="input"]:before {
|
||||
content: '$';
|
||||
}
|
||||
|
||||
[data-ty][data-ty-prompt]:before {
|
||||
content: attr(data-ty-prompt);
|
||||
}
|
||||
|
||||
[data-ty-cursor]:after {
|
||||
content: attr(data-ty-cursor);
|
||||
font-family: monospace;
|
||||
margin-left: 0.5em;
|
||||
-webkit-animation: blink 1s infinite;
|
||||
animation: blink 1s infinite;
|
||||
}
|
||||
|
||||
a[data-terminal-control] {
|
||||
text-align: right;
|
||||
display: block;
|
||||
color: #aebbff;
|
||||
}
|
||||
|
||||
|
||||
/* Cursor animation */
|
||||
|
||||
@-webkit-keyframes blink {
|
||||
50% {
|
||||
opacity: 0;
|
||||
}
|
||||
}
|
||||
|
||||
@keyframes blink {
|
||||
50% {
|
||||
opacity: 0;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
.example-gallery-link {
|
||||
padding: 1em 2em 1em 2em;
|
||||
text-decoration: none !important;
|
||||
color: var(--pst-color-text-base);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
svg.star-icon path.star-icon-path {
|
||||
fill: var(--pst-color-text-base);
|
||||
}
|
||||
|
||||
svg.star-icon {
|
||||
margin-right: 1em;
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 15 KiB |
@@ -0,0 +1,6 @@
|
||||
/* CSS for binder integration */
|
||||
|
||||
div.binder-badge {
|
||||
margin: 1em auto;
|
||||
vertical-align: middle;
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
/* Pandas dataframe css */
|
||||
/* Taken from: https://github.com/spatialaudio/nbsphinx/blob/fb3ba670fc1ba5f54d4c487573dbc1b4ecf7e9ff/src/nbsphinx.py#L587-L619 */
|
||||
|
||||
table.dataframe {
|
||||
border: none !important;
|
||||
border-collapse: collapse;
|
||||
border-spacing: 0;
|
||||
border-color: transparent;
|
||||
color: black;
|
||||
font-size: 12px;
|
||||
table-layout: fixed;
|
||||
}
|
||||
table.dataframe thead {
|
||||
border-bottom: 1px solid black;
|
||||
vertical-align: bottom;
|
||||
}
|
||||
table.dataframe tr,
|
||||
table.dataframe th,
|
||||
table.dataframe td {
|
||||
text-align: right;
|
||||
vertical-align: middle;
|
||||
padding: 0.5em 0.5em;
|
||||
line-height: normal;
|
||||
white-space: normal;
|
||||
max-width: none;
|
||||
border: none;
|
||||
}
|
||||
table.dataframe th {
|
||||
font-weight: bold;
|
||||
}
|
||||
table.dataframe tbody tr:nth-child(odd) {
|
||||
background: #f5f5f5;
|
||||
}
|
||||
table.dataframe tbody tr:hover {
|
||||
background: rgba(66, 165, 245, 0.2);
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 732 B |
Binary file not shown.
|
After Width: | Height: | Size: 425 B |
Binary file not shown.
|
After Width: | Height: | Size: 218 B |
Binary file not shown.
|
After Width: | Height: | Size: 3.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.5 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.0 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 481 B |
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 184 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 189 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 156 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 254 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 263 B |
Binary file not shown.
|
After Width: | Height: | Size: 931 B |
Binary file not shown.
|
After Width: | Height: | Size: 207 B |
Binary file not shown.
|
After Width: | Height: | Size: 3.4 KiB |
@@ -0,0 +1,5 @@
|
||||
<svg width="400" height="201" viewBox="0 0 400 201" xmlns="http://www.w3.org/2000/svg">
|
||||
<path id="ray-text" d="M325.949 134.356V109.785L302.442 66.6406H314.244L330.495 97.3062H331.946L348.198 66.6406H360L336.493 109.785V134.356H325.949ZM253.043 134.364L272.391 66.648H290.771L310.021 134.364H299.283L294.834 118.402H268.328L263.878 134.364H253.043ZM270.94 108.728H292.222L282.354 73.1294H280.807L270.94 108.728ZM198.887 134.364V66.648H227.327C231.519 66.648 235.195 67.3896 238.355 68.8729C241.58 70.2918 244.063 72.3555 245.804 75.0641C247.61 77.7727 248.513 80.9973 248.513 84.7378V85.8019C248.513 90.0583 247.481 93.4763 245.417 96.0559C243.418 98.5711 240.967 100.345 238.065 101.376V102.924C240.516 103.053 242.483 103.892 243.966 105.439C245.449 106.923 246.191 109.083 246.191 111.921V134.364H235.647V113.372C235.647 111.631 235.195 110.244 234.292 109.212C233.39 108.18 231.938 107.664 229.939 107.664H209.334V134.364H198.887ZM209.334 98.1842H226.166C229.907 98.1842 232.809 97.249 234.873 95.3788C236.937 93.4441 237.968 90.8322 237.968 87.5431V86.7692C237.968 83.4802 236.937 80.9005 234.873 79.0303C232.874 77.0956 229.971 76.1282 226.166 76.1282H209.334V98.1842Z" fill="black"/>
|
||||
<path fill-rule="evenodd" clip-rule="evenodd" d="M143.63 101.311L98.3087 146.632L94.9903 143.313L140.311 97.9925L143.63 101.311ZM141.953 102.334L51.4454 102.334V97.6409L141.953 97.6409V102.334ZM94.992 55.9863L140.313 101.307L143.631 97.9886L98.3105 52.6679L94.992 55.9863Z" fill="#02A0CF"/>
|
||||
<path d="M40 88.3163H62.6604V110.977H40V88.3163ZM85.3207 88.3163H107.981V110.977H85.3207V88.3163ZM85.3207 43H107.981V65.6604H85.3207V43ZM85.3207 133.645H107.981V156.306H85.3207V133.645ZM130.641 88.3163H153.301V110.977H130.641V88.3163Z" fill="#02A0CF"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 279 B |
@@ -0,0 +1,20 @@
|
||||
<svg width="157" height="42" viewBox="0 0 157 42" fill="none" xmlns="http://www.w3.org/2000/svg">
|
||||
<rect width="157" height="41.9994" fill="#0066FF"/>
|
||||
<path d="M17.204 25.5V15.896H22.258C22.9113 15.896 23.4527 16.0173 23.882 16.26C24.3207 16.4933 24.6473 16.8247 24.862 17.254C25.0767 17.674 25.184 18.1593 25.184 18.71C25.184 19.3727 25.0347 19.942 24.736 20.418C24.4373 20.894 24.0267 21.2347 23.504 21.44L25.52 25.5H24.064L22.188 21.678H18.534V25.5H17.204ZM18.534 20.544H22.132C22.664 20.544 23.0793 20.3807 23.378 20.054C23.686 19.7273 23.84 19.2793 23.84 18.71C23.84 18.3553 23.7747 18.0567 23.644 17.814C23.5133 17.5713 23.322 17.3847 23.07 17.254C22.818 17.114 22.5053 17.044 22.132 17.044H18.534V20.544ZM29.2832 25.668C28.5738 25.668 28.0045 25.486 27.5752 25.122C27.1458 24.758 26.9312 24.1233 26.9312 23.218V18.136H28.1632V23.05C28.1632 23.358 28.2005 23.61 28.2752 23.806C28.3498 24.002 28.4572 24.156 28.5972 24.268C28.7372 24.38 28.9005 24.4593 29.0872 24.506C29.2832 24.5527 29.4932 24.576 29.7172 24.576C30.0718 24.576 30.3985 24.492 30.6972 24.324C30.9958 24.156 31.2385 23.9133 31.4252 23.596C31.6212 23.2693 31.7192 22.8913 31.7192 22.462V18.136H32.9512V25.5H31.9572L31.8452 24.394H31.7472C31.5512 24.6927 31.3272 24.9353 31.0752 25.122C30.8325 25.3087 30.5572 25.444 30.2492 25.528C29.9505 25.6213 29.6285 25.668 29.2832 25.668ZM34.8625 25.5V18.136H35.8565L35.9685 19.242H36.0665C36.2625 18.9433 36.4818 18.7007 36.7245 18.514C36.9765 18.3273 37.2518 18.192 37.5505 18.108C37.8585 18.0147 38.1852 17.968 38.5305 17.968C38.9972 17.968 39.4032 18.0473 39.7485 18.206C40.1032 18.3647 40.3785 18.6213 40.5745 18.976C40.7798 19.3307 40.8825 19.8113 40.8825 20.418V25.5H39.6505V20.586C39.6505 20.278 39.6085 20.026 39.5245 19.83C39.4498 19.634 39.3425 19.48 39.2025 19.368C39.0718 19.256 38.9085 19.1767 38.7125 19.13C38.5258 19.0833 38.3158 19.06 38.0825 19.06C37.7372 19.06 37.4105 19.144 37.1025 19.312C36.7945 19.48 36.5472 19.7227 36.3605 20.04C36.1832 20.3573 36.0945 20.7353 36.0945 21.174V25.5H34.8625ZM48.7009 25.668C47.9543 25.668 47.3243 25.5327 46.8109 25.262C46.3069 24.982 45.9196 24.5573 45.6489 23.988C45.3876 23.4187 45.2569 22.6953 45.2569 21.818C45.2569 20.9313 45.3876 20.208 45.6489 19.648C45.9196 19.0787 46.3069 18.6587 46.8109 18.388C47.3243 18.108 47.9543 17.968 48.7009 17.968C49.4476 17.968 50.0729 18.108 50.5769 18.388C51.0903 18.6587 51.4776 19.0787 51.7389 19.648C52.0003 20.208 52.1309 20.9313 52.1309 21.818C52.1309 22.6953 52.0003 23.4187 51.7389 23.988C51.4776 24.5573 51.0903 24.982 50.5769 25.262C50.0729 25.5327 49.4476 25.668 48.7009 25.668ZM48.7009 24.632C49.1769 24.632 49.5736 24.5387 49.8909 24.352C50.2083 24.1653 50.4463 23.876 50.6049 23.484C50.7729 23.0827 50.8569 22.5787 50.8569 21.972V21.664C50.8569 21.048 50.7729 20.544 50.6049 20.152C50.4463 19.76 50.2083 19.4707 49.8909 19.284C49.5736 19.0973 49.1769 19.004 48.7009 19.004C48.2249 19.004 47.8236 19.0973 47.4969 19.284C47.1796 19.4707 46.9416 19.76 46.7829 20.152C46.6243 20.544 46.5449 21.048 46.5449 21.664V21.972C46.5449 22.5787 46.6243 23.0827 46.7829 23.484C46.9416 23.876 47.1796 24.1653 47.4969 24.352C47.8236 24.5387 48.2249 24.632 48.7009 24.632ZM53.6613 25.5V18.136H54.6553L54.7673 19.242H54.8653C55.0613 18.9433 55.2806 18.7007 55.5233 18.514C55.7753 18.3273 56.0506 18.192 56.3493 18.108C56.6573 18.0147 56.984 17.968 57.3293 17.968C57.796 17.968 58.202 18.0473 58.5473 18.206C58.902 18.3647 59.1773 18.6213 59.3733 18.976C59.5786 19.3307 59.6813 19.8113 59.6813 20.418V25.5H58.4493V20.586C58.4493 20.278 58.4073 20.026 58.3233 19.83C58.2486 19.634 58.1413 19.48 58.0013 19.368C57.8706 19.256 57.7073 19.1767 57.5113 19.13C57.3246 19.0833 57.1146 19.06 56.8813 19.06C56.536 19.06 56.2093 19.144 55.9013 19.312C55.5933 19.48 55.346 19.7227 55.1593 20.04C54.982 20.3573 54.8933 20.7353 54.8933 21.174V25.5H53.6613Z" fill="white"/>
|
||||
<path d="M88.9463 25.4877L92.4723 16.4961H94.3464L97.885 25.4877H96.3122L95.5128 23.4033H91.2529L90.4535 25.4877H88.9463ZM91.725 22.1578H95.028L94.019 19.523C93.993 19.4448 93.9534 19.3354 93.9012 19.1957C93.8491 19.056 93.7944 18.9028 93.7371 18.7371C93.6799 18.5713 93.6235 18.403 93.5671 18.2321C93.5099 18.0613 93.4603 17.9065 93.4165 17.7668H93.3382C93.2861 17.9326 93.2204 18.1295 93.1413 18.3567C93.0631 18.5839 92.9865 18.8027 92.9116 19.0122C92.8375 19.2218 92.7744 19.3926 92.7214 19.523L91.725 22.1578Z" fill="white"/>
|
||||
<path d="M98.5137 25.4903V18.5957H99.6413L99.7591 19.5786H99.8509C100.017 19.3253 100.214 19.1158 100.441 18.9491C100.668 18.7834 100.919 18.6563 101.195 18.5688C101.47 18.4813 101.774 18.4375 102.105 18.4375C102.542 18.4375 102.927 18.5141 103.259 18.6672C103.591 18.8204 103.851 19.067 104.039 19.4078C104.227 19.7486 104.321 20.2072 104.321 20.7845V25.4903H102.958V20.994C102.958 20.7323 102.925 20.5135 102.859 20.3385C102.794 20.1643 102.699 20.0238 102.577 19.9194C102.455 19.8142 102.311 19.7402 102.145 19.6964C101.979 19.6527 101.795 19.6308 101.595 19.6308C101.28 19.6308 100.994 19.7074 100.736 19.8605C100.479 20.0137 100.271 20.2257 100.113 20.4959C99.9561 20.7668 99.8778 21.0858 99.8778 21.4527V25.4894H98.5145L98.5137 25.4903Z" fill="white"/>
|
||||
<path d="M114.649 25.6473C114.168 25.6473 113.744 25.5951 113.377 25.4899C113.011 25.3847 112.704 25.2408 112.46 25.0574C112.215 24.8739 112.029 24.6534 111.903 24.3959C111.776 24.1384 111.713 23.8523 111.713 23.5376C111.713 23.4938 111.715 23.4543 111.72 23.4198C111.724 23.3853 111.726 23.3541 111.726 23.328H113.076V23.4719C113.076 23.7429 113.15 23.9592 113.299 24.1207C113.447 24.2823 113.647 24.3959 113.896 24.4616C114.145 24.5272 114.418 24.56 114.715 24.56C114.977 24.56 115.22 24.5272 115.443 24.4616C115.666 24.3959 115.849 24.2975 115.993 24.167C116.137 24.0358 116.209 23.87 116.209 23.6689C116.209 23.4071 116.117 23.206 115.934 23.0663C115.751 22.9266 115.51 22.8156 115.213 22.7322C114.916 22.6489 114.605 22.5597 114.282 22.4638C113.993 22.3855 113.707 22.2997 113.424 22.208C113.139 22.1163 112.884 21.9959 112.657 21.8478C112.43 21.6997 112.246 21.5112 112.107 21.284C111.967 21.0568 111.897 20.7732 111.897 20.4324C111.897 20.1092 111.967 19.8248 112.107 19.5808C112.246 19.3359 112.441 19.1305 112.69 18.9648C112.939 18.799 113.236 18.6719 113.581 18.5844C113.926 18.4969 114.304 18.4531 114.715 18.4531C115.125 18.4531 115.523 18.4994 115.855 18.5911C116.187 18.6829 116.468 18.8116 116.701 18.9774C116.932 19.1432 117.109 19.3426 117.232 19.574C117.354 19.8054 117.415 20.052 117.415 20.3146C117.415 20.3667 117.413 20.4214 117.408 20.4787C117.404 20.5359 117.402 20.5729 117.402 20.5897H116.064V20.4719C116.064 20.2977 116.016 20.1395 115.921 19.9998C115.825 19.8601 115.673 19.7465 115.469 19.659C115.263 19.5715 114.991 19.5277 114.65 19.5277C114.423 19.5277 114.222 19.5471 114.046 19.5866C113.871 19.6262 113.728 19.6809 113.614 19.7507C113.5 19.8206 113.415 19.9039 113.358 19.9998C113.301 20.0958 113.273 20.2052 113.273 20.3272C113.273 20.5283 113.345 20.6831 113.489 20.7925C113.633 20.9019 113.823 20.9937 114.06 21.0677C114.296 21.1418 114.549 21.2226 114.82 21.3101C115.135 21.3976 115.454 21.4868 115.777 21.5785C116.1 21.6702 116.399 21.7864 116.675 21.9261C116.95 22.0658 117.173 22.2585 117.343 22.5025C117.514 22.7474 117.599 23.0747 117.599 23.4854C117.599 23.8616 117.525 24.1889 117.376 24.4683C117.227 24.7485 117.02 24.9732 116.753 25.1432C116.486 25.3132 116.174 25.4402 115.815 25.5236C115.457 25.6069 115.068 25.6481 114.649 25.6481V25.6473Z" fill="white"/>
|
||||
<path d="M121.374 25.6498C120.684 25.6498 120.107 25.5211 119.644 25.2636C119.181 25.0061 118.834 24.6106 118.602 24.0771C118.371 23.5444 118.255 22.8669 118.255 22.0456C118.255 21.2243 118.373 20.5494 118.609 20.0209C118.845 19.4925 119.197 19.0969 119.664 18.8344C120.132 18.5727 120.706 18.4414 121.388 18.4414C121.86 18.4414 122.271 18.5003 122.62 18.6181C122.969 18.7359 123.266 18.911 123.511 19.1424C123.756 19.3738 123.937 19.6599 124.055 20.0007C124.173 20.3416 124.231 20.7303 124.231 21.1671H122.856C122.856 20.8002 122.803 20.4989 122.698 20.2625C122.593 20.0268 122.431 19.8493 122.213 19.7315C121.995 19.6136 121.711 19.5547 121.362 19.5547C120.995 19.5547 120.684 19.6397 120.431 19.8106C120.178 19.9805 119.985 20.2389 119.855 20.5839C119.723 20.9289 119.658 21.3817 119.658 21.9404V22.1634C119.658 22.697 119.721 23.1396 119.848 23.4939C119.974 23.8473 120.167 24.1099 120.424 24.2799C120.682 24.4507 121.012 24.5357 121.414 24.5357C121.763 24.5357 122.045 24.4726 122.26 24.3455C122.473 24.2184 122.635 24.035 122.744 23.7951C122.854 23.5553 122.909 23.2641 122.909 22.9233H124.232C124.232 23.334 124.173 23.7059 124.056 24.0375C123.938 24.3699 123.759 24.656 123.518 24.8958C123.277 25.1365 122.978 25.3217 122.62 25.4529C122.261 25.5842 121.847 25.6498 121.374 25.6498Z" fill="white"/>
|
||||
<path d="M132.052 25.4877V16.4961H133.415V25.4877H132.052Z" fill="white"/>
|
||||
<path d="M137.772 25.6498C137.064 25.6498 136.471 25.5211 135.995 25.2636C135.519 25.0061 135.159 24.6106 134.914 24.0771C134.669 23.5444 134.547 22.8669 134.547 22.0456C134.547 21.2243 134.669 20.5359 134.914 20.0075C135.159 19.479 135.521 19.086 136.002 18.8277C136.482 18.5702 137.081 18.4414 137.798 18.4414C138.453 18.4414 139.001 18.566 139.443 18.815C139.884 19.0641 140.219 19.442 140.446 19.9486C140.673 20.4552 140.787 21.093 140.787 21.8622V22.3864H135.95C135.967 22.8762 136.041 23.2843 136.173 23.6117C136.304 23.939 136.503 24.1797 136.769 24.3329C137.036 24.486 137.374 24.5626 137.785 24.5626C138.038 24.5626 138.266 24.5298 138.467 24.4641C138.668 24.3985 138.838 24.3026 138.978 24.1755C139.117 24.0493 139.227 23.8961 139.305 23.7169C139.383 23.5376 139.423 23.3348 139.423 23.1076H140.76C140.76 23.5183 140.69 23.881 140.55 24.1957C140.411 24.5104 140.207 24.7747 139.941 24.9884C139.674 25.203 139.358 25.3663 138.991 25.4799C138.624 25.5935 138.218 25.6507 137.772 25.6507V25.6498ZM135.976 21.3892H139.358C139.358 21.0484 139.321 20.7623 139.247 20.5309C139.173 20.2995 139.065 20.1093 138.925 19.9603C138.786 19.8122 138.619 19.7045 138.427 19.6389C138.235 19.5732 138.017 19.5404 137.772 19.5404C137.405 19.5404 137.092 19.6061 136.834 19.7373C136.577 19.8686 136.377 20.0714 136.238 20.3466C136.098 20.6218 136.01 20.9693 135.976 21.3884V21.3892Z" fill="white"/>
|
||||
<path d="M130.344 19.4212C130.142 19.1065 129.843 18.8625 129.445 18.6933C129.047 18.5225 128.551 18.4375 127.957 18.4375C127.405 18.4375 126.92 18.5124 126.5 18.6605C126.08 18.8086 125.754 19.0257 125.522 19.3093C125.291 19.5946 125.175 19.9413 125.175 20.3528V20.511C125.175 20.5548 125.179 20.5985 125.187 20.6414H126.499V20.4454C126.499 20.288 126.54 20.1424 126.624 20.0052C126.707 19.8706 126.851 19.7562 127.056 19.6644C127.262 19.5727 127.548 19.5264 127.915 19.5264C128.282 19.5264 128.564 19.5744 128.761 19.6703C128.958 19.7663 129.094 19.9017 129.168 20.0768C129.242 20.2518 129.28 20.4529 129.28 20.6801V21.3626C128.632 21.3626 128.035 21.398 127.489 21.4686C126.942 21.5377 126.472 21.6605 126.078 21.8355C125.685 22.0106 125.378 22.2513 125.16 22.5559C124.941 22.8622 124.832 23.2518 124.832 23.7239C124.832 24.1262 124.902 24.4518 125.042 24.7018C125.181 24.9508 125.363 25.1452 125.586 25.2849C125.809 25.4255 126.047 25.5222 126.301 25.5744C126.554 25.6266 126.79 25.6527 127.009 25.6527C127.368 25.6527 127.692 25.6114 127.98 25.529C128.269 25.4457 128.525 25.327 128.748 25.1747C128.97 25.0224 129.161 24.8448 129.318 24.6437V25.4962H130.644V20.5632C130.644 20.1172 130.543 19.7368 130.342 19.4221L130.344 19.4212ZM129.281 22.7932C129.281 23.0465 129.233 23.2804 129.137 23.495C129.041 23.7096 128.909 23.8931 128.743 24.0462C128.578 24.2002 128.378 24.3172 128.147 24.4005C127.915 24.4838 127.664 24.525 127.393 24.525C127.157 24.525 126.951 24.4897 126.776 24.419C126.601 24.35 126.47 24.2473 126.382 24.1119C126.295 23.9764 126.251 23.8072 126.251 23.6069C126.251 23.2737 126.377 23.0145 126.631 22.8252C126.885 22.6375 127.24 22.5088 127.694 22.4389C128.149 22.3682 128.678 22.3346 129.282 22.3346V22.794L129.281 22.7932Z" fill="white"/>
|
||||
<path d="M105.409 26.7656V28.0405H106.784L107.226 26.7656H105.409Z" fill="white"/>
|
||||
<path d="M108.939 25.7197L111.42 18.5938H110.029L108.304 24.0645L108.229 23.8709L106.316 18.5938H104.839L107.581 25.7416L107.227 26.7657H108.577L108.939 25.7197Z" fill="white"/>
|
||||
<path d="M108.578 26.7656H107.228L106.786 28.0414H107.66C107.945 28.0414 108.199 27.8604 108.292 27.5912L108.579 26.7665L108.578 26.7656Z" fill="white"/>
|
||||
<path d="M79.7498 21.2812L77.3145 25.4998H82.2189C82.399 25.4998 82.5656 25.4038 82.6565 25.2473L84.9463 21.2812H79.7498Z" fill="white"/>
|
||||
<path d="M84.9463 20.7185L82.6565 16.7525C82.5665 16.5959 82.3998 16.5 82.2189 16.5H77.3145L79.7498 20.7185H84.9463Z" fill="white"/>
|
||||
<path d="M72.4434 16.4996H77.3142L74.8619 12.2525C74.7719 12.0959 74.6053 12 74.4243 12H69.8447L72.4425 16.4996H72.4434Z" fill="white"/>
|
||||
<path d="M69.3573 12.2812L67.0675 16.2473C66.9775 16.4038 66.9775 16.5957 67.0675 16.7522L69.5198 20.9994L71.9552 16.7808L69.3573 12.2812Z" fill="white"/>
|
||||
<path d="M71.956 25.2185L69.5206 21L67.0675 25.248C66.9775 25.4045 66.9775 25.5964 67.0675 25.7529L69.3573 29.719L71.9552 25.2194L71.956 25.2185Z" fill="white"/>
|
||||
<path d="M69.8447 29.9996H74.4243C74.6044 29.9996 74.7711 29.9037 74.8619 29.7471L77.3142 25.5H72.4434L69.8456 29.9996H69.8447Z" fill="white"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 490 B |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 16 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 394 B |
@@ -0,0 +1,190 @@
|
||||
// Create chat-widget div
|
||||
var chatWidgetDiv = document.createElement('div');
|
||||
chatWidgetDiv.className = 'chat-widget';
|
||||
chatWidgetDiv.innerHTML = `
|
||||
<button id="openChatBtn">Ask AI</button>
|
||||
`;
|
||||
document.body.appendChild(chatWidgetDiv);
|
||||
|
||||
// Create chat-popup div
|
||||
const date = new Date();
|
||||
var chatPopupDiv = document.createElement('div');
|
||||
chatPopupDiv.className = 'chat-popup';
|
||||
chatPopupDiv.id = 'chatPopup';
|
||||
chatPopupDiv.innerHTML = `
|
||||
<div class="chatHeader">
|
||||
<div class="header-wrapper">
|
||||
<h3 class="assistant-title">Ray Docs AI - Ask a question</h3>
|
||||
</div>
|
||||
<button id="closeChatBtn" class="btn">
|
||||
<i class="fas fa-times"></i>
|
||||
</button>
|
||||
</div>
|
||||
<div id="chatContainer" class="chatContentContainer">
|
||||
<div id="result">
|
||||
Please note that the results of this bot are automated and may be incorrect or contain inappropriate information.
|
||||
<div id="anchor"></div>
|
||||
</div>
|
||||
<div class="input-group">
|
||||
<textarea id="searchBar" class="input" rows="3" placeholder="Do not include any personal or confidential information."></textarea>
|
||||
<button id="searchBtn" class="btn btn-primary">Ask AI</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="chatFooter text-right p-2">
|
||||
© Copyright ${date.getFullYear()}, The Ray Team.
|
||||
</div>
|
||||
`;
|
||||
chatPopupDiv.messages = [];
|
||||
document.body.appendChild(chatPopupDiv);
|
||||
|
||||
const anchorDiv = document.getElementById('anchor');
|
||||
const chatContainerDiv = document.getElementById('chatContainer');
|
||||
|
||||
const blurDiv = document.createElement('div');
|
||||
blurDiv.id = 'blurDiv';
|
||||
blurDiv.classList.add('blurDiv-hidden');
|
||||
document.body.appendChild(blurDiv);
|
||||
|
||||
// blur background when chat popup is open
|
||||
document.getElementById('openChatBtn').addEventListener('click', function () {
|
||||
document.querySelector('.search-button__wrapper').classList.remove('show');
|
||||
document.getElementById('chatPopup').style.display = 'block';
|
||||
blurDiv.classList.remove('blurDiv-hidden');
|
||||
});
|
||||
|
||||
// un-blur background when chat popup is closed
|
||||
document.getElementById('closeChatBtn').addEventListener('click', function () {
|
||||
document.getElementById('chatPopup').style.display = 'none';
|
||||
blurDiv.classList.add('blurDiv-hidden');
|
||||
});
|
||||
|
||||
// set code highlighting options
|
||||
marked.setOptions({
|
||||
renderer: new marked.Renderer(),
|
||||
highlight: function (code) {
|
||||
return hljs.highlight('python', code).value;
|
||||
},
|
||||
});
|
||||
|
||||
function highlightCode() {
|
||||
document.querySelectorAll('pre code').forEach((block) => {
|
||||
hljs.highlightBlock(block);
|
||||
});
|
||||
}
|
||||
|
||||
function renderCopyButtons(resultDiv) {
|
||||
let preElements = resultDiv.querySelectorAll('pre');
|
||||
preElements.forEach((preElement, index) => {
|
||||
preElement.style.position = 'relative';
|
||||
|
||||
let uniqueId = `button-id-${index}`;
|
||||
preElement.id = uniqueId;
|
||||
|
||||
// Set the proper attributes to the button to make is a sphinx copy button
|
||||
let copyButton = document.createElement('button');
|
||||
copyButton.className = 'copybtn o-tooltip--left';
|
||||
copyButton.setAttribute('data-tooltip', 'Copy');
|
||||
copyButton.setAttribute('data-clipboard-target', `#${uniqueId}`);
|
||||
copyButton.style.position = 'absolute';
|
||||
copyButton.style.top = '10px';
|
||||
copyButton.style.right = '10px';
|
||||
copyButton.style.opacity = 'inherit';
|
||||
|
||||
let imgElement = document.createElement('img');
|
||||
imgElement.src = window.data.copyIconSrc;
|
||||
imgElement.alt = 'Copy to clipboard';
|
||||
|
||||
copyButton.appendChild(imgElement);
|
||||
preElement.appendChild(copyButton);
|
||||
});
|
||||
}
|
||||
|
||||
const searchBar = document.getElementById('searchBar');
|
||||
const searchBtn = document.getElementById('searchBtn');
|
||||
|
||||
function rayAssistant(event) {
|
||||
const resultDiv = document.getElementById('result');
|
||||
|
||||
const searchTerm = searchBar.value;
|
||||
// handle empty input
|
||||
if (!searchTerm) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (
|
||||
event.type === 'click' ||
|
||||
(event.type === 'keydown' && event.key === 'Enter')
|
||||
) {
|
||||
// for improved UX, we want to equate a carriage return as hitting the send button and prevent default behavior of entering a new line
|
||||
if (event.type === 'keydown' && event.key === 'Enter') {
|
||||
event.preventDefault();
|
||||
}
|
||||
// clear search bar value and change placeholder to prompt user to ask follow up question
|
||||
searchBar.value = '';
|
||||
searchBar.placeholder = 'Ask follow up question here';
|
||||
|
||||
// Add query to the list of messages
|
||||
chatPopupDiv.messages.push({role: 'user', content: searchTerm});
|
||||
|
||||
let msgBlock = document.createElement('div');
|
||||
|
||||
resultDiv.insertBefore(msgBlock, anchorDiv);
|
||||
|
||||
let divider = document.createElement('hr');
|
||||
msgBlock.appendChild(divider);
|
||||
|
||||
let msgHeader = document.createElement('div');
|
||||
msgHeader.style.fontWeight = 'bold';
|
||||
// capitalize first letter of question header
|
||||
msgHeader.innerHTML =
|
||||
searchTerm.charAt(0).toUpperCase() + searchTerm.slice(1);
|
||||
msgBlock.appendChild(msgHeader);
|
||||
|
||||
async function readStream() {
|
||||
try {
|
||||
const response = await fetch(
|
||||
'https://ray-assistant-public-bxauk.cld-kvedzwag2qa8i5bj.s.anyscaleuserdata.com/chat',
|
||||
{
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
Accept: 'application/json',
|
||||
},
|
||||
body: JSON.stringify({messages: chatPopupDiv.messages}),
|
||||
}
|
||||
);
|
||||
const reader = response.body.getReader();
|
||||
let decoder = new TextDecoder('utf-8');
|
||||
let msgContent = document.createElement('div');
|
||||
msgBlock.appendChild(msgContent);
|
||||
msgContent.innerHTML = '';
|
||||
|
||||
let collectChunks = '';
|
||||
|
||||
while (true) {
|
||||
const {done, value} = await reader.read();
|
||||
if (done) {
|
||||
renderCopyButtons(resultDiv);
|
||||
break;
|
||||
}
|
||||
|
||||
const chunk = decoder.decode(value, {stream: true});
|
||||
collectChunks += chunk;
|
||||
let html = marked.parse(collectChunks);
|
||||
html = DOMPurify.sanitize(html);
|
||||
msgContent.innerHTML = html;
|
||||
highlightCode();
|
||||
}
|
||||
chatPopupDiv.messages.push({role: 'assistant', content: collectChunks});
|
||||
// we want to autoscroll to the bottom of the chat container after the answer is streamed in
|
||||
chatContainerDiv.scrollTo(0, chatContainerDiv.scrollHeight);
|
||||
} catch (error) {
|
||||
console.error('Fetch API failed:', error);
|
||||
}
|
||||
}
|
||||
readStream().then((res) => console.log(res));
|
||||
}
|
||||
}
|
||||
|
||||
searchBtn.addEventListener('click', rayAssistant);
|
||||
searchBar.addEventListener('keydown', rayAssistant);
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user