Skip to content

Commit 63ab2a8

Browse files
authored
dev: include the HTML documentation build in the local lint suite (#25586)
## Which issue does this PR close? - Part of #21048. ## Rationale for this change `./dev/rust_lint.sh` runs the non-functional CI checks locally, but not the documentation website build. Sphinx builds with `-W`, so one warning fails CI. A contributor sees that failure only after a push. ## What changes are included in this PR? - `ci/scripts/check_docs_html.sh` runs the existing `docs/build.sh` through uv. It works from any directory. It reports a missing `uv`, `cargo`, `cargo-depgraph`, `dot`, or `make` before the build starts, and installs nothing. It has no write mode. - `dev/rust_lint.sh` runs the new script as a read-only step. - The "Test doc build" job runs the same script. The path filters of the job also watch the script. - `docs/source/contributor-guide/testing.md` documents the command. `docs/build.sh` still owns the Sphinx and dependency-graph commands. The Python dependencies and the Sphinx settings do not change. ## What is the testing strategy for this PR? Local checks with command stubs and a real build: - The script starts the same build from the repository root and from other directories. - `-h`, `--help`, `--write`, `--allow-dirty`, and an unknown flag all fail before any tool runs. - Each missing tool gives one message and a non-zero exit. - Failures in `uv`, `cargo depgraph`, and `dot` keep their exit code. - A real build writes `docs/build/html/index.html` and changes no tracked file. - One Sphinx warning, a page outside every toctree, fails the build. `uv run ./dev/rust_lint.sh` passes with the new step. ## Are there any user-facing changes? No. This changes the contributor tooling only.
1 parent 2306a4b commit 63ab2a8

4 files changed

Lines changed: 104 additions & 4 deletions

File tree

‎.github/workflows/docs_pr.yaml‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,14 @@ on:
2626
push:
2727
paths:
2828
- "docs/**"
29+
- "ci/scripts/check_docs_html.sh"
2930
branches:
3031
- main
3132
- branch-*
3233
pull_request:
3334
paths:
3435
- "docs/**"
36+
- "ci/scripts/check_docs_html.sh"
3537
# manual trigger
3638
# https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
3739
workflow_dispatch:
@@ -63,7 +65,4 @@ jobs:
6365
with:
6466
tool: cargo-depgraph@1.6
6567
- name: Build docs html and check for warnings
66-
run: |
67-
set -x
68-
cd docs
69-
uv run --package datafusion-docs ./build.sh # fails on errors
68+
run: ./ci/scripts/check_docs_html.sh # fails on errors

‎ci/scripts/check_docs_html.sh‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# Builds the documentation website with `docs/build.sh`, the same way the
21+
# "Test doc build" job does. Sphinx runs with `-W`, so a warning fails the
22+
# build.
23+
24+
set -euo pipefail
25+
26+
SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")"
27+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
28+
ROOT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
29+
30+
usage() {
31+
cat >&2 <<USAGE
32+
Usage: $0
33+
34+
Builds the HTML documentation with docs/build.sh and fails on any Sphinx
35+
warning. There is no write mode.
36+
37+
Needs uv, cargo, cargo-depgraph, Graphviz dot, and make. See docs/README.md
38+
for the Python and Graphviz setup.
39+
40+
Each build rewrites docs/build and docs/source/_static/data/deps.svg, which
41+
Git ignores. The HTML entry point is docs/build/html/index.html.
42+
USAGE
43+
exit 1
44+
}
45+
46+
while [[ $# -gt 0 ]]; do
47+
case "$1" in
48+
-h|--help)
49+
usage
50+
;;
51+
*)
52+
usage
53+
;;
54+
esac
55+
shift
56+
done
57+
58+
# Report every missing prerequisite in one run. This check installs nothing.
59+
missing=0
60+
require_tool() {
61+
local cmd="$1"
62+
local hint="$2"
63+
if ! command -v "$cmd" > /dev/null 2>&1; then
64+
echo "[${SCRIPT_NAME}] ${cmd} was not found on PATH. ${hint}" >&2
65+
missing=1
66+
fi
67+
}
68+
69+
require_tool "uv" "Install uv to get Python and the documentation dependencies: https://docs.astral.sh/uv/getting-started/installation/"
70+
require_tool "cargo" "Install the Rust toolchain with rustup: https://rustup.rs/"
71+
require_tool "cargo-depgraph" "Install it with: cargo install cargo-depgraph --version '^1.6' --locked"
72+
require_tool "dot" "Install Graphviz (e.g., brew install graphviz, or apt-get install graphviz)."
73+
require_tool "make" "Install make (e.g., xcode-select --install, or apt-get install make)."
74+
75+
if [[ "${missing}" -ne 0 ]]; then
76+
exit 1
77+
fi
78+
79+
cd "${ROOT_DIR}"
80+
81+
echo "[${SCRIPT_NAME}] Building the documentation website with docs/build.sh"
82+
83+
# `docs/build.sh` owns the dependency graph and Sphinx commands and moves to
84+
# the docs directory itself.
85+
uv run --package datafusion-docs ./docs/build.sh
86+
87+
echo "[${SCRIPT_NAME}] Wrote the HTML documentation to docs/build/html/index.html"

‎dev/rust_lint.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,7 @@ declare -a READONLY_STEPS=(
142142
"ci/scripts/check_circular_dependencies.sh|false"
143143
"ci/scripts/check_unused_dependencies.sh|false"
144144
"ci/scripts/rust_docs.sh|false"
145+
"ci/scripts/check_docs_html.sh|false"
145146
)
146147

147148
for entry in "${WRITE_STEPS[@]}" "${READONLY_STEPS[@]}"; do

‎docs/source/contributor-guide/testing.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -320,6 +320,19 @@ the README:
320320
[`dev/update_function_docs.sh`]: https://github.com/apache/datafusion/blob/main/dev/update_function_docs.sh
321321
[`ci/scripts/check_generated_docs.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_generated_docs.sh
322322

323+
## Documentation HTML Build
324+
325+
[`ci/scripts/check_docs_html.sh`] builds the documentation website, the same
326+
build the "Test doc build" job runs. Sphinx builds with `-W`, so one warning
327+
fails the check. `./dev/rust_lint.sh` runs it. Run the script with `--help`
328+
for the tools it needs and the files it writes.
329+
330+
```shell
331+
./ci/scripts/check_docs_html.sh
332+
```
333+
334+
[`ci/scripts/check_docs_html.sh`]: https://github.com/apache/datafusion/blob/main/ci/scripts/check_docs_html.sh
335+
323336
## Benchmarks
324337

325338
### Criterion Benchmarks

0 commit comments

Comments
 (0)