Allow building docs with a local sphinx_airflow_theme checkout by shahar1 · Pull Request #75001 · apache/airflow

shahar1 · 2026-10-18T12:34:46Z

As titled, this PR adds an option to mount airflow-site/sphinx_airflow_theme when running CI locally.

AI Summary

Changes to the docs theme live in `airflow-site`, so checking them on real Airflow documentation meant building a wheel by hand and force-installing it inside a Breeze shell. This adds `--sphinx-airflow-theme-path` to `breeze build-docs`: the given `sphinx_airflow_theme` sub-folder of an `airflow-site` checkout is mounted read-only into the docs container and installed there before the build, so neither checkout nor `uv.lock` changes. The theme builds with `flit_core`, which does not write into the source folder, so the read-only mount is enough. Only the theme is reinstalled; a dependency the theme adds is installed with it.

breeze build-docs apache-airflow --sphinx-airflow-theme-path ../airflow-site/sphinx_airflow_theme

The option fails with a usage error before any Docker work when the folder has no generated CSS/JS (./site.sh prepare-theme not run, or the airflow-site root passed instead of the sub-folder), since the docs would otherwise build without styles. It also fails when combined with --sdk-docs-only, because the SDK docs do not use the Sphinx theme.

Verified locally with Docker: a marker added to the local theme's layout.html appears in the built HTML and is absent without the option. The install command was also checked against a read-only copy of the theme in the CI image. The Breeze tests for build-docs now share one Docker-free fixture that points AIRFLOW_ROOT_PATH to a temporary folder; before, the existing Python-version test deleted the real generated/**/apis folders. The companion airflow-site change adds ./site.sh build-airflow-docs and an informational CI preview on top of this option.

Not included: --autobuild with automatic rebuild when the mounted theme files change (raised on #70139); tracked in #75374.

Was generative AI tooling used to co-author this PR?

Yes — Claude Code (Claude Opus 5.5)

Generated-by: Claude Code (Claude Opus 5.5) following the guidelines

🤖 Generated with Claude Code

potiuk

Thanks — being able to test unreleased theme changes against the real docs build is useful. A few things before this can go in:

Likely blocker

scripts/ci/docker-compose/mount-sphinx-airflow-theme.yml mounts the theme with read_only: true, but run_docs_build.sh then runs uv pip install --reinstall /opt/sphinx_airflow_theme. The theme is a setuptools project, and building it from a source directory writes *.egg-info / build/ into that directory, so the install should fail with "Read-only file system" and abort the docs build (set -euo pipefail). Has this been run end-to-end with a real airflow-site checkout? Copying the tree to a temp dir inside the container and installing from the copy keeps the checkout clean and avoids the problem.

Behaviour

--no-deps means a theme change that adds or bumps a dependency is silently installed without it, so the local build can diverge from what the released theme would produce.
With --sdk-docs-only, --sphinx-airflow-theme-path is validated and then ignored without any message. Either pass it through or reject the combination.
The sphinx_airflow_theme/static/_gen check lives in the command body and ends in sys.exit(1). A click callback= on the option raising click.BadParameter would give a standard usage error and keep the validation next to the option. It would also help to say explicitly in the help text that the path is the sphinx_airflow_theme sub-folder, not the airflow-site repo root, which is what most people will pass first.
run_docs_build.sh decides whether to install by checking that /opt/sphinx_airflow_theme exists, and that path is also hard-coded in the compose file. Gating on SPHINX_AIRFLOW_THEME_PATH (already exported by Breeze) would keep a single source of truth.

Tests

The docker_mocks fixture uses with patch.multiple(...); the repo convention prefers @mock.patch decorators.
TestBuildDocsSphinxAirflowTheme duplicates the Docker stubbing already in TestBuildDocsPythonVersion. A shared fixture would remove the duplication, and redirecting AIRFLOW_ROOT_PATH to tmp_path there would stop the older class touching the real generated/ tree.
test_theme_path_is_passed_to_shell_params should assert result.exit_code == 0 before reading call_args, otherwise an unrelated failure either passes silently or surfaces as an AttributeError.

Drafted-by: Claude Code (Opus 5.5); reviewed by @potiuk before posting

Andrushika

Thanks! Left some comments.
We might need a follow-up issue for autobuild before merge, since closes: #45576 closes the issue and autobuild was asked for on #70139.

shahar1 · 2026-10-22T06:10:08Z

Thanks for the reviews. The branch is rebased onto main, and the changes are in e721ac9..8cec2d9.

Review summary points, in order:

Read-only mount: no change. The theme builds with flit_core (build-backend = "flit_core.buildapi" in airflow-site/sphinx_airflow_theme/pyproject.toml), which does not write *.egg-info or build/ into the source folder. It was run end to end with a real airflow-site checkout: a marker in layout.html showed up in the built HTML. The install was also checked against a read-only mount in the CI image.
--no-deps is replaced with --reinstall-package sphinx-airflow-theme. Only the theme is reinstalled, and a dependency that the theme adds is installed too.
--sdk-docs-only together with --sphinx-airflow-theme-path is now rejected with a usage error.
The _gen check is now a click callback that raises click.BadParameter. The help text says the path is the sphinx_airflow_theme sub-folder, not the airflow-site root.
run_docs_build.sh now installs the theme only when SPHINX_AIRFLOW_THEME_PATH is set.
6–7. Both build-docs test classes now share one fixture that stubs Docker with autospecced mocks and points AIRFLOW_ROOT_PATH to tmp_path. Before, the existing Python-version test deleted the real generated/**/apis folders.
The tests assert exit_code == 0 before reading call_args. The rejection tests also check the error text, so they fail without the new option.

Follow-up issue for the auto-rebuild mode asked for on #70139: #75374.

Drafted-by: Claude Code (Opus 5.5); reviewed by @shahar1 before posting

Andrushika

LGTM. Thanks!

Changes to the docs theme live in airflow-site, and the only way to see them on real Airflow documentation was to build a wheel by hand, copy it into the Airflow checkout and install it inside Breeze before building. Add a --sphinx-airflow-theme-path option to breeze build-docs. Breeze mounts the given theme folder read-only into the container and installs it there before the build, so an unreleased theme can be checked on the real docs, locally or from airflow-site's own scripts and CI. The folder must contain the generated theme assets, otherwise the build would be silently unstyled. closes: apache#45576

- Reinstall only the theme with --reinstall-package (keeps new theme dependencies) and gate the install on SPHINX_AIRFLOW_THEME_PATH. - Validate the theme path in a click callback raising BadParameter and clarify in the help that it is the sphinx_airflow_theme sub-folder. - Reject --sphinx-airflow-theme-path together with --sdk-docs-only. - Share one autospecced Docker-stubbing fixture between the build-docs tests, redirect AIRFLOW_ROOT_PATH to tmp_path, and tighten the theme tests.

The two rejection tests now check the error text, so they fail without the new option.

prepare-theme copies the existing dist bundle, so edited SCSS/JS only reach _gen after build-landing-pages.

apache/airflow-site#1691 makes prepare-theme rebuild the landing pages when their SCSS/JS sources changed, so the note would be wrong once it lands. The recommended command works with old and new site.sh.

github-actions · 2026-10-24T05:14:18Z

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-4-test

…ckout (apache#75001) (cherry picked from commit 602e150) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

…ckout (#75001) (#75537) (cherry picked from commit 602e150) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

shahar1 requested review from amoghrajesh, ashb, bugraoz93, choo121600, ephraimbuddy, gopidesupavan, jason810496, jedcunningham, jscheffl, potiuk and vatsrahul1001 as code owners October 18, 2026 12:34

boring-cyborg Bot added area:dev-tools backport-to-v3-4-test Backport to v3-4-test labels Oct 18, 2026

shahar1 mentioned this pull request Oct 18, 2026

Build Airflow docs with the theme from this checkout apache/airflow-site#1691

Merged

potiuk reviewed Oct 18, 2026

View reviewed changes

shahar1 added the ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. label Oct 18, 2026

Andrushika reviewed Oct 21, 2026

View reviewed changes

Comment thread scripts/ci/docker-compose/mount-sphinx-airflow-theme.yml Resolved

Comment thread scripts/in_container/run_docs_build.sh Resolved

Comment thread dev/breeze/tests/test_developer_commands.py Resolved

Comment thread dev/breeze/src/airflow_breeze/commands/developer_commands.py Resolved

shahar1 force-pushed the build-docs-with-local-sphinx-theme branch from 279d4a6 to 8cec2d9 Compare October 22, 2026 06:08

shahar1 mentioned this pull request Oct 22, 2026

Auto-rebuild docs when a mounted local sphinx_airflow_theme changes #75374

Open

shahar1 requested a review from potiuk October 22, 2026 06:16

Andrushika approved these changes Oct 22, 2026

View reviewed changes

shahar1 requested a review from kaxil October 23, 2026 13:38

kaxil reviewed Oct 23, 2026

View reviewed changes

Comment thread contributing-docs/11_documentation_building.rst Resolved

kaxil approved these changes Oct 23, 2026

View reviewed changes

shahar1 force-pushed the build-docs-with-local-sphinx-theme branch from 8cec2d9 to 61758b6 Compare October 23, 2026 19:00

shahar1 added 4 commits October 24, 2026 06:34

Assert the rejection messages in the local theme build-docs tests

17d3056

The two rejection tests now check the error text, so they fail without the new option.

Make the sdk-docs-only rejection test independent of terminal width

c90d863

shahar1 added 3 commits October 24, 2026 06:34

Strip styling before matching the sdk-docs-only rejection message

b0ae386

Tell users to rebuild landing pages before prepare-theme

3e0d68d

prepare-theme copies the existing dist bundle, so edited SCSS/JS only reach _gen after build-landing-pages.

Drop the note on prepare-theme copying a stale bundle

0b4ad71

apache/airflow-site#1691 makes prepare-theme rebuild the landing pages when their SCSS/JS sources changed, so the note would be wrong once it lands. The recommended command works with old and new site.sh.

shahar1 force-pushed the build-docs-with-local-sphinx-theme branch from 61758b6 to 0b4ad71 Compare October 24, 2026 03:34

shahar1 merged commit 602e150 into apache:main Oct 24, 2026
147 checks passed

shahar1 deleted the build-docs-with-local-sphinx-theme branch October 24, 2026 05:13

shahar1 added a commit that referenced this pull request Oct 24, 2026

[v3-4-test] Allow building docs with a local sphinx_airflow_theme che…

88b0674

…ckout (#75001) (#75537) (cherry picked from commit 602e150) Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>

sreecharan-desu mentioned this pull request Oct 27, 2026

Auto-rebuild docs when a mounted local sphinx_airflow_theme changes #75743

Open

1 task done

Navigation Menu

Repository navigation

Allow building docs with a local sphinx_airflow_theme checkout - #75001

Allow building docs with a local sphinx_airflow_theme checkout#75001
shahar1 merged 7 commits into
apache:mainfrom
shahar1:build-docs-with-local-sphinx-theme

shahar1 commented Oct 18, 2026 •

edited

Loading

Uh oh!

potiuk left a comment

Uh oh!

Andrushika left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahar1 commented Oct 22, 2026 •

edited

Loading

Uh oh!

Andrushika left a comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 24, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

shahar1 commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Was generative AI tooling used to co-author this PR?

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

Andrushika left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahar1 commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Andrushika left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 24, 2026

Backport successfully created: v3-4-test

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shahar1 commented Oct 18, 2026 •

edited

Loading

shahar1 commented Oct 22, 2026 •

edited

Loading