Skip to content

Allow airflowctl integration tests to use a non-default API server port - #76124

Merged
shahar1 merged 1 commit into
apache:mainfrom
shahar1:ctl016-run2-fix-e2e-host-port
Oct 31, 2026
Merged

shahar1 merged 1 commit into
apache:mainfrom
shahar1:ctl016-run2-fix-e2e-host-port

Conversation

@shahar1

@shahar1 shahar1 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

While testing the airflowctl 0.1.6 (pre-)release candidate locally, I ran into problems when port 8080 was already in use. This PR lets to run the API server and airflowctl integration tests on a different port.

AI Summary `breeze testing airflow-ctl-integration-tests` cannot run on a host where TCP port 8080 is already taken. The harness variable `HOST_PORT` is honoured only by the harness's own `requests` calls: the quickstart compose file it copies hard-codes `8080:8080`, the `auth login` it prepends to every command has no `--api-url`, and the no-auth `version --remote` test falls back to `http://localhost:8080` inside the client. With `HOST_PORT=localhost:18080` the stack cannot even bind, and with the port mapping patched by hand every CLI command still talks to the wrong port (36/36 server-bound tests fail, zero requests reach the API server).

This makes HOST_PORT the single knob:

  • airflowctl reads AIRFLOW_CLI_API_URL as its default API URL: auth login and auth token resolve --api-url at call time as args.api_url or get_default_api_url(), and get_client falls back to it when the stored credentials carry no URL (the version --remote case). Precedence is --api-url, then the URL stored by auth login, then the variable, then http://localhost:8080; documented in cli-and-env-variables-ref.rst.
  • The quickstart compose file publishes the API server on ${AIRFLOW_API_SERVER_PORT:-8080}:8080 (same interpolation style as AIRFLOW_IMAGE_NAME and AIRFLOW_UID); documented in docker-compose/index.rst.
  • The harness derives the port number from HOST_PORT, exports AIRFLOW_API_SERVER_PORT before compose up, and passes AIRFLOW_CLI_API_URL to every CLI command (a test's own env_vars still win).

Defaults are unchanged, so CI behaviour does not change. The --api-url help text is deliberately untouched (its SVG would need regenerating); the variable is documented in the environment-variable reference instead.

Tested:

  • uv run --project airflow-ctl pytest airflow-ctl/tests: 708 passed (new tests: default-URL helper incl. the empty-string case, get_client fallback, auth login / auth token URL resolution with and without --api-url, HOST_PORT port parsing incl. the invalid cases).
  • prek run mypy-airflow-ctl --all-files: 0 errors before and after.
  • End to end on a host where 8080 is held by another process: HOST_PORT=localhost:18080 breeze testing airflow-ctl-integration-tests --image-name apache/airflow:3.4.0rc2-python3.12 (rc2 PROD image built locally from Sync v3-4-stable with v3-4-test to release 3.4.0rc2 #76068) from this branch with no other change: 207 passed, 2 failed. test_airflowctl_no_auth_commands[version --remote] passes; the two failures are providers get apache-airflow-providers-standard in both passes, which a 3.4.0rc2 server answers with API route not found (the endpoint is newer than rc2) and is unrelated to this change.

Follow-ups, not in this PR: mention the variable in the --api-url help (needs the help SVGs regenerated); docker-tests/tests/docker_tests/test_docker_compose_quick_start.py reads HOST_PORT but does not export AIRFLOW_API_SERVER_PORT.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5.1, with Sonnet 5.5 and Opus 5.5 subagents)

Generated-by: Claude Code (Fable 5.1) following the guidelines

… port

The airflowctl integration suite could not run on a host where TCP
8080 is already taken. HOST_PORT was honoured only by the harness's own
requests calls; the compose file hard-coded 8080 and every CLI command
under test fell back to http://localhost:8080.

- airflowctl: add AIRFLOW_CLI_API_URL as the default API URL. It is
  resolved at call time for 'auth login' and 'auth token' when
  --api-url is not given, and is the client fallback when no URL is
  stored in the credentials (this also covers 'version --remote'
  without credentials).
- Compose quickstart: publish the API server on the host port given by
  AIRFLOW_API_SERVER_PORT (default 8080) and document the variable.
- Integration harness: export AIRFLOW_API_SERVER_PORT derived from
  HOST_PORT before compose up, and pass AIRFLOW_CLI_API_URL to every
  CLI command.

The defaults stay at localhost:8080, so CI behaviour does not change.
@shahar1
shahar1 requested a review from potiuk as a code owner October 31, 2026 09:02
@shahar1 shahar1 changed the title Let airflowctl and its integration tests use a non-default API server port Allow airflowctl integration tests to use a non-default API server port. Oct 31, 2026
@shahar1 shahar1 changed the title Allow airflowctl integration tests to use a non-default API server port. Allow airflowctl integration tests to use a non-default API server port Oct 31, 2026
@shahar1
shahar1 enabled auto-merge (squash) October 31, 2026 09:44
@shahar1
shahar1 merged commit f3b4fec into apache:main Oct 31, 2026
90 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: airflow-ctl/v0-1-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ airflow-ctl/v0-1-test PR Link

shahar1 added a commit that referenced this pull request Oct 31, 2026
…a non-default API server port (#76124) (#76137)

The airflowctl integration suite could not run on a host where TCP
8080 is already taken. HOST_PORT was honoured only by the harness's own
requests calls; the compose file hard-coded 8080 and every CLI command
under test fell back to http://localhost:8080.

- airflowctl: add AIRFLOW_CLI_API_URL as the default API URL. It is
  resolved at call time for 'auth login' and 'auth token' when
  --api-url is not given, and is the client fallback when no URL is
  stored in the credentials (this also covers 'version --remote'
  without credentials).
- Compose quickstart: publish the API server on the host port given by
  AIRFLOW_API_SERVER_PORT (default 8080) and document the variable.
- Integration harness: export AIRFLOW_API_SERVER_PORT derived from
  HOST_PORT before compose up, and pass AIRFLOW_CLI_API_URL to every
  CLI command.

The defaults stay at localhost:8080, so CI behaviour does not change.
(cherry picked from commit f3b4fec)

Co-authored-by: Shahar Epstein <60007259+shahar1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants