Skip to content

Isolate otel config-priority test from ambient OTEL env vars - #75842

Merged
jason810496 merged 2 commits into
mainfrom
fix-otel-config-priorities-test-isolation
Oct 28, 2026
Merged

jason810496 merged 2 commits into
mainfrom
fix-otel-config-priorities-test-isolation

Conversation

@vatsrahul1001

Copy link
Copy Markdown
Contributor

Problem

TestOtelMetrics::test_config_priorities has started failing on main (and the release branches) for the six cases that expect the Airflow config host/port to be used when a case provides no OTEL endpoint env var:

FAILED ...test_config_priorities[protocol_is_ignored_if_no_env_endpoint]
FAILED ...test_config_priorities[use_airflow_config]
FAILED ...test_config_priorities[airflow_config_ipv6_loopback_is_bracketed]
FAILED ...test_config_priorities[airflow_config_ipv6_literal_is_bracketed]
FAILED ...test_config_priorities[airflow_config_already_bracketed_ipv6_is_preserved]
FAILED ...test_config_priorities[airflow_config_ipv4_literal_passes_through_unchanged]
  AssertionError: assert '...127.0.0.1:4318...' == '...breeze-otel-collector:4318...'

The test applied only each case's provided_env_vars on top of the ambient environment via env_vars(...), which overlays but does not clear other OTEL_* vars. The RunsOn CI runners export OTEL_EXPORTER_OTLP_ENDPOINT (resolving to 127.0.0.1:4318) for their own telemetry. For the cases that intentionally provide no endpoint env var, that ambient var leaks in and — correctly, per the config-priority rules — wins over the Airflow config, so the resolved endpoint becomes 127.0.0.1:4318 instead of the expected config host. The test is non-hermetic; the production code is correct.

This is why it fails on main and every release branch at once (the runner env is shared), not a regression in the OTEL code.

Fix

Clear all OTEL_* env vars (via monkeypatch.delenv) before applying the case's provided_env_vars, so the test only ever sees its own inputs regardless of the runner environment.

Testing

Reproduced the exact six failures locally by exporting OTEL_EXPORTER_OTLP_ENDPOINT, then confirmed all cases pass with the fix in place even with the ambient var set:

OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318 \
  pytest shared/observability/.../test_otel_logger.py::TestOtelMetrics::test_config_priorities
# 10 passed

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 4.8)

Generated-by: Claude Code (Opus 4.8) following the guidelines

test_config_priorities asserts that the Airflow config host/port is used
when a case provides no OTEL endpoint env var. It applied only the case's
provided_env_vars on top of the ambient environment, so any OTEL_* var
already exported by the runner leaked in. RunsOn CI runners export
OTEL_EXPORTER_OTLP_ENDPOINT (resolving to 127.0.0.1:4318), which wins over
the Airflow config and failed the six config-priority cases on both main
and the release branches.

Clear all OTEL_* env vars before applying provided_env_vars so the test
only sees its own inputs regardless of the runner environment.
Per review: clear ambient OTEL_* vars for every test in TestOtelMetrics
rather than only test_config_priorities.
@jason810496
jason810496 merged commit 5df209d into main Oct 28, 2026
111 checks passed
@jason810496
jason810496 deleted the fix-otel-config-priorities-test-isolation branch October 28, 2026 07:29
@github-actions github-actions Bot added this to the Airflow 3.4.1 milestone Oct 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.4.1 based on: backport label targeting v3-4-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-4-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-4-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Oct 28, 2026
…ars (apache#75842)

* Isolate otel config-priority test from ambient OTEL env vars

test_config_priorities asserts that the Airflow config host/port is used
when a case provides no OTEL endpoint env var. It applied only the case's
provided_env_vars on top of the ambient environment, so any OTEL_* var
already exported by the runner leaked in. RunsOn CI runners export
OTEL_EXPORTER_OTLP_ENDPOINT (resolving to 127.0.0.1:4318), which wins over
the Airflow config and failed the six config-priority cases on both main
and the release branches.

Clear all OTEL_* env vars before applying provided_env_vars so the test
only sees its own inputs regardless of the runner environment.

* Move OTEL_* clearing into a class-scoped autouse fixture

Per review: clear ambient OTEL_* vars for every test in TestOtelMetrics
rather than only test_config_priorities.
(cherry picked from commit 5df209d)

Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
vatsrahul1001 added a commit that referenced this pull request Oct 28, 2026
…ars (#75842) (#75844)

* Isolate otel config-priority test from ambient OTEL env vars

test_config_priorities asserts that the Airflow config host/port is used
when a case provides no OTEL endpoint env var. It applied only the case's
provided_env_vars on top of the ambient environment, so any OTEL_* var
already exported by the runner leaked in. RunsOn CI runners export
OTEL_EXPORTER_OTLP_ENDPOINT (resolving to 127.0.0.1:4318), which wins over
the Airflow config and failed the six config-priority cases on both main
and the release branches.

Clear all OTEL_* env vars before applying provided_env_vars so the test
only sees its own inputs regardless of the runner environment.

* Move OTEL_* clearing into a class-scoped autouse fixture

Per review: clear ambient OTEL_* vars for every test in TestOtelMetrics
rather than only test_config_priorities.
(cherry picked from commit 5df209d)

Co-authored-by: Rahul Vats <43964496+vatsrahul1001@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-4-test Backport to v3-4-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants