Skip to content

Prepare providers release 2026-11-03 - #76490

Merged
o-nikolas merged 7 commits into
apache:mainfrom
aws-mwaa:onikolas/providers/release-2026-11-03
Nov 5, 2026
Merged

o-nikolas merged 7 commits into
apache:mainfrom
aws-mwaa:onikolas/providers/release-2026-11-03

Conversation

@o-nikolas

@o-nikolas o-nikolas commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Prepare the 2026-11-03 provider wave for release-note review: 32 provider version bumps with
regenerated package metadata, documentation and the workspace lockfile.

Baseline for each provider is its last final release. Classified from main at the wave base.
The CI-environment bot commits (the flit_core build-requirement bump and the earlier dependency
bump) are recorded in each provider's changelog excluded block, as in previous waves, and do not on
their own trigger a release.

Prepared versions

Provider Version Provider Version
alibaba 4.0.0 amazon 9.40.0
apache.arrow 0.2.0 apache.hive 10.0.0
apache.kafka 3.0.0 celery 4.0.0
cncf.kubernetes 11.0.0 common.compat 1.23.1
common.io 1.11.1 common.sql 3.0.0
databricks 7.24.0 dbt.cloud 4.11.2
edge3 5.1.1 exasol 5.0.0
fab 3.11.0 ftp 3.16.2
google 22.9.0 http 6.4.0
keycloak 0.12.0 microsoft.azure 15.3.1
microsoft.mssql 4.7.3 mysql 6.8.0
postgres 7.1.2 presto 6.0.0
sendgrid 4.2.6 smtp 3.2.1
snowflake 6.21.0 ssh 7.0.2
standard 2.0.0 teradata 3.7.2
trino 7.0.0 yandex 5.0.0

Version-bump rationale

Major (11). Ten come from the deprecation removals merged on 2026-10-31:

  • alibaba 4.0.0 — removed OSSKeySensor.get_hook (use hook).
  • common.sql 3.0.0, apache.hive 10.0.0, exasol 5.0.0, presto 6.0.0, trino 7.0.0 — removed the
    deprecated get_pandas_df / get_pandas_df_by_chunks hook methods (use
    get_df(..., df_type="pandas")).
  • celery 4.0.0, cncf.kubernetes 11.0.0 — removed the deprecated cleanup_stuck_queued_tasks
    executor method (use revoke_task).
  • standard 2.0.0 — removed the deprecated TimeDeltaSensorAsync (use
    TimeDeltaSensor(..., deferrable=True)).
  • yandex 5.0.0 — removed the deprecated DataprocHook.client (use dataproc_client).

The eleventh is apache.kafka 3.0.0: the EventProducer plugin now emits schema_version 2, with
Dag-run date fields formatted via datetime.isoformat() and a missing date sent as null rather
than "". Consumers that parse those fields need to adapt, so it is breaking for plugin users even
though it does not affect everyone. See the checklist below.

Minor (9).

  • amazon 9.40.0 — new on_kill for SageMakerTrainingOperator, parallel ECS executor launches
    (run_task_concurrency, default 1 preserves current behaviour), SqlToS3Operator paging,
    S3Hook.aload_bytes, an EKS pod-exec operator, Bedrock scheduling options, and fail_on_exit_code
    on the ECS run operator. The per-team EKS executor change is additive; in multi-team mode (an
    experimental feature) a team's [aws_eks_executor] config no longer falls back to the global
    section, and the executor now needs cncf.kubernetes>=11.0.0, which the changelog note calls out.
    The SAML IdP-response parser now disables external-entity resolution (defence-in-depth hardening).
  • google 22.9.0 — new Google Bid Manager operators and a new async BigQueryAsyncHook.aget_records
    (the sync get_records is deprecated, not removed), the IdP-extra-parameters connection fix, plus
    other fixes. The Cloud Logging reader fix is a bug fix: prepare_log_filter is not a supported
    public interface (confirmed by the maintainer on the PR). The "remove unused … helper" PRs touch
    private helpers only (misc); "HO mode" only raises the google-cloud-dataproc floor.
  • fab 3.11.0 — new Dag Bundles permission resource, which takes effect on Airflow 3.4.1+. The
    POST /auth/token and /auth/token/cli change is a bug fix: they now honour only the configured
    AUTH_TYPE.
  • keycloak 0.12.0 — Dag bundles restricted to SuperAdmin in multi-team mode on Airflow 3.4.1+.
  • databricks 7.24.0 — new agent invocation operator and hook.
  • snowflake 6.21.0 — Cortex agent hook aligned to the common.ai managed-agent contract, and new
    Workload Identity Federation support for Snowflake REST API auth.
  • mysql 6.8.0 — asyncmy becomes the default async driver; aiomysql stays installed and gets a
    dedicated [aiomysql] extra.
  • apache.arrow 0.2.0 — AdbcHook.insert_rows now defaults fast_executemany=True and routes through
    the driver's bulk ingest (a changelog note records how to keep the previous path), and the hook now
    builds the ADBC database URI from the connection fields. Pre-1.0, so a behaviour change rides a minor
    bump.
  • http 6.4.0 — HttpEventTrigger now passes asset state store accessors to the response-check
    callable when it accepts them (backward compatible).

Patch (12). common.compat 1.23.1, common.io 1.11.1, dbt.cloud 4.11.2, edge3 5.1.1, ftp 3.16.2,
microsoft.azure 15.3.1, microsoft.mssql 4.7.3, postgres 7.1.2, sendgrid 4.2.6, smtp 3.2.1, ssh 7.0.2,
teradata 3.7.2.

  • microsoft.azure 15.3.1 — S3ToAzureBlobStorageOperator now defaults s3_verify=None, so it follows
    the connection / botocore default and matches both its own docstring and the sibling transfer
    operators. Classified a bug fix; the note tells anyone who relied on the old default to pass
    s3_verify=False.
  • postgres 7.1.2 and microsoft.mssql 4.7.3 — cache the dialect on PostgresHook / MsSqlHook.
  • dbt.cloud 4.11.2 — sync requests now honour timeout_seconds and connection proxies.
  • smtp 3.2.1 also raises the aiosmtplib floor to 5.1.0.

Doc-only

  • openai — only a docstring note merged since 2.2.0, so its .latest-doc-only-change.txt marker is
    repointed and no artifact is released.

Inter-provider dependency pins

  • amazon's cncf.kubernetes extra moves to >=11.0.0 (the per-team executor needs this wave's
    cncf.kubernetes change).
  • snowflake's common.ai extra is set to >=1.0.0 (the managed-agent module it imports first shipped
    in common.ai 1.0.0) rather than auto-bumped to this wave's version.
  • databricks' new common.ai extra is raised to >=1.0.0. It merged as >=0.11.0 (Add Databricks agent invocation operator and hook #74386), but the
    managed-agent base it imports first shipped in common.ai 1.0.0, and 0.11.0 was never finally
    released, so >=1.0.0 is the accurate floor. Flagging for visibility since it adjusts a floor set by
    the original PR.

Not released

  • openlineage, docker, sftp — only test or system-test changes since their last release.
  • hashicorp — only the CI-environment dependency bump.
  • informatica — only a dependency bump in its dev simulator, which is not part of the distribution.
  • anthropic — documentation-only marker, unchanged.
  • common.ai — excluded from this wave at the maintainer's request, to allow more time on pending
    changes. Its unreleased changes stay on main for the next wave.

Rebased onto main

Rebased onto main after review and folded in the provider changes that merged since the wave was
prepared:

apache.kafka ships as a major in this wave, without waiting for #73276.


Was generative AI tooling used to co-author this PR?
  • Yes — Kiro (Claude Opus 4.5)

@o-nikolas o-nikolas closed this Nov 4, 2026
@o-nikolas o-nikolas reopened this Nov 4, 2026

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I compared each changelog entry with the diff of the PR it cites, looking only at that provider's files, and checked related PRs for reverts and dependencies. There are no revert pairs in this wave. The comments below are factual or scope mismatches, not wording preferences.

On the checklist item about the fab and keycloak notes: 3.4.1 matches #76474 and the #76329 milestone, and AccessView.DAG_BUNDLES is not in the 3.4.0rc2 tag. It was also not on v3-4-test when I checked, so the notes depend on that change landing. #76474 is still open, and it also carries a code fix that this wave does not include (see the comment on the fab entry).


Drafted-by: Claude Code (Opus 5.5); reviewed by shahar1 before posting

Comment thread providers/fab/docs/changelog.rst Resolved
Comment thread providers/amazon/docs/changelog.rst Resolved
Comment thread providers/google/docs/changelog.rst Resolved
Comment thread providers/google/docs/changelog.rst Resolved
Comment thread providers/google/docs/changelog.rst Resolved
Comment thread providers/common/ai/docs/changelog.rst Resolved
Comment thread providers/snowflake/docs/changelog.rst Resolved

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mechanical side of this wave checks out. Versions agree across provider.yaml, pyproject.toml, __init__.py, the README, the docs index and the changelog for all 30 providers. The changelogs cover every PR since each provider's last tag, and the common.ai>=1.0.0 floors match what databricks and snowflake import. I ran these checks on b93289a7; the commit after it only edits changelogs.

I'm not approving yet, but I know the remaining items will be addressed: 533b9cd already takes up much of the changelog feedback, and the rest follows once #76474 and #76161 merge and you fold them in at the rebase. Since that changes the fab, keycloak and google content, I'll take another look after the rebase rather than review it twice.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The findings
below are observations, not blockers; an Apache Airflow
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.

More on how Apache Airflow handles maintainer review:
Contributing guide.

@o-nikolas
o-nikolas force-pushed the onikolas/providers/release-2026-11-03 branch from 533b9cd to 9580811 Compare November 5, 2026 00:53
@potiuk
potiuk enabled auto-merge (squash) November 5, 2026 01:47
@potiuk
potiuk disabled auto-merge November 5, 2026 01:47

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rebase and the common.ai exclusion look good: versions agree across all 29 providers and the folded PRs are in the changelogs. Two small inline notes suggest saying that the Dag bundle permission needs Airflow 3.4.1. That also depends on #76329 reaching v3-4-test: its automatic backport failed and it is not on the branch yet.

Smaller observations

  • See inline comments on providers/fab/docs/changelog.rst:42, providers/keycloak/docs/changelog.rst:43.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
Contributing guide.

Comment thread providers/fab/docs/changelog.rst Resolved
Comment thread providers/keycloak/docs/changelog.rst Resolved

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval stands. CI is green at this head and the /auth/token thread is resolved. My two inline notes about the Airflow 3.4.1 wording are not blocking: I am leaving it to the release manager to apply them or resolve them as they are. The v3-4-test backport of #76329 is also left to the maintainers, and it should land before 3.4.1 is cut.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
Contributing guide.

30 provider version bumps with regenerated metadata, documentation and the workspace lockfile.
Add the missing arrow note for the new fast_executemany default, warn about the Airflow 3.0.x example Dag that imports the removed TimeDeltaSensorAsync, reword the BigQuery get_records entry to match what changed, and list the common.ai review loop example under Doc-only.
The cncf.kubernetes floor for amazon and the common.ai floor for snowflake changed after the templates were regenerated, so their README and docs index still listed the old floors.
Correct entries that overstated or misdescribed what shipped, add notes for behaviour changes the review found (the AwsEksExecutor cncf.kubernetes requirement, the BigQueryStreamingBufferEmptySensor default, Viewer and User losing the Dag bundle pages), and list the runtime dependency floors that the CI environment upgrades raised.
Its maintainers asked for a few more days on pending changes, so it moves to the next wave and keeps its 1.1.0 release metadata.
Add the changes that merged after the wave was prepared: the Dag bundle menu fix for Airflow 3.4.0 in fab and keycloak, the Google Cloud IdP extra parameters fix, the AwsDuckDBHook credential refresh, and the new trigger endpoint permissions in the fab docs.
Rebase onto main pulled in more provider changes after approval. Fold the new entries into amazon (SAML IdP XXE hardening), snowflake (Workload Identity Federation), apache.arrow (build ADBC URI from connection fields) and postgres (dialect caching). Release http 6.4.0, microsoft.mssql 4.7.3 and dbt.cloud 4.11.2, which gained code since the cut, and repoint openai's doc-only marker. Apply the reviewer's Airflow 3.4.1 wording to the fab and keycloak Dag bundle entries.
@o-nikolas
o-nikolas force-pushed the onikolas/providers/release-2026-11-03 branch from 9580811 to 6f31dfa Compare November 5, 2026 16:19
@o-nikolas
o-nikolas requested a review from josh-fell as a code owner November 5, 2026 16:19
@o-nikolas
o-nikolas merged commit c4eaf4b into apache:main Nov 5, 2026
155 checks passed
@o-nikolas
o-nikolas deleted the onikolas/providers/release-2026-11-03 branch November 5, 2026 17:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants