Forward workload_identity_impersonation_path in SnowflakeHook by spcvalente · Pull Request #74591 · apache/airflow

spcvalente · 2026-10-12T13:01:14Z

SnowflakeHook now reads workload_identity_impersonation_path from the connection extras (a list or a comma-separated string) and passes it to the connector (snowflake-connector-python>=4.0.0). A workload can then impersonate a role (AWS) or service account (GCP) before authenticating with Workload Identity Federation. Docs and unit tests included.

closes: #74590

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Generated-by: Claude Code following the guidelines

boring-cyborg · 2026-10-12T13:01:19Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

jroachgolf84 · 2026-10-12T14:13:38Z

    * ``token_file_path``: Path to a file holding the OIDC ID token used when ``workload_identity_provider``
      is ``OIDC``. The connector reads the token from this file, which suits projected or rotated tokens
      (for example a Kubernetes service-account token).
+    * ``workload_identity_impersonation_path``: Identities to impersonate, in order, before authenticating with


Should this have _path at the end of it? I would assume anything with a _path at the end is pointing to a path to a file itself.

I think in GCP it's something like _chain. Does that make more sense?

Kept the name on purpose: it's the connector's own parameter (workload_identity_impersonation_path, snowflake-connector-python 4.0.0), same as the other WIF extras (workload_identity_provider, token, token_file_path), so it maps 1:1 to Snowflake's docs. _chain is the Google provider's naming. Agree _path reads like a file though, so the docs now say it's a chain of identities, not a file path.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

jroachgolf84 · 2026-10-12T14:14:57Z

+        "impersonation_path",
+        [
+            ["arn:aws:iam::123456789012:role/a", "arn:aws:iam::123456789012:role/b"],
+            "arn:aws:iam::123456789012:role/a, arn:aws:iam::123456789012:role/b",


Can you update the spacing a bit? It's a little confusing.

Cleaned up: the ARNs live in one constant, the comma-separated case is built from it, and the assert is a single line.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

jroachgolf84 · 2026-10-12T14:16:12Z

        if token_file_path:
            conn_config["token_file_path"] = token_file_path

+        impersonation_path = self._get_field(extra_dict, "workload_identity_impersonation_path")


Did you validate that adding workload_identity_impersonation_path to extra doesn't already propagate it through?

Yes. Running the new test against main's hook fails with KeyError: 'workload_identity_impersonation_path': _get_conn_params builds conn_config from known fields only, so unknown extras never reach snowflake.connector.connect.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

frodo2000 · 2026-10-12T14:25:09Z

Maybe it will be good to add get_connection_form_widgets and get_ui_field_behaviour entries

spcvalente · 2026-10-12T14:27:08Z

Maybe it will be good to add get_connection_form_widgets and get_ui_field_behaviour entries

@frodo2000 good call, added both in 89577b8: a text field, with a placeholder for comma-separated role ARNs (AWS) or service accounts (GCP). The hook already splits a comma-separated string into the list the connector expects.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

gopidesupavan · 2026-10-14T21:50:36Z

+      authenticating with Workload Identity Federation, as a list or a comma-separated string (not a file path;
+      the name matches the connector parameter). For ``AWS``, IAM role ARNs that the workload's credentials
+      assume in turn; for ``GCP``, service account emails. Snowflake then trusts the last identity in the chain
+      instead of the workload's own. Requires ``snowflake-connector-python>=4.0.0``.


The docs say “Requires >=4.0.0”, but nothing enforces it. Could we either raise exception in the hook

Done in df40620: the hook now raises AirflowOptionalProviderFeatureException when the path is set and snowflake-connector-python is older than 4.0.0. Older connectors only warn about an unknown parameter and then authenticate as the workload itself, so failing is safer. Bumping the minimum is blocked by the sqlalchemy>=2.0 TODO in pyproject.toml.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

gopidesupavan · 2026-10-14T21:55:09Z


        assert "workload_identity_provider" in widgets

+    def test_connection_form_exposes_workload_identity_impersonation_path(self):


Add a test that the key is absent when unset, or when the value is only whitespace or commas, like the existing test_get_conn_params_omits_oidc_token_when_unset

Added test_get_conn_params_omits_workload_identity_impersonation_path_when_empty, covering unset, "", " , " and []. The " , " case did forward an empty list before.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

gopidesupavan · 2026-10-14T21:57:00Z


+        impersonation_path = self._get_field(extra_dict, "workload_identity_impersonation_path")
+
+        if impersonation_path:


may be?

if isinstance(impersonation_path, str): impersonation_path = impersonation_path.split(",") if not isinstance(impersonation_path, list): raise ValueError("workload_identity_impersonation_path must be a list or a comma-separated string") impersonation_path = [str(step).strip() for step in impersonation_path if str(step).strip()] if impersonation_path: conn_config["workload_identity_impersonation_path"] = impersonation_path

include list aswell its better to validate

Taken as suggested: it accepts a list or a comma-separated string, strips and drops empty steps either way, and raises ValueError otherwise.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

gopidesupavan · 2026-10-14T22:00:28Z

+        if impersonation_path:
+            if isinstance(impersonation_path, str):
+                impersonation_path = [step.strip() for step in impersonation_path.split(",") if step.strip()]
+            conn_config["workload_identity_impersonation_path"] = impersonation_path


I think this will break when it goes through SQLAlchemy. get_uri() / get_sqlalchemy_engine() build the URL from _get_conn_params(), and snowflake-sqlalchemy’s _url() just does str(v) on anything that isn’t a string. So the list goes into the query string as ['arn:...', 'arn:...'], and the connector gets one string back, not a list. get_conn() and Snowpark would be fine, but anything going through the engine (get_df etc.) wouldn’t get the impersonation chain.

Could we handle it the way private_key and session_parameters are handled? Add it to the exclusion list in _conn_params_to_sqlalchemy_uri and pass it through connect_args in get_sqlalchemy_engine:

Good catch, confirmed: _url() turns the list into a string and the connector then iterates it character by character. It is now excluded from the URI and passed through connect_args, like private_key and session_parameters, with a test on get_sqlalchemy_engine.

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

spcvalente · 2026-10-22T14:04:26Z

Hey @gopidesupavan, rebased onto main and addressed your comments. Could you take another look when you have a moment? 🙏

Drafted-by: Claude Code (Opus 5.5) (no human review before posting)

Read workload_identity_impersonation_path from the connection extras (a list or a comma-separated string) and pass it to the connector, so a workload can impersonate a role or service account before authenticating with Workload Identity Federation. closes: apache#74590

…my engines snowflake-sqlalchemy stringifies non-string URL values, so the impersonation list reached the connector as one string and SQLAlchemy-based calls lost the chain. Connector releases before 4.0.0 ignore the parameter and authenticate as the workload's own identity, so fail loudly instead.

The provider.yaml check now requires conn-fields and placeholders to match the hook's connection form.

spcvalente requested a review from potiuk as a code owner October 12, 2026 13:01

boring-cyborg Bot added area:providers kind:documentation labels Oct 12, 2026

boring-cyborg Bot added the provider:snowflake Issues related to Snowflake provider label Oct 12, 2026

spcvalente mentioned this pull request Oct 12, 2026

Snowflake: forward workload_identity_impersonation_path to the connector #74590

Open

jroachgolf84 reviewed Oct 12, 2026

View reviewed changes

spcvalente force-pushed the snowflake-wif-impersonation-path branch from 14dcfef to cb3b78f Compare October 12, 2026 14:23

spcvalente requested a review from jroachgolf84 October 12, 2026 14:24

eladkal requested a review from gopidesupavan October 13, 2026 07:09

spcvalente force-pushed the snowflake-wif-impersonation-path branch 4 times, most recently from 638e5f7 to 8b87ca5 Compare October 14, 2026 16:01

gopidesupavan reviewed Oct 14, 2026

View reviewed changes

spcvalente force-pushed the snowflake-wif-impersonation-path branch from 8b87ca5 to df40620 Compare October 15, 2026 10:41

spcvalente requested a review from gopidesupavan October 15, 2026 10:42

spcvalente force-pushed the snowflake-wif-impersonation-path branch 6 times, most recently from 3cd3f0c to 37d0a17 Compare October 22, 2026 14:04

spcvalente force-pushed the snowflake-wif-impersonation-path branch from 37d0a17 to 37b009a Compare October 27, 2026 10:37

spcvalente added 4 commits October 27, 2026 11:15

Expose workload_identity_impersonation_path in the connection form

9ab3e8f

Register workload_identity_impersonation_path in provider.yaml

3b5f20e

The provider.yaml check now requires conn-fields and placeholders to match the hook's connection form.

spcvalente force-pushed the snowflake-wif-impersonation-path branch from 37b009a to 3b5f20e Compare October 27, 2026 11:32


		assert "workload_identity_provider" in widgets

		def test_connection_form_exposes_workload_identity_impersonation_path(self):


		impersonation_path = self._get_field(extra_dict, "workload_identity_impersonation_path")

		if impersonation_path:

Conversation

spcvalente commented Oct 12, 2026

Was generative AI tooling used to co-author this PR?

Uh oh!

boring-cyborg Bot commented Oct 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

frodo2000 commented Oct 12, 2026

Uh oh!

spcvalente commented Oct 12, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spcvalente commented Oct 22, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants