Skip to content

Support Workload Identity Federation in Snowflake REST API authentication - #75810

Merged
kaxil merged 8 commits into
apache:mainfrom
omar-massfih:snowflake-rest-wif
Nov 5, 2026
Merged

kaxil merged 8 commits into
apache:mainfrom
omar-massfih:snowflake-rest-wif

Conversation

@omar-massfih

@omar-massfih omar-massfih commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

The problem

SnowflakeSqlApiOperator can't be used with Workload Identity Federation (WIF). If a Snowflake connection is set up for WIF (authenticator=WORKLOAD_IDENTITY), the SQL API hook fails with:

Workload identity federation is not supported for Snowflake REST APIs

SnowflakeHook already supports WIF (#68107), and the connection docs say the same connection works for the SQL API hook, so the two disagreed. The same limit applied to the Cortex Agent hook and the pydanticai_snowflake connection type, because they share the same authentication code.

Snowflake's REST API supports WIF: you send the cloud identity token as Authorization: Bearer WIF.<provider>.<token>, together with the header X-Snowflake-Authorization-Token-Type: WORKLOAD_IDENTITY_FEDERATION (Snowflake docs).

closes: #75808

What this changes

A WIF connection now works for these hooks. The fix is one new branch in SnowflakeRestTokenProvider (utils/_rest_auth.py), which all the REST hooks use.

  • How the token is obtained. The provider asks the cloud for an identity token using that cloud's own library:

    Provider Library How
    AWS boto3 sts:GetWebIdentityToken
    AZURE azure-identity AKS workload identity if the pod has one, otherwise the managed identity (MANAGED_IDENTITY_CLIENT_ID picks a user-assigned one)
    GCP google-auth ID token for the workload's service account
    OIDC none the connection's token or token_file_path
  • No new dependencies. Each library is imported only when a connection uses that cloud. If it's missing, the error names the package to install.

  • Existing behaviour is unchanged. The order is still OAuth, then PAT, then key-pair. A connection that has a private key keeps using key-pair, even if WIF is also configured. WIF is only used when nothing else applies.

  • Tokens are cached. The SQL API operator checks its status repeatedly, so the token is reused until 30 seconds before it expires, instead of calling the cloud every time.

  • Docs. The connection page now says what each provider needs installed.

Worth a look

  • The cloud libraries are not dependencies of the provider. Should boto3, azure-identity and google-auth be added as optional extras?
  • workload_identity_entra_resource and workload_identity_impersonation_path aren't passed through by SnowflakeHook today, so they aren't supported here either.

How it was tested

Unit tests cover each cloud's token helper (with fake cloud libraries), OIDC from a token or a file, caching and refresh near expiry, missing libraries, and that key-pair still wins. They also check the headers from SnowflakeSqlApiHook.get_headers() and a Cortex Agent request. 171 tests pass, and ruff and mypy are clean on the changed code.

Live test on Azure: an Azure Container Apps job, running this branch with a managed identity and no stored secret, ran a query through SnowflakeSqlApiHook against a real Snowflake account. The token came from azure-identity, the SQL API accepted Bearer WIF.AZURE.<token>, and select current_user(), current_role() finished successfully.

Not yet tested live: AKS workload identity, AWS, GCP and OIDC (unit tests only).

Two notes on the setup: I ran the tests in a scratch environment because the repo needs a newer uv than mine, and I skipped the mypy-devel-common pre-commit hook, which can't run on that uv; CI will cover it.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Code (Claude Sonnet 5.5) following the guidelines

@boring-cyborg boring-cyborg Bot added area:providers kind:documentation provider:snowflake Issues related to Snowflake provider labels Oct 27, 2026
@omar-massfih
omar-massfih marked this pull request as ready for review October 27, 2026 17:58
@omar-massfih
omar-massfih requested a review from potiuk as a code owner October 27, 2026 17:58
auto-merge was automatically disabled November 4, 2026 18:35

Head branch was pushed to by a user without write access

@omar-massfih
omar-massfih requested a review from kaxil November 4, 2026 18:47
@kaxil
kaxil enabled auto-merge (squash) November 4, 2026 21:11
…tion

SnowflakeSqlApiHook, SnowflakeCortexAgentHook and the pydanticai_snowflake
connection type rejected connections that use Workload Identity Federation,
even though SnowflakeHook supports them. Authenticate those REST calls with
the workload's cloud identity token, sent as `Bearer WIF.<provider>.<token>`
with the WORKLOAD_IDENTITY_FEDERATION token type, and cache it until shortly
before it expires. A connection that also sets a private key still uses
key-pair authentication.

closes: apache#75808
omar-massfih and others added 6 commits November 4, 2026 21:13
…ibraries

Do not call snowflake.connector.wif_util, which is not a documented API of
the connector. Request the identity token directly: sts:GetWebIdentityToken
through boto3 for AWS, azure-identity for Azure (AKS workload identity or the
managed identity), google-auth for GCP, and the connection's token or
token_file_path for OIDC. Each library is imported only for the provider the
connection names, so none becomes a dependency of the provider.

This also drops the snowflake-connector-python>=4.7.1 requirement for AWS.
WIF connections must report incorrect authenticator settings clearly, and
credential selection and tests must reflect the authentication path actually
used by the REST hooks.

Generated-by: OpenAI Codex
…ooks

Ask STS for a 3600 s GetWebIdentityToken lifetime instead of the 300 s default, so the
deferred SQL API trigger refreshes the token on its event loop about once an hour rather
than every few minutes. List WIF in the SnowflakeCortexAgentHook and SnowflakeSqlApiHook
docstrings, name GetWebIdentityToken in the global-endpoint note, and test a lowercase
authenticator value.
@kaxil
kaxil disabled auto-merge November 5, 2026 14:53
@kaxil
kaxil merged commit b7ceab0 into apache:main Nov 5, 2026
77 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SnowflakeSqlApiOperator: support Workload Identity Federation (WIF) authentication

2 participants