Microsoft Azure Connection: Service principal secret does not get masked when providing connection via URI environment variable

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

apache-airflow-providers-microsoft-azure==9.0.0

Apache Airflow version

apache-airflow==2.8.1

Operating System

Ubuntu 22.04.3 LTS

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

What happened

I set the connection as follows:

export AIRFLOW_CONN_AZURE_DEFAULT='azure://<sp-appid>:<sp-secret>?tenantId=<my-tenant>&subscriptionId=<my-subscription>'

As sp-secret is a password, I would assume it is masked from task logs. However, this is not the case.

I see 2 possible reasons:

There is no concept of hostname (so no @fqdn), which the masking feature might filter on?
There are special characters in the password field. In my case, the following characters are used: UPPERCASE, lowercase, ~ - _

What you think should happen instead

The log should render:

AIRFLOW_CONN_AZURE_DEFAULT=azure://<sp-appid>:***?tenantId=<my-tenant>&subscriptionId=<my-subscription>'

just like it does with for example postgresql connection.

How to reproduce

Add the connection (does not even need to make sense) as environment variables:

export AIRFLOW_CONN_AZURE_DEFAULT='azure://<sp-appid>:<sp-secret>?tenantId=<my-tenant>&subscriptionId=<my-subscription>'

Create simple DAG to print environment variables

Anything else

I have tried adding a fake hostname x by modifying the string to

export AIRFLOW_CONN_AZURE_DEFAULT='azure://<sp-appid>:<sp-secret>@x?tenantId=<my-tenant>&subscriptionId=<my-subscription>'

This still prints the secret unmasked.

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Please fix this issue

Probably same issue as #36688, I'll try to see if that fix solves this issue.

@zwelgbeer

Looks like duplicate fixed in 2.8.2 @zwelgbeer - > Please upgrade your airflow to latest version and report back. Good idea is to upgrade to

@potiuk

@potiuk, I have had the ability to test this on Airflow 2.9.1, unfortunately, my secret is still not masked. I have 2 connection strings that are unmasked:

AIRFLOW_CONN_AZURE=azure://<AAD_CLIENT_ID>:UNMASKED_PASSWORD?tenantId=<AAD_TENANT_ID>&subscriptionId=<SUBSCRIPTION_ID>

AIRFLOW_CONN_DATABRICKS=databricks://<AAD_CLIENT_ID>:UNMASKED_PASSWORD@<workspace_url>.azuredatabricks.net?azure_tenant_id=<AAD_TENANT_ID>

Let me change some (normal) characters in the password:

pO51Q~MLdXXu-H_sOMNECHANGECHARSldk

It shows up unmasked when doing eg. printenv in a BashOperator.

@potiuk

Hi @potiuk i would like to contribute in this issue and it is also marked as good first issue , so can I work on it?

Sure

@w0ut0

Hi @w0ut0, does this issue still occur? I tried reproducing it with this DAG on Airflow 3.1.0 and couldn’t, so I suspect it’s been fixed.

from datetime import timedelta
import pendulum
from airflow.decorators import dag
from airflow.operators.bash import BashOperator
from airflow.hooks.base import BaseHook

@dag(
    dag_id="shell_test_all_providers",
    description="Test secret masking for Postgres, MySQL, Azure, Databricks",
    start_date=pendulum.datetime(2025, 1, 1),
    schedule=None,
    catchup=False,
    tags=["test"],
    default_args={
        "retries": 0,
        "retry_delay": timedelta(minutes=5),
    },
)
def shell_test_all_providers():
    pg_uri    = BaseHook.get_connection("DATAWAREHOUSE").get_uri()
    mysql_uri = BaseHook.get_connection("MYSQL_CONN").get_uri()
    azure_uri = BaseHook.get_connection("AZURE_CONN").get_uri()
    dbrk_uri  = BaseHook.get_connection("DATABRICKS_CONN").get_uri()

    mask_test = BashOperator(
        task_id="masking_demo_all",
        bash_command=(
            'echo "Postgres:     $PG_URI"\n'
            'echo "MySQL:        $MYSQL_URI"\n'
            'echo "Azure:        $AZURE_URI"\n'
            'echo "Databricks:   $DATABRICKS_URI"'
        ),
        env={
            "PG_URI": pg_uri,
            "MYSQL_URI": mysql_uri,
            "AZURE_URI": azure_uri,
            "DATABRICKS_URI": dbrk_uri,
        },
    )

    return mask_test

shell_test_all_providers()

Not able to reproduce with apache-airflow==2.9.1 and apache-airflow-providers-microsoft-azure==9.0.0. I've tested URIs and JSON versions, and in both cases the client secret is being masked correctly.

What I did notice though is that if you run a env or set command within the bash operator, some secrets get exposed through the logs. Should this be of concern?

@w0ut0

Hi @w0ut0, just a quick check: does this issue still exist? Otherwise I think this issue could be closed.

Navigation Menu

Repository navigation

Microsoft Azure Connection: Service principal secret does not get masked when providing connection via URI environment variable #38144

Apache Airflow Provider(s)

Versions of Apache Airflow Providers

Apache Airflow version

Operating System

Deployment

Deployment details

What happened

What you think should happen instead

How to reproduce

Anything else

Are you willing to submit PR?

Code of Conduct

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Microsoft Azure Connection: Service principal secret does not get masked when providing connection via URI environment variable #38144

Description

Apache Airflow Provider(s)

Versions of Apache Airflow Providers

Apache Airflow version

Operating System

Deployment

Deployment details

What happened

What you think should happen instead

How to reproduce

Anything else

Are you willing to submit PR?

Code of Conduct

Activity

boring-cyborg commented on Mar 14, 2024

zwelgbeer commented on Mar 14, 2024

w0ut0 commented on Mar 14, 2024

potiuk commented on Mar 14, 2024

w0ut0 commented on May 17, 2024

sonalshreya commented on Nov 14, 2024

potiuk commented on Nov 14, 2024

aaron-y-chen commented on May 17, 2025

geovalexis commented on Nov 13, 2025

aaron-y-chen commented on Oct 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions