[FrankYang0529]

Why

• Airflow drops every metric that carries a Dag tag with a space, colon or non-ASCII character instead of emitting it.

airflow/shared/observability/src/airflow_shared/observability/metrics/validators.py

Line 47 in a32662c

ALLOWED_CHARACTERS = frozenset(string.ascii_letters + string.digits + "_.-/")

airflow/shared/observability/src/airflow_shared/observability/metrics/validators.py

Lines 115 to 131 in a32662c

	def validate_stat(fn: Callable) -> Callable:
	"""Check if stat name contains invalid characters; logs and does not emit stats if name is invalid."""
	@wraps(fn)
	def wrapper(self, stat: str | None = None, *args, **kwargs) -> Callable | None:
	try:
	if stat is not None:
	handler_stat_name_func = get_current_handler_stat_name_func(
	self.stat_name_handler, self.statsd_influxdb_enabled
	)
	stat = handler_stat_name_func(stat)
	return fn(self, stat, *args, **kwargs)
	except InvalidStatsNameException:
	log.exception("Invalid stat name: %s.", stat)
	return None
	return cast("Callable", wrapper)

• [metrics] dag_tags_in_metrics turns Dag tags into metric tags. DagTag.name validates length only, so any character reaches the StatsD wire.
• With statsd_influxdb_enabled, prepare_stat_with_tags appends those tags to the stat name before validate_stat sees it. The validator then rejects the whole metric, not just the invalid tag.

How

• Normalize both halves of each Dag tag in build_dag_metric_tags() with normalize_name_for_stats.

Verification

• Unit test: uv run --project shared/observability pytest shared/observability/tests/observability/metrics/test_stats.py -xvs
• Integration test

1. Add a Dag with tags.

mkdir -p files/dags && cat > files/dags/dag_with_tag.py <<'PY'
from datetime import datetime

from airflow.sdk import DAG, task

with DAG(
    "dag_with_tag",
    schedule=None,
    start_date=datetime(2025, 1, 1),
    tags=["example", "edge", "integration test"],
):
    @task
    def noop():
        return 1

    noop()
PY

2. Start scheduler

breeze shell --backend sqlite --integration statsd

In container:

export AIRFLOW__METRICS__STATSD_ON=True
export AIRFLOW__METRICS__STATSD_HOST=statsd-exporter
export AIRFLOW__METRICS__STATSD_PORT=9125
export AIRFLOW__METRICS__STATSD_INFLUXDB_ENABLED=True
export AIRFLOW__METRICS__DAG_TAGS_IN_METRICS=True

airflow db reset -y && airflow dags reserialize && airflow scheduler &

3. Trigger the Dag

airflow dags unpause dag_with_tag
airflow dags trigger dag_with_tag

On main branch, the scheduler will show following error, but this doesn't happen in this branch.

2026-09-01T06:46:29.759401Z [error    ] Invalid stat name: dagrun.dependency-check.dag_with_tag,integration test=true,example=true,edge=true,run_type=manual. [airflow._shared.observability.metrics.validators] loc=validators.py:128
Traceback (most recent call last):
  File "/opt/airflow/airflow-core/src/airflow/_shared/observability/metrics/validators.py", line 125, in wrapper
    stat = handler_stat_name_func(stat)
  File "/opt/airflow/airflow-core/src/airflow/_shared/observability/metrics/validators.py", line 213, in stat_name_default_handler
    raise InvalidStatsNameException(
airflow._shared.observability.exceptions.InvalidStatsNameException: The stat name (dagrun.dependency-check.dag_with_tag,integration test=true,example=true,edge=true,run_type=manual) has to be composed of ASCII alphabets, numbers, or the underscore, dot, or dash characters.

4. Check metrics exist

curl -s http://statsd-exporter:9102/metrics | grep airflow_dagrun

• related: Optionally emit Dag tags as metric tags #68568

---

Was generative AI tooling used to co-author this PR?

• Yes - Claude Code

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[potiuk]

nice!

[github-actions]

Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to Airflow 3.3.2 based on: backport label targeting v3-3-test
If this milestone is not correct, please update it to the appropriate milestone.

This comment was generated by Milestone Tag Assistant.

[potiuk]

Not backported - it is in 3.4.0 upcoming functionality

[realtimetodie]

@potiuk I think this change is generally badly designed and implemented at the wrong layer.

build_dag_metric_tags() is shared by multiple metric backends, but #72364 applies StatsD-specific normalization there. That means the same normalization also affects OpenTelemetry metrics, even though OpenTelemetry attributes do not have the same character restrictions.

This is also lossy. A tag such as env:pro d becomes env=pro_d, and non-ASCII characters are replaced as well. Once that happens, the original value cannot be recovered by the OpenTelemetry exporter.

The StatsD problem is real, but the normalization should happen in the StatsD/InfluxDB-specific path, not in the shared DAG tag representation used by every other metrics backend.

[realtimetodie]

I applied a fix for this in #72458.

The change now keeps build_dag_metric_tags() for StatsD/Datadog, while OpenTelemetry metrics use the generic expand_dag_tags() path so the original DAG tag values are preserved.

This keeps the StatsD fix from leaking into OpenTelemetry metric attributes.

[FrankYang0529]

@realtimetodie Thanks for the suggestion. I agree this should not do normalization for all backends. We can do it in prepare_stat_with_tags instead. I leave comment in #72458 (comment).

[realtimetodie]

@FrankYang0529 @potiuk

#72458, which contains the fix, has already been approved by @xBis7. Could someone please approve the workflows? Thank you! 🙌

[FrankYang0529]

I don't have permission to do it, either. Let's wait for other committers.