Repository navigation
Conversation
| @@ -0,0 +1 @@ | |||
| Restore the ``previously_succeeded`` metric, which stopped being emitted when Airflow 3 workers moved to the Task SDK. It is now counted when a worker is handed a task instance that already succeeded and the Execution API refuses to start it. | |||
There was a problem hiding this comment.
Thanks for the PR
I think we can drop this newsfragment.
This PR restores an existing metric, so it looks like an ordinary bug fix, which does not require a newsfragment.
fat-catTW
left a comment
There was a problem hiding this comment.
Also, I think we could add test for the metric tags with dag_tags_in_metrics=True and a team-scoped Dag?
|
Thanks for the review @fat-catTW!
|
fat-catTW
left a comment
There was a problem hiding this comment.
Thanks for the update!
Look nice.
luc-pimentel
left a comment
There was a problem hiding this comment.
Thanks for picking up the issue I opened. I ran its reproduction against this PR's current head with airflow standalone on Python 3.15, SQLite and the LocalExecutor, with StatsD pointed at a local UDP listener. It runs a one-task Dag once, then hands the succeeded task instance back to the executor's own launch path, so ti_run gets a second start request.
- First run:
ti.start,ti.finishandti_successesarrive, andpreviously_succeededis not counted. - Second start of the succeeded task:
ti_runstill answers 409invalid_statewithprevious_state: success, andpreviously_succeededarrives once. On the same setup without this change, nothing is counted.
I also have a unit test written for the report: a succeeded task instance, one PATCH /execution/task-instances/{id}/run, and an assert on previously_succeeded with the task instance's stats_tags. It fails without this change, as it should, and passes on this branch.
It seems like this fixes what I reported. Curious what the maintainers think of the approach.
Drafted-by: Claude Code 2.1.317 (Claude Opus 5.5); reviewed by @luc-pimentel before posting
previously_succeeded("Number of previously succeeded task instances") is listed in the metrics registry and docs, but its only emit site isTaskInstance._check_and_change_state_before_execution, which nothing calls in Airflow 3. Workers move a task to running through theti_runExecution API route instead, and when they are handed a task instance that already succeeded,ti_runanswers 409invalid_statewithprevious_state: successwithout counting anything.This emits the counter from that 409 branch when the previous state is
success, following the approach #67592 used to restoretask.queued_durationin the same route. Tags matchTaskInstance.stats_tags(Dag run tags, including team and Dag tags when enabled, plustask_id). The Task SDK client only retries 5xx and connection errors, so each refused start is counted once.Tests: added
test_ti_run_previously_succeeded_metric, coveringsuccess(counted) andfailed/skipped/up_for_retry(409 but not counted). Thesuccesscase fails without the fix.closes: #75366
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Claude Opus 5.5) following the guidelines