Repository navigation
Allow Human-in-the-loop tasks to be answered while still running - #75413
Merged
Merged
Conversation
kaxil
requested review from
XD-DENG,
amoghrajesh,
ashb,
bbovenzi,
bugraoz93,
choo121600,
ephraimbuddy,
guan404ming,
henry3260,
jason810496,
pierrejeambrun,
rawwar,
ryanahamilton,
shubhamraj-git and
vatsrahul1001
as code owners
October 22, 2026 14:52
kaxil
marked this pull request as draft
October 22, 2026 14:52
Contributor
|
So the task state won't always be the orange "awaiting input"? We'll need to make sure the UX shows a user that there is a pending action. Should I help on that front as a follow up PR? |
A running task could write its Human-in-the-loop request but not read the answer: the task supervisor served CreateHITLDetailPayload and rejected GetHITLDetailResponse as an unhandled request, since only the triggerer needed it. Serve it from the task supervisor too, with a HITLDetailResponseResult reply type in the Task SDK comms. The Execution API route is already scoped to ti:self, so a task can only read its own response. The UI and the pending-actions queries treated a HITL task as waiting for a response only while it was deferred or awaiting_input. A task that checks for its answer before parking is running at that point, so count it as waiting too, but only while the attempt that wrote the request runs it for the first time: an attempt that started after the request was written has not asked yet, and a resumed run (for example a timeout without defaults) has stopped waiting. The response endpoint refuses answers to a running task in those two cases, which it used to accept and then lose.
…ker's clock A running task instance's unanswered request is always its own, since every attempt gets a new task instance id. So a running task waits on its first run, or on a resumed run that asked again after the server resumed it: compare created_at with scheduled_dttm, both stamped by the server, instead of with the worker's start_date. That keeps refusing a run resumed after a timeout, and accepts an operator that asks again from a resumed run. The response endpoint resumes only a parked task; a running one reads the answer itself. The supervisor and the triggerer share one handler and one HITLDetailResponseResult, since core requires the Task SDK that ships it. List the new supervisor body under 2027-02-28, the in-progress version, since 2026-10-30 ships in 3.4.
The Task SDK client retries POST /hitlDetails/{ti_id}, and the endpoint
rewrites the request and clears any stored response. Now that a running
task can be answered, a reviewer could answer the request the first
attempt stored, get a 200, and then lose the answer to the client's
retry. The client now sends an Idempotency-Key per request, reused by
its retries; the endpoint stores it and returns the stored request
unchanged when a POST repeats it.
kaxil
force-pushed
the
hitl-running-task-response
branch
from
November 4, 2026 18:41
8d50460 to
f1565a4
Compare
Member
Author
|
@bbovenzi Yeah, during |
kaxil
marked this pull request as ready for review
November 4, 2026 19:25
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Human-in-the-loop task can write its request from a running task, but it cannot read the answer: the task supervisor rejects
GetHITLDetailResponsewith400 Unhandled request. This PR lets a running task read its own response, and makes the response endpoint, the Dags list and the UI treat a task that is still running while it checks for the answer as waiting. An operator can then pick up a quick answer before it parks inawaiting_input, without a trip through the scheduler. The provider side (response_poll_windowon the HITL operators) is the follow-up PR stacked on this one.Design rationale
The supervisor serves
GetHITLDetailResponsethroughhandle_get_hitl_detail_responseinrequest_handlers.py, and the triggerer now uses the same handler and the sameHITLDetailResponseResultinstead of its own copy. Core requires Task SDK>=1.5.0, which ships the class; a backport to 3.4 would have to keep the triggerer's copy. The Execution API route is already scoped toti:self, so a task can only read its own response.When does a running task count as waiting? Every attempt gets a new task instance id, so a running task's request is always the one it wrote, and a retry can't be mistaken for the attempt that asked. It waits on its first run (
next_methodunset), or on a resumed run that asked again after the server resumed it, meaning the request'screated_atis at or afterscheduled_dttm, which every resume path stamps. A run resumed after a timeout without defaults isn't waiting, so #75645's refusal still holds for it. Both timestamps come from the server; the worker's clock never enters into it.HITLDetail.is_awaited_by()is the Python form andawaiting_response_clause()the SQL one, used by the Dags list "needs review" filter and its pending actions.The response endpoint accepts an answer while the task waits by that rule and returns
409otherwise, as #75645 does for a task no longer waiting. For a running task it only stores the answer: the task reads it itself, and if it parks first, the park transition resumes it with the answer. Only a parked task gets resumed by the endpoint.Why the new
hitl_detail.idempotency_keycolumn? The Task SDK client retriesPOST /hitlDetails/{ti_id}, and the endpoint rewrites the request and clears any stored response. Once a running task can be answered, a reviewer could answer the request the first attempt stored, get a200, and lose the answer to the retry. The client now sends anIdempotency-Keyheader per request (its retries reuse it), the endpoint stores it, and a POST that repeats it gets the stored request back unchanged. Content can't stand in for the key, since a resumed run may legitimately ask the same question again. I put it in a header rather than the request body so the request model, the supervisor schema and the Go and TypeScript SDKs stay as they are. An older client sends no key and behaves as today.In the UI, the Required Actions list, the needs-review button and the tab count request and count
runningrows alongsidedeferredandawaiting_input. The UI has nonext_method, so while a run resumed after a timeout is failing (a few seconds) it can still show the form; the server refuses that answer.HITLDetailResponseResultis a new supervisor body, soschema.jsonand the vendored Go and TypeScript copies are regenerated. It's listed inv2027_02_28.py, the in-progress version.Known issues
created_atcomes from the API server andscheduled_dttmfrom whichever process resumed the task (the scheduler, for a timeout), so the rule still compares two server clocks. The two writes are a full resume and re-ask apart.{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.