Repository navigation
'cause' key need to be limited to 255 characters to avoid failures due to trigger_reason field #34676
Description
Apache Airflow version
2.7.1
What happened
In the dbt Airflow provider code (referenced below), a field called trigger_reason is mapped to the JSON cause key:value pair, that you would directly set in Postman.
When trigger_reason is not set in Airflow (None) it defaults to creating a calculated string:
if self.trigger_reason is None:
self.trigger_reason = (
f"Triggered via Apache Airflow by task {self.task_id!r} in the {self.dag.dag_id} DAG."
)I'm going to guess that in this case, the trigger_reason isn't set in the airflow task, and therefore the task_id and the dag_id are both so long (?) that they are causing this calculated string to exceed 255 characters.
That being said, we should handle this better and truncate it at 254 regardless.
What you think should happen instead
Basically the summary is that on this line;
cause needs to be limited to 255 characters
How to reproduce
the trigger_reason creating a calculate string
def trigger_job_run(
self,
job_id: int,
cause: str,
account_id: Optional[int] = None,
steps_override: Optional[List[str]] = None,
schema_override: Optional[str] = None,
additional_run_config: Optional[Dict[str, Any]] = None,
) -> Response:
"""
Triggers a run of a dbt Cloud job.
:param job_id: The ID of a dbt Cloud job.
:param cause: Description of the reason to trigger the job.[2023-09-26, 08:47:38 EDT] {{http.py:161}} ERROR - HTTP error: Bad Request
[2023-09-26, 08:47:38 EDT] {{http.py:162}} ERROR - {"status":{"code":400,"is_success":false,"user_message":"The request was invalid. Please double check the provided data and try again.","developer_message":""},"data":{"cause":"must be between 0 and 255 characters"}}Operating System
unknown
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
boring-cyborg commentedon Sep 28, 2023
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
eladkal commentedon Sep 29, 2023
cc @josh-fell
uroy-personal commentedon Sep 29, 2023
/assign @uroy-personal
uroy-personal commentedon Oct 10, 2023
This change is in progress. Will raise the PR soon for review.
eladkal commentedon Mar 30, 2024
@uroy-personal are you still working on this issue?
uroy-personal commentedon Apr 2, 2024
Hi @eladkal ,
I worked on this and then lost track due to few issues on my side. I will finish it hopefully by this week. Is that okay?
boraberke commentedon Apr 10, 2024
Hi @uroy-personal, if you have stopped working on this issue, I can take over.
uroy-personal commentedon Apr 10, 2024
Hi @boraberke ,
Yes you can take over. I am unassigning myself. Thanks.