Repository navigation
Status of testing Providers that were prepared on November 24, 2024 #44324
Description
Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
Provider celery: 3.8.5rc1
Provider cncf.kubernetes: 10.0.1rc1
- Bugfix KubernetesJobOperator.on_kill() (#44131): @moiseenkov
- Allow for retry when tasks are stuck in queued (#43520): @dimberman
- Rename execution_date to logical_date across codebase (#43902): @sunank200
Provider google: 11.0.0rc1
- Remove deprecated functionality from Google provider (#43953): @moiseenkov
- feat: add OpenLineage support for BigQueryToBigQueryOperator (#44214): @kacpermuda
- Introduce gcp advance (V3) API translate native datasets operators (#44271): @olegkachur-e
- Introduce new gcp TranslateText and TranslateTextBatch operators (#43860): @olegkachur-e
- Add gcloud command to DataprocCreateClusterOperator to be able to create dataproc on GKE cluster (#44185): @MaksYermak
- Fix incorrect query in
BigQueryAsyncHook.create_job_for_partition_get(#44225): @sean-rose
Linked issues: - Fix Dataplex Data Quality partial update (#44262): @amirmor1
- Bump
google-cloud-translateto3.16(#44297): @kaxil - fix google datacatalog operator tests (#44281): @gopidesupavan
- Rename execution_date to logical_date across codebase (#43902): @sunank200
Provider standard: 0.0.2rc1
- Fix TriggerDagRunOperator extra_link when trigger_dag_id is templated (#42810): @fredthomsen
- Move
TriggerDagRunOperatorto standard provider (#44053): @hardeybisey
Linked issues: - Move filesystem sensor to standard provider (#43890): @kunaljubce
Linked issues: - Rename execution_date to logical_date across codebase (#43902): @sunank200
All users involved in the PRs:
@amirmor1 @dimberman @hardeybisey @fredthomsen @MaksYermak @sunank200 @kunaljubce @kacpermuda @sean-rose @kaxil @olegkachur-e @moiseenkov @gopidesupavan
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Activity
moiseenkov commentedon Nov 25, 2024
Hi,
#44131 and #43953 look good
hardeybisey commentedon Nov 25, 2024
Hi, #44053 works as expected.
fredthomsen commentedon Nov 25, 2024
#42810 is good.
kacpermuda commentedon Nov 25, 2024
#44214 is good
amirmor1 commentedon Nov 25, 2024
#44262 works as expected
sean-rose commentedon Nov 26, 2024
I verified that #44225 works.
kunaljubce commentedon Nov 27, 2024
@eladkal So I verified that #43890 works with
apache-airflow>=2.10.0-However, there's an error when I do the same with
apache-airflow==2.9.3-Unless I am doing something wrong, in which case please let me know, it seems that this is happening because of #41021 which seems to be linked to the airflow 2.10.0 milestone. If that's the case, maybe we should update the documentation for it on PyPi?
potiuk commentedon Nov 27, 2024
Good catch. Apparently we miss compatibility class for that in "common.compat" and the sensor has no corresponding tests that would detect it in "providers/tests/standard/sensors" - the test for filesystem sensor is still in "tests/sensors" that's why our test suite did not catch it.
Nope. We want to keep 2.8+ compatibility. We should add compatibilty
StartTriggerArgsclass in "common.compat" similarly to other "after 2.8" features, and import the class from there in the sensor.I guess @eladkal - since it's not a regression and standard provider is still 0.* it's not a blocker for the release and can be fixed as a follow-up.
eladkal commentedon Nov 27, 2024
@potiuk is correct, we don't consider this as a regression as standard provider is not yet "stable".
Specifically for 2.9.x the sensor is reachable by importing from core path. The PR in question just moved it from core to providers so while it doesn't work as intended there is simple workaround for 2.9
eladkal commentedon Nov 27, 2024
@kunaljubce if you can raise a PR with a fix that would be much appreciated
eladkal commentedon Nov 27, 2024
Thank you everyone. Providers are released.
I invite everyone to help improve providers for the next release, a list of open issues can be found here.