Repository navigation
Status of testing Providers that were prepared on May 26, 2024 #39842
Description
Body
I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?
The guidelines on how to test providers can be found in
Verify providers by contributors
Let us know in the comment, whether the issue is addressed.
Those are providers that require testing as there were some substantial changes introduced:
Provider alibaba: 2.8.1rc1
Provider amazon: 8.23.0rc1
- Amazon Bedrock - Retrieve and RetrieveAndGenerate (#39500): @ferruzzi
- Introduce Amazon Comprehend Service (#39592): @gopidesupavan
- [OpenLineage] fix: empty dataset name for AthenaOperator (#39677): @kacpermuda
- Fix default value for aws batch operator retry strategy (#39608): @0x26res
- Sagemaker trigger: pass the job name as part of the event (#39671): @vincbeck
Linked issues: - Handle task adoption for batch executor (#39590): @vincbeck
Linked issues: - NeptuneStopDbClusterOperator - Handle invalid cluster states (#38287): @ellisms
Linked issues: - Fix automatic termination issue in EmrOperator by ensuring
waiter_max_attemptsis set for deferrable triggers (#38658): @beobest2 - Resolving EMR deprecated warnings (#39743): @dirrao
Linked issues:
Provider apache.beam: 5.7.1rc1
Provider apache.druid: 3.10.1rc1
- Pass SSL arg to all requests in DruidOperator (#39066): @danielbe11
Linked issues:
Provider apache.hive: 8.1.1rc1
Provider apache.kafka: 1.4.1rc1
Provider apprise: 1.3.1rc1
Provider atlassian.jira: 2.6.1rc1
Provider celery: 3.7.1rc1
- ECS Executor: Set tasks to RUNNING state once active (#39212): @o-nikolas
- Remove compat code for 2.7.0 - its now the min Airflow version (#39591): @jedcunningham
Provider cncf.kubernetes: 8.3.0rc1
- Add timeout when watching pod events in k8s executor (#39551): @dstandish
- Add retry logic for KubernetesCreateResourceOperator and KubernetesJobOperator (#39201): @MaksYermak
- Fix deprecated calls in
cncf.kubernetesprovider (#39381): @Taragolis
Linked issues: - Move Kubernetes cli to provider package (#39587): @romsharon98
- Remove compat code for 2.7.0 - its now the min Airflow version (#39591): @jedcunningham
- Replace pod_manager.read_pod_logs with client.read_namespaced_pod_log in KubernetesPodOperator._write_logs (#39112): @Lee-W
Provider common.io: 1.3.2rc1
- [openlineage] fix: OpenLineage in FileTransferOperator for Airflow 2.8 (#39755): @kacpermuda
Linked issues:
Provider common.sql: 1.14.0rc1
- Add
parametersas template field for SqlSensor (#39588): @josh-fell - DbAPiHook: Don't log a warning message if placeholder is None and make sure warning message is formatted correctly (#39690): @dabla
Linked issues: - DbAPiHook executemany parameter of insert_rows should not be deprecated (#39630): @dabla
- Add typing for SqlSensor (#39773): @josh-fell
Provider databricks: 6.5.0rc1
- add deferrable support to
DatabricksNotebookOperator(#39295): @rawwar
Linked issues: - [FEAT] get all failed tasks errors in exception (#39354): @gaurav7261
- Better typing for BaseOperator
defer(#39742): @jedcunningham
Provider dbt.cloud: 3.8.1rc1
- Scheduler to handle incrementing of try_number (#39336): @dstandish
- Validate dbt
trigger_reasonfield to be less than 255 characters (#38896): @boraberke - Add (optional) dependency between dbt-cloud and openlineage providers (#39366): @tatiana
Provider docker: 3.12.0rc1
- Add args to docker service ContainerSpec (#39464): @guydriesen
- Add support to define Resources on DockerSwarmOperator (#39027): @guydriesen
- Limit requests in botocore upgrade test (#39747): @potiuk
Linked issues: - Pin requests due to incompatibility with docker-py (#39740): @eladkal
Provider elasticsearch: 5.4.1rc1
- Make elastic search index_pattern more configurable (#38423): @pankajastro
- Scheduler to handle incrementing of try_number (#39336): @dstandish
Provider fab: 1.1.1rc1
- Simplify action name retrieval in FAB auth manager (#39358): @jedcunningham
- Add
jmespathas an explicit dependency (#39350): @jedcunningham
Provider google: 10.19.0rc1
- Introduce anonymous credentials in GCP base hook (#39695): @shahar1
Linked issues: - Remove parent_model version suffix if passed to Vertex AI operators (#39640): @e-galan
Linked issues: - Fix BigQueryCursor execute method if the location is missing (#39659): @e-galan
- Fix acknowledged functionality in deferrable mode for PubSubPullSensor (#39711): @MaksYermak
- Reroute AutoML operator links to Google Translation links (#39668): @e-galan
- Pin google-cloud-bigquery to < 3.21.0 (#39583): @pankajastro
- openlineage: remove
openlineage.commondependencies in Google and Snowflake providers. (#39614): @JDarDagran - Deprecate AutoML Tables operators (#39752): @e-galan
Linked issues: - Resolve deprecation warnings in Azure FileShare-to-GCS tests (#39599): @josh-fell
Linked issues: - [openlineage] typo: wrong OpenLineage facet key in spec (#39782): @kacpermuda
- removed stale code (#39744): @msoni1369
Provider hashicorp: 3.7.1rc1
- inherit key "verify" from env into session so that kv_client can read it properly (#38614): @chz8494
Provider microsoft.mssql: 3.7.1rc1
- use
cached_propertyto reuseget_connectionsvalue inmssqlprovider (#39575): @rawwar
Linked issues: - remove extra return in
get_connof mssql provider (#39582): @rawwar
Provider openlineage: 1.8.0rc1
- Scheduler to handle incrementing of try_number (#39336): @dstandish
- [openlineage] fix: Prevent error when extractor can't be imported (#39736): @kacpermuda
Linked issues: - openlineage: improve spawned OpenLineage process in scheduler (#39735): @JDarDagran
Linked issues: - [openlineage] Update conf retrieval docstring and adjust pool_size (#39721): @kacpermuda
- openlineage: remove
openlineage.commondependencies in Google and Snowflake providers. (#39614): @JDarDagran - openlineage: use
ProcessPoolExecutoroverThreadPoolExecutor. (#39235): @JDarDagran
Linked issues: - feat: Add custom provider runtime checks (#39609): @kacpermuda
Linked issues: - openlineage: notify that logged exception was caught (#39493): @mobuchowski
- [OpenLineage] Add more debug logs to facilitate debugging (#39136): @kacpermuda
- openlineage: add missing
dag_state_change_process_pool_sizeinprovider.yaml. (#39674): @JDarDagran
Linked issues:
Provider pinecone: 2.0.0rc3
- Pinecone provider support for
pinecone-client>=3 (#37307): @rawwar
Linked issues: - Fix the pinecone system test (#39365): @sunank200
- Fix the argument type of input_vectors in pinecone upsert (#39688): @sunank200
Linked issues:
Provider postgres: 5.11.1rc1
- fix: use
sqlalchemy_urlproperty inget_urifor postgresql provider (#38831): @rawwar
Linked issues:
Provider sftp: 4.10.1rc1
- Fix SFTPSensor.newer_than not working with jinja logical ds/ts expression (#39056): @grrolland
Linked issues:
Provider snowflake: 5.5.1rc1
- openlineage: remove
openlineage.commondependencies in Google and Snowflake providers. (#39614): @JDarDagran - Remove unused
copy_into_postifxparam from docstring (#39454): @josh-fell
Provider yandex: 3.11.1rc1
- AIP-21: yandexcloud: rename files, emit deprecation warning (#39618): @uzhastik
- yandex provider: bump version for yq http client package (#39548): @uzhastik
- add doc about Yandex Query operator (#39445): @uzhastik
All users involved in the PRs:
@pankajastro @dabla @sunank200 @e-galan @shahar1 @boraberke @JDarDagran @0x26res @ellisms @EricGao888 @grrolland @danielbe11 @tatiana @kacpermuda @gaurav7261 @gopidesupavan @mobuchowski @Lee-W @eladkal
Committer
- I acknowledge that I am a maintainer/committer of the Apache Airflow project.
Activity
uzhastik commentedon May 26, 2024
Looks good, this version has been checked on my test dag and old active airflow server.
Is it possible to preview changes in doc? I have one PR in doc.
gopidesupavan commentedon May 26, 2024
Hi @eladkal , i have tested #39592 , it is working fine. thanks.
potiuk commentedon May 26, 2024
All my changes checked.
shahar1 commentedon May 26, 2024
#39695 is working fine
Updating setup.cfg with RC provider packages on apache/airflow#39842
Updating setup.cfg with RC provider packages on apache/airflow#39842
EricGao888 commentedon May 27, 2024
Hi @eladkal , I've tested #39627 and it is working fine. THX : )
grrolland commentedon May 27, 2024
Updating setup.cfg with RC provider packages on apache/airflow#39842
Updating setup.cfg with RC provider packages on apache/airflow#39842
Updating setup.cfg with RC provider packages on apache/airflow#39842
Updating setup.cfg with RC provider packages on apache/airflow#39842
Updating setup.cfg with RC provider packages on apache/airflow#39842
11 remaining items
VladaZakharova commentedon May 28, 2024
Changes #39711 and #39371 @MaksYermak work well
pankajastro commentedon May 28, 2024
#38423 and #39583 looks good. thank you!
The issue we were trying to capture for no longer happens in the late…
boraberke commentedon May 28, 2024
#38896 looks good. Thanks!
Fix CI issues (#1005)
⬆ [pre-commit.ci] pre-commit autoupdate (#1000)
Joffreybvn commentedon May 29, 2024
#39690 is fine - No more warning logs when executing a SQL query !
msoni1369 commentedon May 29, 2024
eladkal commentedon May 30, 2024
Thank you everyone. Providers are released.
Provider
cncf.kubernetesis excluded and will have rc2 shortlyI invite everyone to help improve providers for the next release, a list of open issues can be found here.
danielbe11 commentedon Jun 5, 2024
I see it's closed, but all good with me too. Thanks!
Fix CI issues (#1005)
⬆ [pre-commit.ci] pre-commit autoupdate (#1000)
Fix CI issues (astronomer#1005)
⬆ [pre-commit.ci] pre-commit autoupdate (astronomer#1000)