Skip to content

Status of testing Providers that were prepared on December 08, 2023 #36117

Closed

Description

@eladkal

Body

I have a kind request for all the contributors to the latest provider packages release.
Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comment, whether the issue is addressed.

Those are providers that require testing as there were some substantial changes introduced:

Provider amazon: 8.13.0rc1

Provider cncf.kubernetes: 7.11.0rc1

Provider docker: 3.9.0rc1

Provider elasticsearch: 5.3.0rc1

Provider google: 10.13.0rc1

Provider http: 4.8.0rc1

Provider jdbc: 4.2.0rc1

Provider microsoft.azure: 8.4.0rc1

Provider microsoft.mssql: 3.6.0rc1

Provider openai: 1.1.0rc1

Provider openlineage: 1.3.0rc1

Provider trino: 5.5.0rc1

Provider weaviate: 1.1.0rc1

All users involved in the PRs:
@Taragolis @vincbeck @romsharon98 @syedahsn @kacpermuda @raphaelauv @melugoyal @ginolegigot @atrbgithub @flolas @JDarDagran @nathadfield @potiuk @ephraimbuddy @Joffreybvn @pankajkoti @hussein-awala
@VladaZakharova @stavdav143 @eladkal

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.

Activity

  1. added
    kind:metaHigh-level information important to the community
    on Dec 7, 2023
  2. melugoyal commented on Dec 8, 2023

    @melugoyal

    #35694 working as expected

  3. atrbgithub commented on Dec 8, 2023

    @atrbgithub
    Contributor

    An issue has been raised with #34919

    Unfortunately it looks like it has broken GCSObjectsWithPrefixExistenceSensor.

    Testing myself I can confirm that the following code is able to create the error:

    from airflow.providers.google.cloud.hooks.gcs import GCSHook
    res = GCSHook().list(
            bucket_name='a-testbucket',
            prefix='a/prefix/in/the/bucket/'
    )
    print(res)

    Please do not include this change.

    Edit: I've raised a PR #36130 which appears to fix this issue for me when testing locally.

  4. eladkal commented on Dec 9, 2023

    @eladkal
    ContributorAuthor

    Docker provider will be excluded from rc1

  5. hussein-awala commented on Dec 9, 2023

    @hussein-awala
    Member

    I tested my change on the Kubernetes executor, and most of the changes in the Amazon provider, all work as expected.

  6. pankajastro commented on Dec 9, 2023

    @pankajastro
    Member

    As @atrbgithub has mentioned above #34919 is breaking an existing DAG. so we might need to exclude Google provider

  7. eladkal commented on Dec 9, 2023

    @eladkal
    ContributorAuthor

    so we might need to exclude Google provider

    Yes. Google provider will also have RC2

  8. potiuk commented on Dec 10, 2023

    @potiuk
    Member

    Tested mssql - all looks good.

    One more thing I noticed - the removed daskexecutor has not been prepared. We have this in the release process, so it's easy to miss:

    NOTE! When you want to release a provider marked for removal (needed in order to prepare last release of the provider), package for the provider will not be prepared when you prepare documentation for all providers - you have to specifically use the provider name in a separate command. For example to prepare documentation for removed.provider provider marked for removal you need to run separately this command:
    
    breeze release-management prepare-provider-packages --package-format both removed.provider
    

    But maybye we will change it in the future - we have more removed providers coming. So possibly we shoudl automatically add removed providers to the list of providers prepared (both for documentation and package building - so that we do not have to remember to prepare them. That was probably bad idea to not add them in the first place.

    @eladkal -> I guess we should include rc2 (even if there was no rc1) for daskexecutor.

  9. potiuk commented on Dec 10, 2023

    @potiuk
    Member

    One small update - as @pankajastro noticed in #32319 (comment) - there is a case where we are not handling properly the fact that handler can return either list of rows or row - in both ODBC and DAtabricks oviders, so IMHO they should also be moved to RC2 (cc: @Joffreybvn )

  10. potiuk commented on Dec 10, 2023

    @potiuk
    Member

    Fix to the process to automatically generate removed providers here: #36160

  11. Joffreybvn commented on Dec 11, 2023

    @Joffreybvn
    Contributor

    Fix for #32319 (comment) is here -> #36161

  12. nathadfield commented on Dec 11, 2023

    @nathadfield
    Contributor

    #36072 has been tested and working as expected.

  13. eladkal commented on Dec 12, 2023

    @eladkal
    ContributorAuthor

    Providers: Google, ODBC, Databricks, Docker are excluded from RC1.

  14. eladkal commented on Dec 12, 2023

    @eladkal
    ContributorAuthor

    Thank you everyone.
    Providers are released (excluding Google, ODBC, Databricks, Docker which we will have RC2 soon)
    I invite everyone to help improve providers for the next release, a list of open issues can be found here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind:metaHigh-level information important to the communitytesting statusStatus of testing releases

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions