Support alternative GCP universes #1142
Conversation
S3NS is a new deployment of GCP, with operations delegated by Google to Thales: https://www.s3ns.io/en S3NS is officially supported by Google, but is totally separate from the regular GCP, with notably its own API domains. We're porting our platform to S3NS, which includes CNPG configured with the Barman Cloud plugin for backups, which doesn't work there right now because we can't configure the Google *universe*. To connect to S3NS, official clients need to be configured with a universe value, that overrides the base API domain. `GOOGLE_CLOUD_UNIVERSE_DOMAIN` is the official environment variable used to target S3NS: https://documentation.s3ns.fr/docs/overview/tpc-key-differences#key_differences_for_developers Unfortunately Barman uses the v1 client that doesn't read this variable, and only accepts the universe as part of the `client_options` field in the constructor, so we have to pass it in like this.
|
This is blocking us from deploying on S3NS, since without backups are DBs are nothing 🙂 Happy to discuss this further. S3NS is very new (they went GA less than two months ago) but will certainly grow in usage. Note that the v2 Storage client reads the environment variable by default: https://github.com/googleapis/python-storage/blob/f7095faf0a81239894ff9d277849788b62eb6ac5/google/cloud/_storage_v2/services/storage/client.py#L464 Switching to it would of course fix the issue, but the fix in this PR should be enough to support S3NS without a big upgrade. |
mgarstecki
changed the title
|
@mgarstecki, thanks for your PR. |
|
@gcalacoci happy new year ! I pushed a fix for the linter issues 🙂 Do you need anything else for the PR to be merged ? |
|
We'll be porting this to our internal pipe-line. Expect this in the following release. |
|
Thank you ! |
|
@martinmarques what is the expected timeline for the release ? |
3 participants
In multiple countries, local hosting companies are partnering with GCP to offer EU-sovereign GCP-like environments. Such partnerships include one with T-Systems in Germany or one with Thales in France: S3NS.
For that, Google introduced the notion of
universein their SDKs and CLI tools, to basically point them at non-Google infrastructure.We're currently porting our platform to S3NS, which includes CNPG with the Barman Cloud plugin for backups, which doesn't work there right now because we can't configure the Google universe.
To connect to other universes, official clients need to be configured with a universe value, that overrides the base API domain. GOOGLE_CLOUD_UNIVERSE_DOMAIN is the official environment variable used to target non-GCP universes: https://documentation.s3ns.fr/docs/overview/tpc-key-differences#key_differences_for_developers
Unfortunately Barman uses the v1 client that doesn't read this variable, and only accepts the universe as part of the
client_optionsfield in the constructor, so we have to pass it in like this.