From 174ac161a7f17d1932e45273015fd51442f32477 Mon Sep 17 00:00:00 2001 From: Joe Berni <99652623+josephberni@users.noreply.github.com> Date: Tue, 2 May 2023 20:28:30 +0100 Subject: [PATCH 1/2] allow-table-extended-to-handle-more-than-2048-chars (#326) Resolves [#326](https://github.com/databricks/dbt-databricks/pull/326) ### Description This PR allows the adapter to handle strings of more than 2048 characters when running show tables extended in Databricks. This is currently preventing users from being able to run this command when using dbt if they have a large number of tables in any given schema. The way this is done is that it will pass the `*` in instances where the string length of the pipe separated tables is more than 2048 chars. I have successfully tested both unit tests and `tox -e integration-databricks-sql-endpoint`. ### Checklist - [x] I have run this code in development and it appears to resolve the stated issue - [x] This PR includes tests, or tests are not required/relevant for this PR - [x] I have updated the `CHANGELOG.md` and added information about my change to the "dbt-databricks next" section. --- CHANGELOG.md | 3 +++ dbt/adapters/databricks/impl.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b6d954f0..6c138a216 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,6 +63,9 @@ ### Under the hood Throw an error if a model has an enforced contract. ([#322](https://github.com/databricks/dbt-databricks/pull/322)) +### Fixes +- Fix issue where the show tables extended command is limited to 2048 characters. ([#326](https://github.com/databricks/dbt-databricks/pull/326)) + ## dbt-databricks 1.4.2 (February 17, 2023) ### Fixes diff --git a/dbt/adapters/databricks/impl.py b/dbt/adapters/databricks/impl.py index e7b328f9b..7cff8f748 100644 --- a/dbt/adapters/databricks/impl.py +++ b/dbt/adapters/databricks/impl.py @@ -451,7 +451,7 @@ def _get_one_catalog( schema_relation = self.Relation.create( database=database, schema=schema, - identifier="|".join(table_names), + identifier=("|".join(table_names) if len("|".join(table_names)) < 2048 else "*"), quote_policy=self.config.quoting, ) for relation, information in self._list_relations_with_information(schema_relation): From e9ba5eb0468b1f57b1e6dae8bbf2ba5e8a784d3d Mon Sep 17 00:00:00 2001 From: Jesse Whitehouse Date: Mon, 24 Jul 2023 13:29:27 -0400 Subject: [PATCH 2/2] Move changelog update to the correct section Signed-off-by: Jesse Whitehouse --- CHANGELOG.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c138a216..9c912325e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ ## dbt-databricks 1.5.x TBD +### Fixes + +- Fix issue where the show tables extended command is limited to 2048 characters. ([#326](https://github.com/databricks/dbt-databricks/pull/326)) + ## dbt-databricks 1.5.5 (July 7, 2023) - Fixed issue where starting a terminated cluster in the python path would never return @@ -63,9 +67,6 @@ ### Under the hood Throw an error if a model has an enforced contract. ([#322](https://github.com/databricks/dbt-databricks/pull/322)) -### Fixes -- Fix issue where the show tables extended command is limited to 2048 characters. ([#326](https://github.com/databricks/dbt-databricks/pull/326)) - ## dbt-databricks 1.4.2 (February 17, 2023) ### Fixes