Describe the bug
A clear and concise description of what the bug is. What command did you run? What happened?
Currently when attempting to run dbt docs generate you get a failure message if you have a large number of tables in any given schema. This is because the package will attempt to run the following code show table extended in <schema_name> like '<table_1>|<table_2>|...'.
The call to glue has the following constraint 'Length Constraints: Minimum length of 0. Maximum length of 2048.' which results in the query failing and ultimately an inability to create dbt docs when using glue on any version above 1.3
Steps To Reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.
Use glue as your metastore
Have a large number of tables in a given schema
Attempt to run dbt docs generate
Expected behavior
A clear and concise description of what you expected to happen.
dbt docs generate to work regardless of how many tables you have in a given schema.
Screenshots and log output
If applicable, add screenshots or log output to help explain your problem.
System information
The output of dbt --version:
failed to satisfy constraint: Member must have length less than or equal to 2048 (Service: AWSGlue; Status Code: 400; Error Code: ValidationException; Request ID: <REQUEST_ID>; Proxy: null))
The operating system you're using:
databricks-cli==0.17.5
dbt-core==1.4.6
dbt-databricks==1.4.2
dbt-spark[PyHive]==1.4.1
elementary-data==0.7.7
graphviz==0.20.1
pre-commit==3.1.1
PyYAML==6.0
rich==13.3.2
shyaml==0.6.2
sqlfluff-templater-dbt==2.0.6
sqlfluff==2.0.6
The output of python --version:
3.8.10
Additional context
Add any other context about the problem here.
Describe the bug
A clear and concise description of what the bug is. What command did you run? What happened?
Currently when attempting to run dbt docs generate you get a failure message if you have a large number of tables in any given schema. This is because the package will attempt to run the following code show table extended in <schema_name> like '<table_1>|<table_2>|...'.
The call to glue has the following constraint 'Length Constraints: Minimum length of 0. Maximum length of 2048.' which results in the query failing and ultimately an inability to create dbt docs when using glue on any version above 1.3
Steps To Reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.
Use glue as your metastore
Have a large number of tables in a given schema
Attempt to run dbt docs generate
Expected behavior
A clear and concise description of what you expected to happen.
dbt docs generate to work regardless of how many tables you have in a given schema.
Screenshots and log output
If applicable, add screenshots or log output to help explain your problem.
System information
The output of
dbt --version:The operating system you're using:
databricks-cli==0.17.5
dbt-core==1.4.6
dbt-databricks==1.4.2
dbt-spark[PyHive]==1.4.1
elementary-data==0.7.7
graphviz==0.20.1
pre-commit==3.1.1
PyYAML==6.0
rich==13.3.2
shyaml==0.6.2
sqlfluff-templater-dbt==2.0.6
sqlfluff==2.0.6
The output of
python --version:3.8.10
Additional context
Add any other context about the problem here.