dbt-databricks 1.6.X overeager introspection (describe extended ...) of all tables in schema -- REGRESSION

[T-Dunlap]

Describe the bug

This issue has cropped up again for dbt-databricks==1.6.6. Opening this for a customer of mine, so I don't have access to detailed information, but this was running on dbt Cloud against hive metastore.

Steps To Reproduce

See Jeremy's ticket here.

Expected behavior

We should not be running describe extended on everything in the schema... only the relevant models.

Screenshots and log output

Please reach out to me for the full logs more details.

System information

The output of dbt --version:

18:08:53 Running with dbt=1.6.6

The operating system you're using:
Running in dbt Cloud

The output of python --version:
Unknown

Additional context

This is causing unexpected load times for minor transformations.

[benc-db]

dbt=1.6.6 does not necessarily imply dbt-databricks 1.6.6; do you have a screenshot verifying that?

[benc-db]

Ok, got additional context out of band from dbt. This is not a regression, just things working as expected for UC, unfortunately. I'm hoping that the changes to 'get catalog by relations' in 1.7.0 will remove the need for this (basically we didn't have a good way of discerning views from materialized views and had to use describe extended to find out), and if that doesn't do it, then I'll push a fix for dbt-databricks once I get confirmation that Materialized Views are distinguishable from Views in the information schema (something I'm trying to get fixed behind the scenes).

[benc-db]

Please try 1.7.0 and let me know if this helps. I think there still is probably at least one more scenario that is doing too much describe extended, but 1.7.0 should have significant improvements on this front.

[benc-db]

Actually hold that. We found a regression in doc generate performance that I'm fixing now. Look for 1.7.1 next week.