Skip to content

Creating materialized views do not set table properties #577

@thijs-nijhuis

Description

@thijs-nijhuis

Describe the bug

To make use to Delta's ChangeDataFeed I set 'delta.enableChangeDataFeed': true as one of the table properties. I set is at the project level for all models like this:

dbt_model_name:
  +tblproperties:
    'delta.enableChangeDataFeed': true

All models materialized as table will get this property upon creation. Models materialized as 'materialized_view' however do not. I also tried setting it in the models config directly like this:

{{
  config(
    materialized = 'materialized_view',
    tblproperties={'delta.enableChangeDataFeed': 'true'},
    )
}}

Unfortunately, that didn't work either. I think it is important to be able to set this property because it is a requirement to be able to to do an incremental refresh on a materialized view. So if I would want to create an incremental flow of multiple models using MV's this property needs to be set otherwise it will always execute a full compute.

Steps To Reproduce

Set the table property to a model as described above and run the project. MV should not be there as this property can only be set upon creating the mv.

Expected behavior

I expect all property I specify through the tblproperties config that are supported by that type of object to be set.

Screenshots and log output

view tables tab from catalog when created via dbt.
image

System information

The output of dbt --version:

Core:
  - installed: 1.7.6
  - latest:    1.7.7 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - databricks: 1.7.4 - Ahead of latest version!
  - spark:      1.7.1 - Up to date!

The operating system you're using:
windows 11

The output of python --version:
Python 3.11.4

Additional context

I can create the view manually with that property and verified that the next mv (the one that selects from it) won't do a full refresh when this property is set.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions