Skip to content

Assets do not show up as consumed #50873

Closed

Description

@bolkedebruin

Apache Airflow version

3.0.1

If "Other Airflow 2 version" selected, which one?

No response

What happened?

I am playing around with Assets.

This is the generating DAG / Asset (Confusing!)

from airflow.sdk import Metadata, asset

@asset(uri="file:///tmp/example.csv", schedule=None)
def write_example(self):
    with open("/tmp/example.csv", "w") as f:
        f.write("Hallo")

    yield Metadata(self, {"row_count": 0})

I have two consuming DAGs, one "asset" syntax based and one taskflow based:

from airflow.sdk import Asset, asset
from airflow.sdk import DAG, dag, task

write_example = Asset("write_example")

#@asset(schedule=None)

@dag
def my_dag():

    @task
    def consume_csv(write_example):
        print("Bla")

my_dag()

and

from airflow.sdk import Asset, asset
from airflow.sdk import DAG, dag, task

@asset(schedule=None)
def consume_csv(write_example):
    print("Bla")

Neither of the two DAGs generate a "consume" number at the write_example asset / dag

What you think should happen instead?

The number of consuming DAGs should read "2" for the "write_example" asset

How to reproduce

see above

Operating System

Linux

Versions of Apache Airflow Providers

not relevant

Deployment

Docker-Compose

Deployment details

not relevant

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. added
    priority:mediumBug that should be fixed before next release but would not block a release
    on May 21, 2025
  2. uranusjr commented on May 21, 2025

    @uranusjr
    Member

    Currently assets are only shown as consumed if they are used in schedule, not inlets. I feel we might want to do some more thinking instead of just adding those. It’s probably best to have three fields: producing (things that list the asset as outlet), consuming (list as inlet), and scheduled (list in schedule). Not sure about the names here.

    cc @bbovenzi for ideas.

  3. added
    area:UIRelated to UI/UX. For Frontend Developers.
    and removed
    needs-triagelabel for new issues that we didn't triage yet
    on May 21, 2025
  4. bolkedebruin commented on May 22, 2025

    @bolkedebruin
    ContributorAuthor

    I can understand that, however the examples I shared above are inlets, i.e. consumed by the task.

  5. uranusjr commented on May 22, 2025

    @uranusjr
    Member

    Yes I noticed. I think the term consuming is confusing anyway.

  6. bolkedebruin commented on May 22, 2025

    @bolkedebruin
    ContributorAuthor

    Ah gotcha, I misread your comment. It is now confusing indeed. To me "consumed" means "used worked on". We could consider the following alternatives:

    Data Dependency / Triggers (or Schedules to avoid naming conflicts)

    or

    Consumed by / Triggers (or Schedules)

  7. bolkedebruin commented on May 22, 2025

    @bolkedebruin
    ContributorAuthor

    Adding:

    from airflow.sdk import Asset, asset
    from airflow.sdk import DAG, dag, task
    
    write_example = Asset("write_example")
    
    #@asset(schedule=None)
    
    @dag
    def my_dag():
    
        @task
        def consume_csv(write_example):
            print("Bla")
    
    my_dag()
    

    This sets it as a schedule and a it is an inlet. However

    1. When triggering the Asset (write_example) this dag does not get kicked off
    2. It also does not become visible as "consumed" in the Asset

    I might be doing something unexpected write_example = Asset("write_example"), but how to do it otherwise?

  8. 13 remaining items

  9. bbovenzi commented on Jun 5, 2025

    @bbovenzi
    Contributor

    So for the dag-asset dependency graphs. We will show edges for the following:

    • from producing_tasks to the asset
    • asset to the consuming_task
    • asset to scheduled_dags

    Should all of these edges be styled the same or the task<>asset ones should be shown differently to represent a Data Dependency rather than the normal task scheduling dependency.

    We can also add a filter to /dag_dependencies to only show asset<>task relationships.

  10. self-assigned this
    on Jun 5, 2025
  11. uranusjr commented on Jun 6, 2025

    @uranusjr
    Member

    I'd say it makes sense for schedule references to be shown differently.

  12. uranusjr commented on Jun 19, 2025

    @uranusjr
    Member

    Backend part merged, now we can put the additional info in UI.

  13. removed their assignment
    on Jun 19, 2025
  14. removed their assignment
    on Jul 15, 2025
  15. bbovenzi commented on Jul 15, 2025

    @bbovenzi
    Contributor

    Unassigning myself since I'm about to go on vacation. Someone else can feel free to pick it up in the meantime.

  16. pierrejeambrun commented on Aug 21, 2025

    @pierrejeambrun
    Member

    Moving for 3.1+ considering other priorities I'm not sur we will be able to get that in. We can reconsider if we have some bandwidth available or if someone is willing to work on it.

  17. phanikumv commented on Oct 27, 2025

    @phanikumv
    Contributor

    @guan404ming would you like to pick this?

  18. guan404ming commented on Oct 29, 2025

    @guan404ming
    Member

    Sure, I could take a look at this one!

  19. self-assigned this
    on Oct 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

affected_version:3.0Issues Reported for 3.0area:UIRelated to UI/UX. For Frontend Developers.kind:bugThis is a clearly a bugpriority:mediumBug that should be fixed before next release but would not block a release

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions