Repository navigation
Closed
Labels
affected_version:3.0Issues Reported for 3.0Issues Reported for 3.0area:UIRelated to UI/UX. For Frontend Developers.Related to UI/UX. For Frontend Developers.kind:bugThis is a clearly a bugThis is a clearly a bugpriority:mediumBug that should be fixed before next release but would not block a releaseBug that should be fixed before next release but would not block a release
Milestone
Description
Apache Airflow version
3.0.1
If "Other Airflow 2 version" selected, which one?
No response
What happened?
I am playing around with Assets.
This is the generating DAG / Asset (Confusing!)
from airflow.sdk import Metadata, asset
@asset(uri="file:///tmp/example.csv", schedule=None)
def write_example(self):
with open("/tmp/example.csv", "w") as f:
f.write("Hallo")
yield Metadata(self, {"row_count": 0})
I have two consuming DAGs, one "asset" syntax based and one taskflow based:
from airflow.sdk import Asset, asset
from airflow.sdk import DAG, dag, task
write_example = Asset("write_example")
#@asset(schedule=None)
@dag
def my_dag():
@task
def consume_csv(write_example):
print("Bla")
my_dag()
and
from airflow.sdk import Asset, asset
from airflow.sdk import DAG, dag, task
@asset(schedule=None)
def consume_csv(write_example):
print("Bla")
Neither of the two DAGs generate a "consume" number at the write_example asset / dag
What you think should happen instead?
The number of consuming DAGs should read "2" for the "write_example" asset
How to reproduce
see above
Operating System
Linux
Versions of Apache Airflow Providers
not relevant
Deployment
Docker-Compose
Deployment details
not relevant
Anything else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
affected_version:3.0Issues Reported for 3.0Issues Reported for 3.0area:UIRelated to UI/UX. For Frontend Developers.Related to UI/UX. For Frontend Developers.kind:bugThis is a clearly a bugThis is a clearly a bugpriority:mediumBug that should be fixed before next release but would not block a releaseBug that should be fixed before next release but would not block a release
Activity
uranusjr commentedon May 21, 2025
Currently assets are only shown as consumed if they are used in schedule, not inlets. I feel we might want to do some more thinking instead of just adding those. It’s probably best to have three fields: producing (things that list the asset as outlet), consuming (list as inlet), and scheduled (list in schedule). Not sure about the names here.
cc @bbovenzi for ideas.
bolkedebruin commentedon May 22, 2025
I can understand that, however the examples I shared above are inlets, i.e. consumed by the task.
uranusjr commentedon May 22, 2025
Yes I noticed. I think the term consuming is confusing anyway.
bolkedebruin commentedon May 22, 2025
Ah gotcha, I misread your comment. It is now confusing indeed. To me "consumed" means "used worked on". We could consider the following alternatives:
Data Dependency / Triggers (or Schedules to avoid naming conflicts)
or
Consumed by / Triggers (or Schedules)
bolkedebruin commentedon May 22, 2025
Adding:
This sets it as a schedule and a it is an inlet. However
I might be doing something unexpected
write_example = Asset("write_example"), but how to do it otherwise?13 remaining items
bbovenzi commentedon Jun 5, 2025
So for the dag-asset dependency graphs. We will show edges for the following:
Should all of these edges be styled the same or the task<>asset ones should be shown differently to represent a Data Dependency rather than the normal task scheduling dependency.
We can also add a filter to /dag_dependencies to only show asset<>task relationships.
uranusjr commentedon Jun 6, 2025
I'd say it makes sense for schedule references to be shown differently.
uranusjr commentedon Jun 19, 2025
Backend part merged, now we can put the additional info in UI.
bbovenzi commentedon Jul 15, 2025
Unassigning myself since I'm about to go on vacation. Someone else can feel free to pick it up in the meantime.
pierrejeambrun commentedon Aug 21, 2025
Moving for
3.1+considering other priorities I'm not sur we will be able to get that in. We can reconsider if we have some bandwidth available or if someone is willing to work on it.phanikumv commentedon Oct 27, 2025
@guan404ming would you like to pick this?
guan404ming commentedon Oct 29, 2025
Sure, I could take a look at this one!