Skip to content

Grid page visibility in the latest 2.3.1 release #23960

New issue
New issue
Closed as not planned
Closed as not planned
Grid page visibility in the latest 2.3.1 release#23960
Labels
area:UIRelated to UI/UX. For Frontend Developers.duplicateIssue that is duplicatedkind:bugThis is a clearly a bug
Milestone
Airflow 2.3.2

Description

@stepanof
stepanof
opened on May 27, 2022

Apache Airflow version

2.3.1 (latest released)

What happened

Hello team!

I have faced the problem with Grid visibility in the latest 2.3.1 release.

My issue (example)

  1. Here is the end of dag code:
...
<code>
...
telegram_alert = TelegramOperator(
    task_id='telegram_alert'
    ,telegram_conn_id='telegram_bot_id'
    ,chat_id='telegram_chat_id'
    ,text="blablabla"
    ,dag=dag
)
python_task_1 >> python_task_2 >> puller >> telegram_alert

Grid:
image

  1. Delete task with telegram alert:
    Code in the end of dag code:

...
<code>
...
python_task_1 >> python_task_2 >> puller

Grid after deleting (Telegram alert task disappeared):
image

Grid after triggering dag again:
image

  1. Return task with telegram alert in the dag as it was at the begining.
    Grid:
    image

PS:

As I remember in 2.3.0 there wasn't such bug.
I tried to roll back airflow image (from 2.3.1 to 2.3.0) and restart containers but faced error during starting webserver:
alembic.util.exc.CommandError: Can't locate revision identified by '1de7bc13c950'

I see the same error when try to execute such command:
airflow db downgrade --to-version 2.3.0
Please help to me to roll back the airflow version or may be I can get some patch which fix this bug.

What you think should happen instead

The Grid must be visible at any time at any case

How to reproduce

Steps to reproduce:

  1. Create dag with N tasks
  2. Trigger dag
  3. Delete 1 task from dag
  4. Trigger dag
  5. Return deleted task to dag
  6. Try to see DagRun history on the Grid page.

Operating System

NAME="Ubuntu" VERSION="21.04 (Hirsute Hippo)" ID=ubuntu ID_LIKE=debian

Versions of Apache Airflow Providers

apache-airflow-providers-amazon==3.3.0
apache-airflow-providers-celery==2.1.4
apache-airflow-providers-docker==2.6.0
apache-airflow-providers-elasticsearch==3.0.3
apache-airflow-providers-google==6.8.0
apache-airflow-providers-grpc==2.0.4
apache-airflow-providers-hashicorp==2.2.0
apache-airflow-providers-http==2.1.2
apache-airflow-providers-imap==2.2.3
apache-airflow-providers-microsoft-azure==3.8.0
apache-airflow-providers-microsoft-mssql==2.1.3
apache-airflow-providers-mysql==2.2.3
apache-airflow-providers-odbc==2.0.4
apache-airflow-providers-postgres==4.1.0
apache-airflow-providers-redis==2.0.4
apache-airflow-providers-sendgrid==2.0.4
apache-airflow-providers-sftp==2.6.0
apache-airflow-providers-sqlite==2.1.3
apache-airflow-providers-ssh==2.4.3
apache-airflow-providers-telegram==2.0.4
apache-airflow-providers-vertica==2.1.3

Deployment

Docker-Compose

Deployment details

No response

Anything else

Many dags became unvisible on the Grid page and it is critical for us

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. stepanof
    added
    area:core
    kind:bugThis is a clearly a bug
    on May 27, 2022

  2. boring-cyborg commented on May 27, 2022

    @boring-cyborg
    boring-cyborgBot
    on May 27, 2022

    Thanks for opening your first issue here! Be sure to follow the issue template!

  3. stepanof commented on May 27, 2022

    @stepanof
    stepanof
    on May 27, 2022
    Author

    WebServer log contains:

    [2022-05-27 15:57:00,679] {manager.py:511} ERROR - Creation of Permission View Error: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_permission_id_view_menu_id_key"
DETAIL:  Key (permission_id, view_menu_id)=(13, 479) already exists.
[SQL: INSERT INTO ab_permission_view (id, permission_id, view_menu_id) VALUES (nextval('ab_permission_view_id_seq'), %(permission_id)s, %(view_menu_id)s) RETURNING ab_permission_view.id]
[parameters: {'permission_id': 13, 'view_menu_id': 479}]
(Background on this error at: http://sqlalche.me/e/14/gkpj)


[2022-05-27 15:57:00,795] {manager.py:570} ERROR - Add Permission to Role Error: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "ab_permission_view_role_permission_view_id_role_id_key"
DETAIL:  Key (permission_view_id, role_id)=(1648, 1) already exists.
[SQL: INSERT INTO ab_permission_view_role (id, permission_view_id, role_id) VALUES (nextval('ab_permission_view_role_id_seq'), %(permission_view_id)s, %(role_id)s) RETURNING ab_permission_view_role.id]
[parameters: {'permission_view_id': 1648, 'role_id': 1}]
(Background on this error at: http://sqlalche.me/e/14/gkpj)

  4. stepanof commented on May 27, 2022

    @stepanof
    stepanof
    on May 27, 2022
    Author

    What DevTools found:
    image

  5. stepanof commented on May 27, 2022

    @stepanof
    stepanof
    on May 27, 2022
    Author

    Looks similar to problem in pull 23332
    Similar things:
    There is problem with visibility of grid actions as well I have.
    There is problem with permissions as well my webserver log contain permissions error.

    I can't go deeper. This is my limit of understanding.
    I will be quite glad to your feedback and help.

  6. Smirn08 commented on May 27, 2022

    @Smirn08
    Smirn08
    on May 27, 2022

    @stepanof you can try to delete old dag dagruns at /dagrun/list/

  7. gbonazzoli commented on May 27, 2022

    @gbonazzoli
    gbonazzoli
    on May 27, 2022

    @stepanof

    Same problem here !

    I have some DAGS that are not showing up in the grid layout.
    In 2.3.0 there was no problem, it starts after upgrade Airflow to 2.3.1

    I suspect that the cause is some problems on the permission due to problem with AUTH_LDAP config that I have.

  8. potiuk commented on May 27, 2022

    @potiuk
    potiuk
    on May 27, 2022
    Member

    @gbonazzoli @stepanof almost for sure the problem is Duplicate of #23908 which is already solved in (very soon) 2.3.2 release where #23947 fixes it. It woudl be great if the both of you test the 2.3.2 RC1 that should appear likely within hours.

  9. potiuk
    added this to the Airflow 2.3.2 milestone on May 27, 2022
  10. potiuk
    added
    duplicateIssue that is duplicated
    on May 27, 2022
  11. uranusjr
    added
    area:UIRelated to UI/UX. For Frontend Developers.
    and removed
    area:core
    on May 29, 2022

  12. stepanof commented on May 29, 2022

    @stepanof
    stepanof
    on May 29, 2022
    Author

    @potiuk I will be happy to test 2.3.2 RC1. Please let me know how I can get it.

  13. gbonazzoli commented on May 30, 2022

    @gbonazzoli
    gbonazzoli
    on May 30, 2022

    @potiuk @stepanof

    I can snapshot my 2.3.1 Environment, install 2.3.2 RC1 and work on it in production.
    And if there aren't important regressions keep it without regarding the final results on the grid page.

    I only need that 2.3.2 RC1 is installable through pip install.

    Is it possible ?

  14. potiuk commented on May 30, 2022

    @potiuk
    potiuk
    on May 30, 2022
    Member

    I only need that 2.3.2 RC1 is installable through pip install.

    Yep. Once we get it out you will be able to install it via pip. I will ping here when we do. It took a bit more time though.

  15. potiuk commented on May 30, 2022

    @potiuk
    potiuk
    on May 30, 2022
    Member

    It's out https://pypi.org/project/apache-airflow/2.3.2rc1/

  16. stepanof commented on May 30, 2022

    @stepanof
    stepanof
    on May 30, 2022
    Author

    @potiuk what about docker image? Are you going to publish 2.3.2rc1 to docker hub?

  17. potiuk commented on May 30, 2022

    @potiuk
    potiuk
    on May 30, 2022
    Member

    @potiuk what about docker image? Are you going to publish 2.3.2rc1 to docker hub?

    Just being pushed. 3.7 is out already.

  18. potiuk commented on May 30, 2022

    @potiuk
    potiuk
    on May 30, 2022
    Member

    Screenshot 2022-05-30 at 15 22 44

  19. potiuk commented on May 30, 2022

    @potiuk
    potiuk
    on May 30, 2022
    Member

    All regular images ready. Now preparing slim images.

  20. stepanof commented on May 30, 2022

    @stepanof
    stepanof
    on May 30, 2022
    Author

    Great, thanks a lot! I will be back soon with feedback about Grid visibility :)

  21. stepanof commented on May 30, 2022

    @stepanof
    stepanof
    on May 30, 2022
    Author

    image
    image
    Everything works without errors.
    Thanks a lot for your prompt help!!!

  22. gbonazzoli commented on May 30, 2022

    @gbonazzoli
    gbonazzoli
    on May 30, 2022

    2.3.rc1 is good!

    Everything works as far as the Grid Page is concerned!

    image

    It seems also faster !!!

    I will let you know if we encounter some regression.

    Thanks!

  23. hsnprsd commented on May 31, 2022

    @hsnprsd
    hsnprsd
    on May 31, 2022
    Contributor

    Hi!
    When will airflow 2.3.2 be released?

  24. potiuk commented on May 31, 2022

    @potiuk
    potiuk
    on May 31, 2022
    Member

    https://lists.apache.org/thread/ry8vnxt4mptoos20cdm2wm856985poqw

  25. jedcunningham commented on Jun 2, 2022

    @jedcunningham
    jedcunningham
    on Jun 2, 2022
    Member

    Closing as this is a dup of #23908 and is fixed by #23947. Thanks for the report and testing rc1 @stepanof & @gbonazzoli. I invite you both to test 2.3.2rc2 and vote on the release: https://lists.apache.org/thread/08x514oto8r7359qk44dqh7297270dd5

    Btw, if you do ever want to downgrade in the future, you have to downgrade while still on 2.3.1, as that version has the migrations between 2.3.0 and 2.3.1, while 2.3.0 does not. So remember to downgrade the db first, then roll back to the old image.

  26. jedcunningham
    closed this as not plannedon Jun 2, 2022

  27. gbonazzoli commented on Jun 2, 2022

    @gbonazzoli
    gbonazzoli
    on Jun 2, 2022

    @jedcunningham

    I've just upgraded to 2.3.2rc2 and everything is ok.

    We were running 2.3.2rc1 on production since last monday and we had non problem at all.

  28. potiuk commented on Jun 2, 2022

    @potiuk
    potiuk
    on Jun 2, 2022
    Member

    Thanks for confirmiing @gbonazzoli :). Great to get such feedback from our users when release a bug fix !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:UIRelated to UI/UX. For Frontend Developers.duplicateIssue that is duplicatedkind:bugThis is a clearly a bug

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions