Repository navigation
Webserver reports "ImportError: Module "airflow.providers.google.cloud.operators.mlengine" does not define a "AIPlatformConsoleLink" attribute/class" #28393
Description
Apache Airflow Provider(s)
Versions of Apache Airflow Providers
apache-airflow 2.5.0
apache-airflow-providers-apache-beam 4.1.0
apache-airflow-providers-cncf-kubernetes 5.0.0
apache-airflow-providers-google 8.6.0
apache-airflow-providers-grpc 3.1.0
Apache Airflow version
2.5.0
Operating System
Debian 11
Deployment
Official Apache Airflow Helm Chart
Deployment details
KubernetesExecutor
What happened
We are seeing this stacktrace on our webserver when a task is clicked:
10.253.8.251 - - [15/Dec/2022:18:32:58 +0000] "GET /object/next_run_datasets/recs_ranking_purchase_ranker_dag HTTP/1.1" 200 2 "https://web.airflow.etsy-syseng-gke-prod.etsycloud.com/dags/recs_ranking_purchase_ranker_dag/code" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"
raise ImportError(f'Module "{module_path}" does not define a "{class_name}" attribute/class')
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/module_loading.py", line 38, in import_string
imported_class = import_string(class_name)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers_manager.py", line 275, in _sanity_check
Traceback (most recent call last):
During handling of the above exception, another exception occurred:
AttributeError: module 'airflow.providers.google.cloud.operators.mlengine' has no attribute 'AIPlatformConsoleLink'
return getattr(module, class_name)
File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/module_loading.py", line 36, in import_string
Traceback (most recent call last):
[2022-12-15 18:32:58,068] {providers_manager.py:243} WARNING - Exception when importing 'airflow.providers.google.cloud.operators.mlengine.AIPlatformConsoleLink' from 'apache-airflow-providers-google' package
ImportError: Module "airflow.providers.google.cloud.operators.mlengine" does not define a "AIPlatformConsoleLink" attribute/class
What you think should happen instead
These errors should now appear.
How to reproduce
Start webserver anew, navigate to a dag, click on a task, and tail webserver logs
Anything else
This YAML file is being utilized as config which then results in the import error here: https://github.com/apache/airflow/blob/providers-google/8.6.0/airflow/providers_manager.py#L885-L891
extra-links:
- airflow.providers.google.cloud.operators.bigquery.BigQueryConsoleLink
- airflow.providers.google.cloud.operators.bigquery.BigQueryConsoleIndexableLink
- airflow.providers.google.cloud.operators.mlengine.AIPlatformConsoleLink
We should remove this from extra-links as it was removed as of apache-airflow-providers-google 8.5.0
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
uranusjr commentedon Dec 16, 2022
AIPlatformConsoleLink was removed (replaced by individual operator links) in #26836 but the declaration never got updated. We should probably add a pre-commit hook to prevent this from happening in the future.
vchiapaikeo commentedon Dec 18, 2022
Hey @uranusjr , I'm happy to remove this from the config but modifying the pre-commit hook is a bit outside my wheelhouse. Is it okay if we scope the pre-commit hook edits out to another issue instead?
uranusjr commentedon Dec 19, 2022
You can submit a PR to do the removal (and additions—see #26836 for what hooks were added) and the pre-commit check can be dealt separately later.
vchiapaikeo commentedon Dec 19, 2022
Looks like the additions were properly added here: https://github.com/apache/airflow/pull/26836/files#diff-b8d2d329a113cf05d9bbd18b125843e86a3ce5c3c39704d29ad58bb83bb07861R1035-R1039
PR for removal is here: #28449