Repository navigation
Add capability in Airflow connections to validate host #31460
Description
Apache Airflow version
2.6.1
What happened
While creating connections in airflow, the form doesn't check for the correctness of the format of the host provided. For instance, we can proceed providing something like this which is not a valid url: spark://k8s://100.68.0.1:443?deploy-mode=cluster. It wont instantly fail but will return faulty hosts and other details if called later.
Motivation:
#31376 (comment)
What you think should happen instead
The Connection form can have a validator which should check for these scenarios and report if there is an issue to save developers and users time later.
How to reproduce
- Go to airflow connections form
- Fill in connection host as:
spark://k8s://100.68.0.1:443?deploy-mode=cluster, other details can be anything - Create the connection
- Run
airflow connections get <name> - The host and schema will be wrong
Operating System
Macos
Versions of Apache Airflow Providers
No response
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Activity
potiuk commentedon May 22, 2023
note that this one is a bit tricky. It literally needs to be "per-connection type". The host part might vary wildly between different types of connections and it does not have to conform to a standard URL or anything. There are likely connection types that are using host as host name (and there it could theoretically contain even multiple :// ://, some other treat is a URL, some other have likely different expectations.
So I don't think there is "one rule to apply to all of them".
[-]Airflow connections doesn't check for the correctness of host[/-][+]Add capabipity for Airflow connections to validate host[/+][-]Add capabipity for Airflow connections to validate host[/-][+]Add capability in Airflow connections to validate host[/+]hussein-awala commentedon May 23, 2023
this validation will be added by #31465