diff --git a/airflow-core/docs/howto/email-config.rst b/airflow-core/docs/howto/email-config.rst index 4bf43d07d771b..45d8c566341a4 100644 --- a/airflow-core/docs/howto/email-config.rst +++ b/airflow-core/docs/howto/email-config.rst @@ -18,80 +18,32 @@ Email Configuration =================== -You can configure the email that is being sent in your ``airflow.cfg`` -by setting a ``subject_template`` and/or a ``html_content_template`` -in the ``[email]`` section. +Airflow sends task emails through the function set in ``[email] email_backend``. By default, email is sent +over SMTP by the SMTP provider (``apache-airflow-providers-smtp``). To set up the SMTP server and its +credentials, see :doc:`apache-airflow-providers-smtp:email-backend` and +:doc:`apache-airflow-providers-smtp:connections/smtp`. + +The ``[email]`` section also sets the connection, sender and message templates used for any backend: .. code-block:: ini [email] email_backend = airflow.utils.email.send_email_smtp + email_conn_id = smtp_default + from_email = "John Doe " subject_template = /path/to/my_subject_template_file html_content_template = /path/to/my_html_content_template_file -Equivalent environment variables look like: - -.. code-block:: sh - - AIRFLOW__EMAIL__EMAIL_BACKEND=airflow.utils.email.send_email_smtp - AIRFLOW__EMAIL__SUBJECT_TEMPLATE=/path/to/my_subject_template_file - AIRFLOW__EMAIL__HTML_CONTENT_TEMPLATE=/path/to/my_html_content_template_file - -You can configure a sender's email address by setting ``from_email`` in the ``[email]`` section like: - -.. code-block:: ini - - [email] - from_email = "John Doe " - -Equivalent environment variables look like: - -.. code-block:: sh - - AIRFLOW__EMAIL__FROM_EMAIL="John Doe " - -By default, ``email_backend`` is set to ``airflow.utils.email.send_email_smtp``, which sends email over SMTP. - -To configure SMTP settings, checkout the :ref:`SMTP ` section in the standard configuration. -If you do not want to store the SMTP credentials in the config or in the environment variables, you can create a -connection called ``smtp_default`` of ``Email`` type, or choose a custom connection name and set the ``email_conn_id`` with its name in -the configuration & store SMTP username-password in it. Other SMTP settings like host, port etc always gets picked up -from the configuration only. The connection can be of any type (for example 'HTTP connection'). - -.. image:: ../img/ui-dark/email_connection.png - :align: center - :alt: create email connection - -If you want to check which email backend is currently set, you can use ``airflow config get-value email email_backend`` command as in -the example below. - -.. code-block:: bash - - $ airflow config get-value email email_backend - airflow.utils.email.send_email_smtp - -To access the task's information you use `Jinja Templating `_ in your template files. - -For example a ``html_content_template`` file could look like this: - -.. code-block:: - - Try {{try_number}} out of {{max_tries + 1}}
- Exception:
{{exception_html}}
- Log: Link
- Host: {{ti.hostname}}
- Mark success: Link
+The templates use `Jinja `_ and can access the task's information, for +example ``{{ti.log_url}}``. .. note:: - For more information on setting the configuration, see :doc:`set-config` - -Alternative email backends ---------------------------- -Instead of sending email over SMTP, you can point ``email_backend`` at an implementation provided by a -community provider to send email through a third-party service's API. To use one, install the relevant -provider distribution, set ``email_backend`` to the dotted path of its ``send_email`` function, and set -``email_conn_id`` to a connection holding the credentials it needs. + The functions in ``airflow.utils.email`` are deprecated and now live in + ``airflow.providers.smtp.utils.emailer``. Only the Python imports moved: keep ``email_backend`` set to + ``airflow.utils.email.send_email_smtp``. Airflow treats that value as "send with ``SmtpNotifier``", + which reads all SMTP settings from the Connection. Any other value, including the provider path, calls + that function instead, and the SMTP provider's function reads the server settings from ``[smtp]``. -The list of email backends provided by community-managed providers is available in +To send email through another service, set ``email_backend`` to a backend from another provider; see :doc:`apache-airflow-providers:core-extensions/email-backends`. diff --git a/airflow-core/newsfragments/75414.significant.rst b/airflow-core/newsfragments/75414.significant.rst new file mode 100644 index 0000000000000..8feb51b2682a3 --- /dev/null +++ b/airflow-core/newsfragments/75414.significant.rst @@ -0,0 +1,20 @@ +``airflow.utils.email`` is deprecated in favor of the SMTP provider + +The functions in ``airflow.utils.email`` (``send_email``, ``send_email_smtp``, ``send_mime_email``, +``build_mime_message`` and ``get_email_address_list``) now live in +``airflow.providers.smtp.utils.emailer``. The old names still work and raise a ``DeprecatedImportWarning``. +Airflow now requires ``apache-airflow-providers-smtp>=3.2.0``. + +Keep ``[email] email_backend`` set to ``airflow.utils.email.send_email_smtp``. Airflow treats that value as +"send task emails with ``SmtpNotifier``", which reads all SMTP settings from the Connection. Changing it to +the provider path sends through the ``[smtp]`` settings instead. + +If you call these functions directly: + +* The SMTP Connection is looked up with ``BaseHook.get_connection``. A script that runs outside Airflow's + processes needs the Connection in an environment variable or a secrets backend, not only in the metadata + database. +* Settings are read through the Task SDK configuration, so values changed at runtime with + ``airflow.configuration.conf.set()`` are not seen. ``airflow.cfg`` and environment variables work as before. +* Tests that patch ``airflow.utils.email.send_mime_email`` no longer intercept the call made by + ``send_email_smtp``. Patch ``airflow.providers.smtp.utils.emailer.send_mime_email`` instead. diff --git a/airflow-core/pyproject.toml b/airflow-core/pyproject.toml index 573be540ffaa7..a843480c11c72 100644 --- a/airflow-core/pyproject.toml +++ b/airflow-core/pyproject.toml @@ -158,7 +158,7 @@ dependencies = [ "apache-airflow-providers-common-compat>=1.7.4", "apache-airflow-providers-common-io>=1.6.3", "apache-airflow-providers-common-sql>=1.28.1", - "apache-airflow-providers-smtp>=2.3.1", + "apache-airflow-providers-smtp>=3.2.0", "apache-airflow-providers-standard>=1.9.0", # Start of shared logging dependencies "msgspec>=0.19.0", diff --git a/airflow-core/src/airflow/utils/email.py b/airflow-core/src/airflow/utils/email.py index 46f24fba0f509..89bb754862a5b 100644 --- a/airflow-core/src/airflow/utils/email.py +++ b/airflow-core/src/airflow/utils/email.py @@ -15,327 +15,41 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. -from __future__ import annotations - -import collections.abc -import logging -import os -import re -import smtplib -import ssl -from collections.abc import Iterable -from email.mime.application import MIMEApplication -from email.mime.multipart import MIMEMultipart -from email.mime.text import MIMEText -from email.utils import formatdate -from typing import Any - -from airflow.configuration import conf -from airflow.exceptions import AirflowException - -log = logging.getLogger(__name__) - - -def send_email( - to: list[str] | Iterable[str], - subject: str, - html_content: str, - files: list[str] | None = None, - dryrun: bool = False, - cc: str | Iterable[str] | None = None, - bcc: str | Iterable[str] | None = None, - mime_subtype: str = "mixed", - mime_charset: str = "utf-8", - conn_id: str | None = None, - custom_headers: dict[str, Any] | None = None, - **kwargs, -) -> None: - """ - Send an email using the backend specified in the *EMAIL_BACKEND* configuration option. - - :param to: A list or iterable of email addresses to send the email to. - :param subject: The subject of the email. - :param html_content: The content of the email in HTML format. - :param files: A list of paths to files to attach to the email. - :param dryrun: If *True*, the email will not actually be sent. Default: *False*. - :param cc: A string or iterable of strings containing email addresses to send a copy of the email to. - :param bcc: A string or iterable of strings containing email addresses to send a - blind carbon copy of the email to. - :param mime_subtype: The subtype of the MIME message. Default: "mixed". - :param mime_charset: The charset of the email. Default: "utf-8". - :param conn_id: The connection ID to use for the backend. If not provided, the default connection - specified in the *EMAIL_CONN_ID* configuration option will be used. - :param custom_headers: A dictionary of additional headers to add to the MIME message. - No validations are run on these values, and they should be able to be encoded. - :param kwargs: Additional keyword arguments to pass to the backend. - """ - backend = conf.getimport("email", "EMAIL_BACKEND") - backend_conn_id = conn_id or conf.get("email", "EMAIL_CONN_ID") - from_email = conf.get("email", "from_email", fallback=None) - - to_list = get_email_address_list(to) - to_comma_separated = ", ".join(to_list) - - return backend( - to_comma_separated, - subject, - html_content, - files=files, - dryrun=dryrun, - cc=cc, - bcc=bcc, - mime_subtype=mime_subtype, - mime_charset=mime_charset, - conn_id=backend_conn_id, - from_email=from_email, - custom_headers=custom_headers, - **kwargs, - ) - - -def send_email_smtp( - to: str | Iterable[str], - subject: str, - html_content: str, - files: list[str] | None = None, - dryrun: bool = False, - cc: str | Iterable[str] | None = None, - bcc: str | Iterable[str] | None = None, - mime_subtype: str = "mixed", - mime_charset: str = "utf-8", - conn_id: str = "smtp_default", - from_email: str | None = None, - custom_headers: dict[str, Any] | None = None, - **kwargs, -) -> None: - """ - Send an email with html content. - - :param to: Recipient email address or list of addresses. - :param subject: Email subject. - :param html_content: Email body in HTML format. - :param files: List of file paths to attach to the email. - :param dryrun: If True, the email will not be sent, but all other actions will be performed. - :param cc: Carbon copy recipient email address or list of addresses. - :param bcc: Blind carbon copy recipient email address or list of addresses. - :param mime_subtype: MIME subtype of the email. - :param mime_charset: MIME charset of the email. - :param conn_id: Connection ID of the SMTP server. - :param from_email: Sender email address. - :param custom_headers: Dictionary of custom headers to include in the email. - :param kwargs: Additional keyword arguments. - - >>> send_email("test@example.com", "foo", "Foo bar", ["/dev/null"], dryrun=True) - """ - smtp_mail_from = conf.get("smtp", "SMTP_MAIL_FROM") - - if smtp_mail_from is not None: - mail_from = smtp_mail_from - else: - if from_email is None: - raise ValueError( - "You should set from email - either by smtp/smtp_mail_from config or `from_email` parameter" - ) - mail_from = from_email - - msg, recipients = build_mime_message( - mail_from=mail_from, - to=to, - subject=subject, - html_content=html_content, - files=files, - cc=cc, - bcc=bcc, - mime_subtype=mime_subtype, - mime_charset=mime_charset, - custom_headers=custom_headers, - ) - - send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun) - - -def build_mime_message( - mail_from: str | None, - to: str | Iterable[str], - subject: str, - html_content: str, - files: list[str] | None = None, - cc: str | Iterable[str] | None = None, - bcc: str | Iterable[str] | None = None, - mime_subtype: str = "mixed", - mime_charset: str = "utf-8", - custom_headers: dict[str, Any] | None = None, -) -> tuple[MIMEMultipart, list[str]]: - """ - Build a MIME message that can be used to send an email and returns a full list of recipients. - - :param mail_from: Email address to set as the email's "From" field. - :param to: A string or iterable of strings containing email addresses to set as the email's "To" field. - :param subject: The subject of the email. - :param html_content: The content of the email in HTML format. - :param files: A list of paths to files to be attached to the email. - :param cc: A string or iterable of strings containing email addresses to set as the email's "CC" field. - :param bcc: A string or iterable of strings containing email addresses to set as the email's "BCC" field. - :param mime_subtype: The subtype of the MIME message. Default: "mixed". - :param mime_charset: The charset of the email. Default: "utf-8". - :param custom_headers: Additional headers to add to the MIME message. No validations are run on these - values, and they should be able to be encoded. - :return: A tuple containing the email as a MIMEMultipart object and a list of recipient email addresses. - """ - to = get_email_address_list(to) +"""Deprecated: the email utilities moved to :mod:`airflow.providers.smtp.utils.emailer`.""" - msg = MIMEMultipart(mime_subtype) - msg["Subject"] = subject - if mail_from: - msg["From"] = mail_from - msg["To"] = ", ".join(to) - recipients = to - if cc: - cc = get_email_address_list(cc) - msg["CC"] = ", ".join(cc) - recipients += cc - - if bcc: - # don't add bcc in header - bcc = get_email_address_list(bcc) - recipients += bcc - - msg["Date"] = formatdate(localtime=True) - mime_text = MIMEText(html_content, "html", mime_charset) - msg.attach(mime_text) - - for fname in files or []: - basename = os.path.basename(fname) - with open(fname, "rb") as file: - part = MIMEApplication(file.read(), Name=basename) - part.add_header("Content-Disposition", "attachment", filename=basename) - part["Content-ID"] = f"<{basename}>" - msg.attach(part) - - if custom_headers: - for header_key, header_value in custom_headers.items(): - msg[header_key] = header_value - - return msg, recipients - - -def send_mime_email( - e_from: str, - e_to: str | list[str], - mime_msg: MIMEMultipart, - conn_id: str = "smtp_default", - dryrun: bool = False, -) -> None: - """ - Send a MIME email. - - :param e_from: The email address of the sender. - :param e_to: The email address or a list of email addresses of the recipient(s). - :param mime_msg: The MIME message to send. - :param conn_id: The ID of the SMTP connection to use. - :param dryrun: If True, the email will not be sent, but a log message will be generated. - """ - smtp_host = conf.get_mandatory_value("smtp", "SMTP_HOST") - smtp_port = conf.getint("smtp", "SMTP_PORT") - smtp_starttls = conf.getboolean("smtp", "SMTP_STARTTLS") - smtp_ssl = conf.getboolean("smtp", "SMTP_SSL") - smtp_retry_limit = conf.getint("smtp", "SMTP_RETRY_LIMIT") - smtp_timeout = conf.getint("smtp", "SMTP_TIMEOUT") - smtp_user = None - smtp_password = None - - if conn_id is not None: - try: - from airflow.models import Connection - - airflow_conn = Connection.get_connection_from_secrets(conn_id) - smtp_user = airflow_conn.login - smtp_password = airflow_conn.password - except AirflowException: - pass - if smtp_user is None or smtp_password is None: - log.debug("No user/password found for SMTP, so logging in with no authentication.") - - if not dryrun: - for attempt in range(smtp_retry_limit + 1): - log.info("Email alerting: attempt %s", str(attempt + 1)) - try: - smtp_conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl) - except smtplib.SMTPServerDisconnected: - if attempt == smtp_retry_limit: - raise - else: - if smtp_starttls: - smtp_conn.starttls(context=_get_ssl_context()) - if smtp_user and smtp_password: - smtp_conn.login(smtp_user, smtp_password) - log.info("Sent an alert email to %s", e_to) - smtp_conn.sendmail(e_from, e_to, mime_msg.as_string()) - smtp_conn.quit() - break - - -def get_email_address_list(addresses: str | Iterable[str]) -> list[str]: - """ - Return a list of email addresses from the provided input. - - :param addresses: A string or iterable of strings containing email addresses. - :return: A list of email addresses. - :raises TypeError: If the input is not a string or iterable of strings. - """ - if isinstance(addresses, str): - return _get_email_list_from_str(addresses) - if isinstance(addresses, collections.abc.Iterable): - if not all(isinstance(item, str) for item in addresses): - raise TypeError("The items in your iterable must be strings.") - return list(addresses) - raise TypeError(f"Unexpected argument type: Received '{type(addresses).__name__}'.") +from __future__ import annotations +from typing import TYPE_CHECKING -def _get_ssl_context() -> ssl.SSLContext | None: - """ - Return the SSL context configured via the ``email.ssl_context`` option. +from airflow.utils.deprecation_tools import add_deprecated_classes - ``"default"`` produces :func:`ssl.create_default_context`; ``"none"`` - returns ``None`` so callers that explicitly want to skip certificate - validation (for example, against a self-signed SMTP server in a - lab environment) can still do so. - """ - ssl_context_string = conf.get("email", "SSL_CONTEXT") - if ssl_context_string == "default": - return ssl.create_default_context() - if ssl_context_string == "none": - return None - raise RuntimeError( - f"The email.ssl_context configuration variable must " - f"be set to 'default' or 'none' and is '{ssl_context_string}." +if TYPE_CHECKING: + # Lets type checkers resolve the deprecated names, which only exist at runtime via __getattr__. + from airflow.providers.smtp.utils.emailer import ( + build_mime_message as build_mime_message, + get_email_address_list as get_email_address_list, + send_email as send_email, + send_email_smtp as send_email_smtp, + send_mime_email as send_mime_email, ) - -def _get_smtp_connection(host: str, port: int, timeout: int, with_ssl: bool) -> smtplib.SMTP: - """ - Return an SMTP connection to the specified host and port, with optional SSL encryption. - - :param host: The hostname or IP address of the SMTP server. - :param port: The port number to connect to on the SMTP server. - :param timeout: The timeout in seconds for the connection. - :param with_ssl: Whether to use SSL encryption for the connection. - :return: An SMTP connection to the specified host and port. - """ - if not with_ssl: - return smtplib.SMTP(host=host, port=port, timeout=timeout) - return smtplib.SMTP_SSL(host=host, port=port, timeout=timeout, context=_get_ssl_context()) - - -def _get_email_list_from_str(addresses: str) -> list[str]: - """ - Extract a list of email addresses from a string. - - The string can contain multiple email addresses separated - by any of the following delimiters: ',' or ';'. - - :param addresses: A string containing one or more email addresses. - :return: A list of email addresses. - """ - pattern = r"\s*[,;]\s*" - return re.split(pattern, addresses) +_MOVED_TO_PROVIDER = { + "send_email": "airflow.providers.smtp.utils.emailer.send_email", + "send_email_smtp": "airflow.providers.smtp.utils.emailer.send_email_smtp", + "build_mime_message": "airflow.providers.smtp.utils.emailer.build_mime_message", + "send_mime_email": "airflow.providers.smtp.utils.emailer.send_mime_email", + "get_email_address_list": "airflow.providers.smtp.utils.emailer.get_email_address_list", +} + +# Keeps "from airflow.utils.email import *" importing the deprecated names. +__all__ = list(_MOVED_TO_PROVIDER) + +add_deprecated_classes( + {__name__: _MOVED_TO_PROVIDER}, + package=__name__, + extra_message=( + "The provider looks up the SMTP Connection with BaseHook.get_connection and reads settings through " + "the Task SDK configuration, so values changed at runtime with airflow.configuration.conf.set() are " + "not seen" + ), +) diff --git a/airflow-core/tests/unit/utils/test_email.py b/airflow-core/tests/unit/utils/test_email.py index 33f242fc72f85..a772fe27e0624 100644 --- a/airflow-core/tests/unit/utils/test_email.py +++ b/airflow-core/tests/unit/utils/test_email.py @@ -17,404 +17,85 @@ # under the License. from __future__ import annotations -import json -import os -from email.mime.application import MIMEApplication from email.mime.multipart import MIMEMultipart -from email.mime.text import MIMEText -from smtplib import SMTPServerDisconnected from unittest import mock import pytest -from airflow.configuration import conf +from airflow.providers.smtp.utils import emailer from airflow.utils import email +from airflow.utils.deprecation_tools import DeprecatedImportWarning -from tests_common.test_utils.config import conf_vars +DEPRECATED_NAMES = [ + "send_email", + "send_email_smtp", + "build_mime_message", + "send_mime_email", + "get_email_address_list", +] -EMAILS = ["test1@example.com", "test2@example.com"] -send_email_test = mock.MagicMock() +@pytest.mark.parametrize("name", DEPRECATED_NAMES) +def test_deprecated_names_resolve_to_provider(name): + with pytest.warns( + DeprecatedImportWarning, match=f"airflow.providers.smtp.utils.emailer.{name}" + ) as record: + assert getattr(email, name) is getattr(emailer, name) + assert "BaseHook.get_connection" in str(record[0].message) -class TestEmail: - def setup_method(self): - conf.remove_option("email", "EMAIL_BACKEND") - conf.remove_option("email", "EMAIL_CONN_ID") - def test_get_email_address_single_email(self): - emails_string = "test1@example.com" +def test_star_import_exposes_deprecated_names(): + namespace: dict = {} + with pytest.warns(DeprecatedImportWarning): + exec("from airflow.utils.email import *", namespace) - assert email.get_email_address_list(emails_string) == [emails_string] - - def test_get_email_address_comma_sep_string(self): - emails_string = "test1@example.com, test2@example.com" - - assert email.get_email_address_list(emails_string) == EMAILS - - def test_get_email_address_colon_sep_string(self): - emails_string = "test1@example.com; test2@example.com" - - assert email.get_email_address_list(emails_string) == EMAILS - - def test_get_email_address_list(self): - emails_list = ["test1@example.com", "test2@example.com"] - - assert email.get_email_address_list(emails_list) == EMAILS - - def test_get_email_address_tuple(self): - emails_tuple = ("test1@example.com", "test2@example.com") - - assert email.get_email_address_list(emails_tuple) == EMAILS - - def test_get_email_address_invalid_type(self): - emails_string = 1 - - with pytest.raises(TypeError): - email.get_email_address_list(emails_string) - - def test_get_email_address_invalid_type_in_iterable(self): - emails_list = ["test1@example.com", 2] - - with pytest.raises(TypeError): - email.get_email_address_list(emails_list) - - @mock.patch("airflow.utils.email.send_email") - def test_default_backend(self, mock_send_email): - res = email.send_email("to", "subject", "content") - mock_send_email.assert_called_once_with("to", "subject", "content") - assert mock_send_email.return_value == res - - @mock.patch("airflow.utils.email.send_email_smtp") - def test_custom_backend(self, mock_send_email): - with conf_vars( - { - ("email", "email_backend"): "unit.utils.test_email.send_email_test", - ("email", "email_conn_id"): "smtp_default", - } - ): - email.send_email("to", "subject", "content") - send_email_test.assert_called_once_with( - "to", - "subject", - "content", - files=None, - dryrun=False, - cc=None, - bcc=None, - mime_charset="utf-8", - mime_subtype="mixed", - conn_id="smtp_default", - from_email=None, - custom_headers=None, - ) - assert not mock_send_email.called - - @mock.patch("airflow.utils.email.send_email_smtp") - @conf_vars( - { - ("email", "email_backend"): "unit.utils.test_email.send_email_test", - ("email", "email_conn_id"): "smtp_default", - ("email", "from_email"): "from@test.com", - } - ) - def test_custom_backend_sender(self, mock_send_email_smtp): - email.send_email("to", "subject", "content") - _, call_kwargs = send_email_test.call_args - assert call_kwargs["from_email"] == "from@test.com" - assert not mock_send_email_smtp.called - - def test_build_mime_message(self): - mail_from = "from@example.com" - mail_to = "to@example.com" - subject = "test subject" - html_content = "Test" - custom_headers = {"Reply-To": "reply_to@example.com"} - - msg, recipients = email.build_mime_message( - mail_from=mail_from, - to=mail_to, - subject=subject, - html_content=html_content, - custom_headers=custom_headers, - ) - - assert "From" in msg - assert "To" in msg - assert "Subject" in msg - assert "Reply-To" in msg - assert [mail_to] == recipients - assert msg["To"] == ",".join(recipients) - - def test_build_mime_message_escapes_attachment_filename(self, tmp_path): - # A quote in the filename must not break out of the quoted - # Content-Disposition value and inject extra parameters. - malicious = 'report.txt"; x-evil="1' - attachment = tmp_path / malicious - attachment.write_bytes(b"data") - - msg, _ = email.build_mime_message( - mail_from="from@example.com", - to="to@example.com", - subject="subject", - html_content="", - files=[os.fspath(attachment)], - ) - - part = msg.get_payload()[-1] - assert part.get_filename() == malicious - assert "x-evil" not in dict(part.get_params(header="Content-Disposition")) + assert {name: namespace.get(name) for name in DEPRECATED_NAMES} == { + name: getattr(emailer, name) for name in DEPRECATED_NAMES + } @pytest.mark.db_test -class TestEmailSmtp: - @pytest.fixture(autouse=True) - def setup_test_cases(self, monkeypatch): - monkeypatch.setenv( # Set the default smtp connection for all test cases - "AIRFLOW_CONN_SMTP_DEFAULT", - json.dumps({"conn_type": "smtp", "login": "user", "password": "p@$$word"}), - ) - - @mock.patch("airflow.utils.email.send_mime_email") - def test_send_smtp(self, mock_send_mime, tmp_path): - path = tmp_path / "testfile" - path.write_text("attachment") - email.send_email_smtp("to", "subject", "content", files=[os.fspath(path)]) - assert mock_send_mime.called - _, call_args = mock_send_mime.call_args - assert conf.get("smtp", "SMTP_MAIL_FROM") == call_args["e_from"] - assert call_args["e_to"] == ["to"] - msg = call_args["mime_msg"] - assert msg["Subject"] == "subject" - assert conf.get("smtp", "SMTP_MAIL_FROM") == msg["From"] - assert len(msg.get_payload()) == 2 - filename = f'attachment; filename="{path.name}"' - assert filename == msg.get_payload()[-1].get("Content-Disposition") - mimeapp = MIMEApplication("attachment") - assert mimeapp.get_payload() == msg.get_payload()[-1].get_payload() - - @mock.patch("airflow.utils.email.send_mime_email") - def test_send_smtp_with_multibyte_content(self, mock_send_mime): - email.send_email_smtp("to", "subject", "🔥", mime_charset="utf-8") - assert mock_send_mime.called - _, call_args = mock_send_mime.call_args - msg = call_args["mime_msg"] - mimetext = MIMEText("🔥", "mixed", "utf-8") - assert mimetext.get_payload() == msg.get_payload()[0].get_payload() - - @mock.patch("airflow.utils.email.send_mime_email") - def test_send_bcc_smtp(self, mock_send_mime, tmp_path): - path = tmp_path / "testfile" - path.write_text("attachment") - email.send_email_smtp( - "to", - "subject", - "content", - files=[os.fspath(path)], - cc="cc", - bcc="bcc", - custom_headers={"Reply-To": "reply_to@example.com"}, - ) - assert mock_send_mime.called - _, call_args = mock_send_mime.call_args - assert conf.get("smtp", "SMTP_MAIL_FROM") == call_args["e_from"] - assert call_args["e_to"] == ["to", "cc", "bcc"] - msg = call_args["mime_msg"] - assert msg["Subject"] == "subject" - assert conf.get("smtp", "SMTP_MAIL_FROM") == msg["From"] - assert len(msg.get_payload()) == 2 - assert f'attachment; filename="{path.name}"' == msg.get_payload()[-1].get("Content-Disposition") - mimeapp = MIMEApplication("attachment") - assert mimeapp.get_payload() == msg.get_payload()[-1].get_payload() - assert msg["Reply-To"] == "reply_to@example.com" - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_airflow_config(self, mock_smtp, mock_smtp_ssl, monkeypatch): - monkeypatch.delenv("AIRFLOW_CONN_SMTP_DEFAULT", raising=False) - mock_smtp.return_value = mock.Mock() - msg = MIMEMultipart() - email.send_mime_email("from", "to", msg, dryrun=False) - mock_smtp.assert_called_once_with( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - ) - assert not mock_smtp_ssl.called - assert mock_smtp.return_value.starttls.called - mock_smtp.return_value.sendmail.assert_called_once_with("from", "to", msg.as_string()) - assert mock_smtp.return_value.quit.called - - @mock.patch("smtplib.SMTP") - def test_send_mime_conn_id(self, mock_smtp, monkeypatch): - monkeypatch.setenv( - "AIRFLOW_CONN_SMTP_TEST_CONN", - json.dumps({"conn_type": "smtp", "login": "test-user", "password": "test-p@$$word"}), - ) - msg = MIMEMultipart() - email.send_mime_email("from", "to", msg, dryrun=False, conn_id="smtp_test_conn") - mock_smtp.return_value.login.assert_called_once_with("test-user", "test-p@$$word") - mock_smtp.return_value.sendmail.assert_called_once_with("from", "to", msg.as_string()) - assert mock_smtp.return_value.quit.called +class TestDeprecatedSendWithDbOnlyConnection: + """Which processes find an SMTP Connection that exists only in the metadata database.""" - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_ssl_none_context(self, mock_smtp, mock_smtp_ssl): - mock_smtp_ssl.return_value = mock.Mock() - with conf_vars({("smtp", "smtp_ssl"): "True", ("email", "ssl_context"): "none"}): - email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False) - assert not mock_smtp.called - mock_smtp_ssl.assert_called_once_with( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - context=None, - ) - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - @mock.patch("ssl.create_default_context") - def test_send_mime_ssl_default_context_if_not_set(self, create_default_context, mock_smtp, mock_smtp_ssl): - mock_smtp_ssl.return_value = mock.Mock() - with conf_vars({("smtp", "smtp_ssl"): "True"}): - email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False) - assert not mock_smtp.called - assert create_default_context.called - mock_smtp_ssl.assert_called_once_with( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - context=create_default_context.return_value, - ) - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - @mock.patch("ssl.create_default_context") - def test_send_mime_ssl_default_context_with_value_set_to_default( - self, create_default_context, mock_smtp, mock_smtp_ssl + @pytest.fixture(autouse=True) + def _db_only_connection(self, monkeypatch): + from airflow.models import Connection + from airflow.utils.session import create_session + + from tests_common.test_utils.db import clear_db_connections + + monkeypatch.delenv("AIRFLOW_CONN_SMTP_DB_ONLY", raising=False) + monkeypatch.delenv("_AIRFLOW_PROCESS_CONTEXT", raising=False) + clear_db_connections(add_default_connections_back=False) + with create_session() as session: + session.add( + Connection(conn_id="smtp_db_only", conn_type="smtp", login="db-user", password="db-pass") + ) + yield + clear_db_connections() + + @pytest.mark.parametrize( + ("process_context", "expected_login"), + [ + pytest.param("server", ("db-user", "db-pass"), id="server-process-finds-db-connection"), + pytest.param(None, None, id="plain-script-sends-without-login"), + ], + ) + @mock.patch("smtplib.SMTP", autospec=True) + def test_send_mime_email_db_only_connection( + self, mock_smtp, monkeypatch, process_context, expected_login ): - mock_smtp_ssl.return_value = mock.Mock() - with conf_vars({("smtp", "smtp_ssl"): "True", ("email", "ssl_context"): "default"}): - email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False) - assert not mock_smtp.called - assert create_default_context.called - mock_smtp_ssl.assert_called_once_with( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - context=create_default_context.return_value, - ) - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_noauth(self, mock_smtp, mock_smtp_ssl): - mock_smtp.return_value = mock.Mock() - with conf_vars( - { - ("smtp", "smtp_user"): None, - ("smtp", "smtp_password"): None, - } - ): - email.send_mime_email("from", "to", MIMEMultipart(), dryrun=False) - assert not mock_smtp_ssl.called - mock_smtp.assert_called_once_with( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - ) - assert not mock_smtp.login.called - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_dryrun(self, mock_smtp, mock_smtp_ssl): - email.send_mime_email("from", "to", MIMEMultipart(), dryrun=True) - assert not mock_smtp.called - assert not mock_smtp_ssl.called - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_complete_failure(self, mock_smtp: mock.Mock, mock_smtp_ssl: mock.Mock): - mock_smtp.side_effect = SMTPServerDisconnected() - msg = MIMEMultipart() - with pytest.raises(SMTPServerDisconnected): - email.send_mime_email("from", "to", msg, dryrun=False) - mock_smtp.assert_any_call( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - ) - assert mock_smtp.call_count == conf.getint("smtp", "SMTP_RETRY_LIMIT") + 1 - assert not mock_smtp_ssl.called - assert not mock_smtp.return_value.starttls.called - assert not mock_smtp.return_value.login.called - assert not mock_smtp.return_value.sendmail.called - assert not mock_smtp.return_value.quit.called - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - @mock.patch("ssl.create_default_context") - def test_send_mime_ssl_complete_failure(self, create_default_context, mock_smtp, mock_smtp_ssl): - mock_smtp_ssl.side_effect = SMTPServerDisconnected() - msg = MIMEMultipart() - with conf_vars({("smtp", "smtp_ssl"): "True"}): - with pytest.raises(SMTPServerDisconnected): - email.send_mime_email("from", "to", msg, dryrun=False) - - mock_smtp_ssl.assert_any_call( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - context=create_default_context.return_value, - ) - assert create_default_context.called - assert mock_smtp_ssl.call_count == conf.getint("smtp", "SMTP_RETRY_LIMIT") + 1 - assert not mock_smtp.called - assert not mock_smtp_ssl.return_value.starttls.called - assert not mock_smtp_ssl.return_value.login.called - assert not mock_smtp_ssl.return_value.sendmail.called - assert not mock_smtp_ssl.return_value.quit.called - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_custom_timeout_retrylimit(self, mock_smtp, mock_smtp_ssl): - mock_smtp.side_effect = SMTPServerDisconnected() - msg = MIMEMultipart() - - custom_retry_limit = 10 - custom_timeout = 60 - - with conf_vars( - { - ("smtp", "smtp_retry_limit"): str(custom_retry_limit), - ("smtp", "smtp_timeout"): str(custom_timeout), - } - ): - with pytest.raises(SMTPServerDisconnected): - email.send_mime_email("from", "to", msg, dryrun=False) - - mock_smtp.assert_any_call( - host=conf.get("smtp", "SMTP_HOST"), port=conf.getint("smtp", "SMTP_PORT"), timeout=custom_timeout - ) - assert not mock_smtp_ssl.called - assert mock_smtp.call_count == custom_retry_limit + 1 - - @mock.patch("smtplib.SMTP_SSL") - @mock.patch("smtplib.SMTP") - def test_send_mime_partial_failure(self, mock_smtp, mock_smtp_ssl): - final_mock = mock.Mock() - side_effects = [SMTPServerDisconnected(), SMTPServerDisconnected(), final_mock] - mock_smtp.side_effect = side_effects - msg = MIMEMultipart() - - email.send_mime_email("from", "to", msg, dryrun=False) - - mock_smtp.assert_any_call( - host=conf.get("smtp", "SMTP_HOST"), - port=conf.getint("smtp", "SMTP_PORT"), - timeout=conf.getint("smtp", "SMTP_TIMEOUT"), - ) - assert mock_smtp.call_count == side_effects.index(final_mock) + 1 - assert not mock_smtp_ssl.called - assert final_mock.starttls.called - final_mock.sendmail.assert_called_once_with("from", "to", msg.as_string()) - assert final_mock.quit.called + if process_context: + monkeypatch.setenv("_AIRFLOW_PROCESS_CONTEXT", process_context) + with pytest.warns(DeprecatedImportWarning): + send_mime_email = email.send_mime_email + + send_mime_email("from@example.com", ["to@example.com"], MIMEMultipart(), conn_id="smtp_db_only") + + if expected_login: + mock_smtp.return_value.login.assert_called_once_with(*expected_login) + else: + mock_smtp.return_value.login.assert_not_called() + assert mock_smtp.return_value.sendmail.called