Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
17bfe78
Make common.ai hooks report misconfiguration instead of failing silently
Lee-W Oct 7, 2026
f7ab6f8
Skip the LLMRetryPolicy example Dag on cores without RetryPolicy support
Lee-W Oct 8, 2026
12a7cbb
Describe what the keyword-only model argument tests pin
Lee-W Oct 8, 2026
c4374b6
Keep a null MCP stdio timeout from falling back to fastmcp's global i…
Lee-W Oct 8, 2026
c99c332
Accept the MCP stdio args shapes the connection form produces
Lee-W Oct 8, 2026
872b152
Fail closed when a provider class rejects the connection's credentials
Lee-W Oct 8, 2026
ea7cccf
Stop the Snowflake Cortex test patching the removed env-var fallback
Lee-W Oct 9, 2026
2633c28
Show which fields a provider class rejected when failing closed
Lee-W Oct 10, 2026
5f9b4da
Document that a missing mcp extra raises AirflowOptionalProviderFeatu…
Lee-W Oct 10, 2026
ba6b4d7
State that google: on the Vertex connection needs api_key and no Vert…
Lee-W Oct 12, 2026
57450dc
Resolve the retry policies' hook through get_hook() so Azure connecti…
Lee-W Oct 14, 2026
31018e0
Say the provider could not be built from the connection's fields, not…
Lee-W Oct 14, 2026
df8fa69
Describe what the Azure and Vertex connections forward to a prefixed …
Lee-W Oct 14, 2026
6a42d17
Stop echoing the MCP stdio args value in its validation errors
Lee-W Oct 19, 2026
1dad87b
Name GoogleCloudProvider as the class that takes the Vertex credentials
Lee-W Oct 19, 2026
64db0e8
Announce the breaking changes to the 1.0.0 hooks and retry policy in …
Lee-W Oct 23, 2026
8c01407
Describe the fail-closed ValueError, not the retired fallback, in the…
Lee-W Oct 23, 2026
b285246
Deprecate passing llm_model and embed_model positionally to the Llama…
Lee-W Oct 25, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions providers/common/ai/docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,38 @@ Changelog
``"PT1M30S"`` for a ``timedelta``. Tuples and sets in that model now arrive as lists. See
:doc:`structured_output`.

.. warning::
The following changes are breaking for code written against 1.0.0:

* ``PydanticAIHook``: when a connection supplies credentials (password or host) and the provider class
rejects the resulting keyword arguments (``TypeError``), the hook now raises ``ValueError``. In 1.0.0
it logged a warning and fell back to environment-variable authentication, which could authenticate
as a different identity than the connection. Fix the connection fields for that provider, or remove
the credentials from the connection if environment-variable authentication is what you want.
* ``airflow.providers.common.ai.policies.retry``: importing it without the Airflow ``RetryPolicy``
support (Airflow older than 3.3) now raises ``AirflowOptionalProviderFeatureException`` instead of
``ImportError``. It is not an ``ImportError`` subclass (it derives from ``AirflowException``), so a
Dag that guards the import with ``except ImportError:`` as in the 1.0.0 example Dag now fails to
parse on those versions. Catch both, importing the exception from
``airflow.providers.common.compat.sdk`` (works on Airflow 2.11 and 3.x)::

from airflow.providers.common.compat.sdk import AirflowOptionalProviderFeatureException

try:
from airflow.providers.common.ai.policies.retry import LLMRetryPolicy
except (ImportError, AirflowOptionalProviderFeatureException):
LLMRetryPolicy = None

.. note::
``LlamaIndexHook`` and ``LangChainHook`` (both experimental, see :doc:`stability`):
passing ``embed_model`` and ``llm_model`` positionally is deprecated. In 1.0.0 they were the third and
fourth positional parameters (after ``llm_conn_id`` and ``embed_conn_id``), and the two hooks listed
them in opposite order (``embed_model`` then ``llm_model`` for ``LlamaIndexHook``, ``llm_model`` then
``embed_model`` for ``LangChainHook``). Positional calls still work and emit an
``AirflowProviderDeprecationWarning``, but support will be removed in a future release. Pass them by
keyword, e.g. ``LlamaIndexHook(llm_model="gpt-5")``, which also avoids the meanings swapping between
the two hooks.

1.1.0
.....

Expand Down
3 changes: 2 additions & 1 deletion providers/common/ai/docs/connections/mcp.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ Environment (Extra field)

Timeout (Extra field)
Connection init timeout in seconds for the ``stdio`` transport. Ignored
for ``http``/``sse``. Default: ``10``.
for ``http``/``sse``. Must be a non-negative finite number; ``0`` or ``null``
disables the timeout. Default: ``10``.

Examples: ``30``

Expand Down
13 changes: 9 additions & 4 deletions providers/common/ai/docs/connections/pydantic_ai_azure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,15 @@ Model
this connection type's own platform, so nothing else needs naming it
explicitly. Writing the ``azure:`` prefix yourself has the same effect and is
still accepted. A name prefixed with a *different*, recognized platform (e.g.
``openai:gpt-5``) is used verbatim instead, pinning that platform and
bypassing Azure OpenAI entirely -- a name is only treated as already prefixed
when the segment before its first ``:`` is itself a real pydantic-ai
provider, not merely present.
``openai:gpt-5``) is used verbatim instead -- a name is only treated as
already prefixed when the segment before its first ``:`` is itself a real
pydantic-ai provider, not merely present. That other platform's provider
class is built from the fields this connection forwards (``API Key``,
``Azure Endpoint`` as ``azure_endpoint``, ``API Version``), so it works only
if it accepts all of them. ``OpenAIProvider`` does not accept
``azure_endpoint`` or ``api_version``, so ``openai:`` with an endpoint or API
version set fails with a ``ValueError`` naming the fields passed; it never
silently falls back to environment-variable credentials.

API Key (Password field)
The Azure OpenAI API key.
Expand Down
10 changes: 7 additions & 3 deletions providers/common/ai/docs/connections/pydantic_ai_vertex.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,12 @@ Model
mean Vertex AI Express Mode credentials (see "Credentials" below), so its
presence alone cannot tell the two platforms apart. To reach the
Generative Language API instead, prefix the model explicitly with
``google:`` -- that spelling routes to a different provider regardless of
which fields this connection sets.
``google:``. The ``google:`` provider accepts only the ``API Key`` and
``Custom Endpoint URL`` fields (without an API key it reads ``GOOGLE_API_KEY``), so on this
connection type that spelling must not be combined with Vertex-only fields
(``GCP Project``, ``Location / Region``, ``Service Account Info``): the
connection fails with a ``ValueError`` instead of silently falling back to
environment-variable credentials.

GCP Project
Google Cloud project ID. Falls back to the ``GOOGLE_CLOUD_PROJECT``
Expand Down Expand Up @@ -87,7 +91,7 @@ API Key
``project`` / ``location`` / ``service_account_info`` (those select the
credentials/ADC path instead, which takes precedence and nulls the API
key). For the Generative Language API
(non-Vertex, API-key-only), use the ``google:`` prefix on the generic
(non-Vertex; API key or ``GOOGLE_API_KEY``), use the ``google:`` prefix on the generic
:doc:`pydantic_ai` connection instead.

Service Account Info
Expand Down
16 changes: 6 additions & 10 deletions providers/common/ai/docs/provider_fallback.rst
Original file line number Diff line number Diff line change
Expand Up @@ -198,16 +198,12 @@ Two checks, neither of which requires waiting for a real outage:

*Test the connection.* ``test_connection`` on the primary resolves every connection in the
chain, so a fallback with a missing ``model`` or an unknown connection ID is reported by name
there rather than discovered mid-incident. Credential fields a provider class rejects with a
``TypeError`` are caught by the hook, which retries with the env-var-based provider
constructor and logs a warning either way; if the required env var is also missing, that
retry raises ``pydantic_ai.exceptions.UserError``, which ``test_connection`` does surface
since it wraps the whole resolution in a broad exception handler. What it cannot show is the
opposite case: the env var *is* set on the worker, the retry quietly succeeds, and
``test_connection`` reports success even though the credentials you configured on the
connection were silently ignored -- check the logs for that warning rather than relying on
``test_connection`` alone. It also does not call the provider, so a well-formed but revoked
key still passes -- that is what the drill below is for.
there rather than discovered mid-incident. Fields a provider class cannot be built from (a
``TypeError``) fail the resolution with a ``ValueError`` naming the connection, the provider and
the fields passed, with the provider's ``TypeError`` text -- the hook never falls back to
environment-variable auth, which would authenticate as a different identity -- and
``test_connection`` reports it as a failure. It does not call the provider, so a well-formed
but revoked key still passes -- that is what the drill below is for.

*Drill it.* Point the primary at an endpoint nothing listens on and run the Dag. The task
should still succeed, and the run summary in its log names the model that answered:
Expand Down
10 changes: 5 additions & 5 deletions providers/common/ai/docs/self_hosted_models.rst
Original file line number Diff line number Diff line change
Expand Up @@ -317,11 +317,11 @@ its OpenAI provider, and there's no separate ``ollama`` extra.

``LiteLLMProvider`` is an exception: its constructor takes ``api_base``
instead of ``base_url``, so passing ``host`` raises a ``TypeError`` that
the hook catches, logs a warning, and falls back to environment-variable
auth, ignoring ``host`` entirely. Point a LiteLLM proxy's OpenAI-compatible
endpoint via ``openai:<model>`` instead -- this is the same proxy role
used in the gateway section above, just addressed through the working
prefix rather than the rejected ``litellm:`` one.
the hook re-raises as a ``ValueError``: the connection is rejected rather
than falling back to environment-variable auth. Point a LiteLLM proxy's
OpenAI-compatible endpoint via ``openai:<model>`` instead -- this is the
same proxy role used in the gateway section above, just addressed through
the working prefix rather than the rejected ``litellm:`` one.

.. _self_hosted_models_other_servers:

Expand Down
9 changes: 5 additions & 4 deletions providers/common/ai/docs/toolsets/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,11 @@ Every toolset imports from the ``airflow.providers.common.ai.toolsets`` package
SQLToolset,
)

``SQLToolset`` and ``DataFusionToolset`` depend on optional extras. If the extra is not
installed, the first access to the name raises ``AirflowOptionalProviderFeatureException``.
``MCPToolset`` imports without its extra; the missing dependency raises ``ImportError``
when the toolset first connects to the MCP server.
``SQLToolset``, ``DataFusionToolset`` and ``MCPToolset`` depend on optional extras. If the
extra is not installed, they raise ``AirflowOptionalProviderFeatureException``. For
``SQLToolset`` and ``DataFusionToolset`` this happens on the first access to the name.
``MCPToolset`` imports without its extra; it raises only when the toolset first connects
to the MCP server.

Every toolset here implements pydantic-ai's
`AbstractToolset <https://ai.pydantic.dev/toolsets/>`__ interface, so it works in any
Expand Down
2 changes: 1 addition & 1 deletion providers/common/ai/provider.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ connection-types:
- 'null'
vertexai:
label: Force Vertex AI Mode
description: "Ignored (kept for compatibility); mode is now selected via the Model field's prefix."
description: "Deprecated and ignored; the Model field's prefix selects the mode."
schema:
type:
- boolean
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
from dataclasses import replace
from datetime import timedelta

from airflow.providers.common.compat.sdk import dag, task
from airflow.providers.common.compat.sdk import AirflowOptionalProviderFeatureException, dag, task

try:
from airflow.providers.common.ai.policies.retry import (
Expand Down Expand Up @@ -181,6 +181,7 @@ def task_ambiguous():
task_ambiguous()

example_llm_retry_policy_classifier()
except ImportError:
# RetryPolicy requires Airflow 3.3+; example DAG is skipped on older versions.
except (ImportError, AirflowOptionalProviderFeatureException):
# RetryPolicy requires Airflow 3.3+; policies/retry.py raises AirflowOptionalProviderFeatureException
# (not an ImportError) on older versions, so the example Dag is skipped there.
pass
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ def get_provider_info():
},
"vertexai": {
"label": "Force Vertex AI Mode",
"description": "Ignored (kept for compatibility); mode is now selected via the Model field's prefix.",
"description": "Deprecated and ignored; the Model field's prefix selects the mode.",
"schema": {"type": ["boolean", "null"]},
},
"api_key": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,10 @@

from __future__ import annotations

import warnings
from typing import TYPE_CHECKING, Any

from airflow.exceptions import AirflowProviderDeprecationWarning
from airflow.providers.common.compat.sdk import BaseHook

if TYPE_CHECKING:
Expand Down Expand Up @@ -68,12 +70,13 @@ class LangChainHook(BaseHook):
single hook instance.
:param llm_model: Chat model identifier in ``provider:name`` format
(e.g. ``"openai:gpt-5"``, ``"anthropic:claude-sonnet-5"``).
Overrides ``extra["model"]`` on the connection.
Overrides ``extra["model"]`` on the connection. Passing it positionally
(1.0.0 order: ``llm_model, embed_model``) is deprecated.
:param embed_model: Embedding model identifier in ``provider:name`` format
(e.g. ``"openai:text-embedding-3-small"``). Overrides
``extra["embed_model"]`` on the connection. When ``embedding_kwargs``
supplies ``provider`` explicitly, use a model name without the provider
prefix.
prefix. Passing it positionally (1.0.0 order: ``llm_model, embed_model``) is deprecated.
:param embedding_kwargs: Additional keyword arguments to pass to the embedding
model constructor without filtering. Values can override hook-provided
settings, including the endpoint and credentials. In particular,
Expand All @@ -96,12 +99,32 @@ def __init__(
self,
llm_conn_id: str | None = None,
embed_conn_id: str | None = None,
*args: Any,
llm_model: str | None = None,
embed_model: str | None = None,
*,
embedding_kwargs: dict[str, Any] | None = None,
**kwargs: Any,
) -> None:
# ``*args`` only exists to keep the 1.0.0 positional order working: ``llm_model``, ``embed_model``.
if len(args) > 2:
raise TypeError(
f"{type(self).__name__}.__init__() takes from 1 to 5 positional arguments "
f"but {3 + len(args)} were given"
)
legacy_values = dict(zip(("llm_model", "embed_model"), args))
keyword_values = {"llm_model": llm_model, "embed_model": embed_model}
for name in legacy_values:
# A keyword of None counts as not provided, so ``name=None`` plus a positional keeps the positional.
if keyword_values[name] is not None:
raise TypeError(f"{type(self).__name__}.__init__() got multiple values for argument '{name}'")
warnings.warn(
f"Parameter `{name}` passed positionally is deprecated. Use `{name}=...` instead; "
f"positional support will be removed in a future release.",
category=AirflowProviderDeprecationWarning,
stacklevel=2,
)
llm_model = legacy_values.get("llm_model", llm_model)
embed_model = legacy_values.get("embed_model", embed_model)
super().__init__(**kwargs)
# Resolve at runtime so a future subclass with its own
# `default_conn_name` is honoured. A bare `llm_conn_id: str =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,10 @@
from __future__ import annotations

import inspect
import warnings
from typing import TYPE_CHECKING, Any

from airflow.exceptions import AirflowProviderDeprecationWarning
from airflow.providers.common.compat.sdk import (
AirflowOptionalProviderFeatureException,
BaseHook,
Expand Down Expand Up @@ -75,7 +77,8 @@ class LlamaIndexHook(BaseHook):
still fails on the model name, not on connectivity.
``get_embedding_model()`` raises immediately at construction;
``get_llm()`` defers the error until the first call that reads
``.metadata`` (``.chat()`` / ``.complete()``).
``.metadata`` (``.chat()`` / ``.complete()``); ``test_connection()``
reads it, so it reports such a model name as a failure.

Connection fields:

Expand All @@ -95,10 +98,10 @@ class LlamaIndexHook(BaseHook):
embedding provider. Falls back to ``llm_conn_id`` when not set.
:param embed_model: Embedding model name (e.g.
``"text-embedding-3-small"``). Overrides ``extra["embed_model"]``
on the connection.
on the connection. Passing it positionally (1.0.0 order: ``embed_model, llm_model``) is deprecated.
:param llm_model: LLM model name (e.g. ``"gpt-5"``). Overrides
``extra["llm_model"]`` on the connection. Required when calling
:meth:`get_llm`.
:meth:`get_llm`. Passing it positionally (1.0.0 order: ``embed_model, llm_model``) is deprecated.
:param embedding_kwargs: Additional keyword arguments to pass to the embedding
model constructor without filtering. Connection ``api_key`` and ``api_base``
values take precedence at the top level, but nested options supported by the
Expand All @@ -115,12 +118,32 @@ def __init__(
self,
llm_conn_id: str | None = None,
embed_conn_id: str | None = None,
embed_model: str | None = None,
*args: Any,
llm_model: str | None = None,
*,
embed_model: str | None = None,
embedding_kwargs: dict[str, Any] | None = None,
**kwargs: Any,
) -> None:
# ``*args`` only exists to keep the 1.0.0 positional order working: ``embed_model``, ``llm_model``.
if len(args) > 2:
raise TypeError(
f"{type(self).__name__}.__init__() takes from 1 to 5 positional arguments "
f"but {3 + len(args)} were given"
)
legacy_values = dict(zip(("embed_model", "llm_model"), args))
keyword_values = {"embed_model": embed_model, "llm_model": llm_model}
for name in legacy_values:
# A keyword of None counts as not provided, so ``name=None`` plus a positional keeps the positional.
if keyword_values[name] is not None:
raise TypeError(f"{type(self).__name__}.__init__() got multiple values for argument '{name}'")
warnings.warn(
f"Parameter `{name}` passed positionally is deprecated. Use `{name}=...` instead; "
f"positional support will be removed in a future release.",
category=AirflowProviderDeprecationWarning,
stacklevel=2,
)
embed_model = legacy_values.get("embed_model", embed_model)
llm_model = legacy_values.get("llm_model", llm_model)
super().__init__(**kwargs)
# Resolve at runtime so a future per-vendor subclass with its own
# ``default_conn_name`` is honoured.
Expand Down Expand Up @@ -233,15 +256,16 @@ def get_llm(self) -> LLM:

def test_connection(self) -> tuple[bool, str]:
"""
Test connection by resolving the LLM.
Test connection by resolving the LLM and reading its metadata.

Validates that the model identifier is valid and the provider can be
instantiated with the supplied credentials. Does NOT make an LLM API
call -- that would be expensive and fail for reasons unrelated to
connectivity (quotas, billing, rate limits).
Reading ``.metadata`` makes LlamaIndex validate the model name against its
OpenAI-only allowlist, so an Ollama or vLLM model name fails here instead of at the
first ``.chat()`` call. Does NOT make an LLM API call -- that would be expensive and
fail for reasons unrelated to connectivity (quotas, billing, rate limits), so a
revoked API key or an unreachable **host** still passes.
"""
try:
self.get_llm()
self.get_llm().metadata
return True, "Model resolved successfully."
except Exception as e:
return False, str(e)
Loading