Skip to content

Make common.ai hooks report misconfiguration instead of failing silently - #74451

Merged
Lee-W merged 18 commits into
apache:mainfrom
astronomer:common-ai-hooks-hardening
Oct 26, 2026
Merged

Lee-W merged 18 commits into
apache:mainfrom
astronomer:common-ai-hooks-hardening

Conversation

@Lee-W

@Lee-W Lee-W commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Why

apache-airflow-providers-common-ai 1.0.0 shipped on 2026-10-19, and the hooks it exposes are now public API. This PR fixes silent failures and signature inconsistencies in the hooks, and announces the behaviour changes in the changelog.

What

  • PydanticAIHook now fails closed when a provider class rejects the fields forwarded from the connection: get_conn() raises a ValueError naming the connection, the provider, the fields passed and the provider's TypeError text instead of rebuilding the provider from environment variables, so test_connection reports the failure too. This also covers connections in fallback_conn_ids. docs/provider_fallback.rst is updated to match.
  • MCPHook validates the stdio args extra and the timeout extra. args may be a list of strings, a single string, a JSON array string, or null (treated as empty); timeout must be a non-negative finite number, and 0 and null both mean no init timeout (null is mapped to 0 so FASTMCP_CLIENT_INIT_TIMEOUT cannot re-enable one). Invalid values raise ValueError naming the connection. docs/connections/mcp.rst is updated.
  • MCPHook.get_conn raises AirflowOptionalProviderFeatureException instead of a bare ImportError when the mcp dependency is missing. Importing the LLMRetryPolicy module on Airflow < 3.3 now raises the same exception.
  • LlamaIndexHook.test_connection now reads the LLM's .metadata, so a model name that LlamaIndex's OpenAI class rejects (for example an Ollama or vLLM model) is reported as a failure. Before, it passed and failed only on the first .chat() call. It still does not call the LLM, so a revoked key or unreachable host still passes (docstring updated).
  • Deprecated: passing llm_model and embed_model positionally to LlamaIndexHook and LangChainHook emits an AirflowProviderDeprecationWarning. The two hooks had these two parameters in opposite positional order, so positional calls were easy to get wrong. Pass them by keyword. Positional calls still work until support is removed in a future release.
  • The vertexai connection field is labelled "Deprecated and ignored" in provider.yaml and get_provider_info.py. Behaviour is
    unchanged, since the field was already ignored.

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch 3 times, most recently from e11d2ca to 727b835 Compare October 9, 2026 07:01
@Lee-W
Lee-W requested a review from potiuk as a code owner October 9, 2026 07:01
@Lee-W
Lee-W requested a review from kaxil October 9, 2026 07:34
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch 2 times, most recently from 9de8f2d to ed40ddf Compare October 11, 2026 09:38
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch from ed40ddf to 28074b5 Compare October 13, 2026 06:39
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch 2 times, most recently from 91db3ae to 31c7231 Compare October 14, 2026 16:19
@Lee-W
Lee-W requested a review from kaxil October 15, 2026 07:54
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch from 31c7231 to 199a7e7 Compare October 15, 2026 07:54
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch 2 times, most recently from cdc4388 to 22919fe Compare October 22, 2026 03:01
Lee-W added 18 commits October 25, 2026 10:58
The provider is heading to 1.0, after which changing a public hook's
behaviour needs a deprecation cycle, so this fixes the silent failures
and signature inconsistencies now.

- PydanticAIHook.test_connection names the provider and fields when the
  connection's credentials were rejected and env-var auth was used.
- MCPHook validates the stdio "args" and "timeout" extras, and raises
  AirflowOptionalProviderFeatureException when the mcp extra is missing,
  as does the LLMRetryPolicy module on Airflow < 3.3.
- LlamaIndexHook.test_connection reports a model name LlamaIndex's
  OpenAI class rejects (e.g. Ollama models) instead of passing.
- LangChainHook and LlamaIndexHook take llm_model / embed_model as
  keyword-only arguments; their positional order differed.
- The vertexai connection field is labelled deprecated and ignored.
The example's guard caught only ImportError, but LLMRetryPolicy raises AirflowOptionalProviderFeatureException on Airflow < 3.3, so the example Dag failed to import there instead of being skipped.
The docstrings described the old llm_model/embed_model ordering; they now state the behaviour under test.
…nit timeout

fastmcp replaces init_timeout=None with FASTMCP_CLIENT_INIT_TIMEOUT, so a null timeout still timed out on workers that set it. Map None to 0 before building the toolset, and parametrize the accepted-timeout test.
A blank Arguments field is stored as null and is now treated as an empty list. A string starting with [ is parsed as a JSON array, as the field description promises, instead of being passed as one literal argument; malformed JSON raises a ValueError naming the connection.
get_conn() used to log a warning and rebuild the provider from environment variables, authenticating as a different identity than the connection names. It now raises a ValueError naming the connection, the provider and the ignored fields, so test_connection reports the failure too. The _env_auth_fallbacks bookkeeping that only existed to report the old retry is removed.
PydanticAIHook no longer falls back to infer_provider when a provider class rejects the connection's credentials, so the test patched an attribute that is gone. A model that resolves now proves the explicit-credentials path, because rejected credentials raise a ValueError.
The ValueError raised by get_conn() when a provider class rejects the connection's credentials now carries the provider's TypeError text and lists the fields as "fields passed" rather than "ignored", so test_connection says which field broke. The LiteLLMProvider note, the provider fallback guide and the Snowflake Cortex test docstring now describe the ValueError instead of the removed warning and environment-variable fallback.
…reException

The toolsets page said MCPToolset raises ImportError on first connect when the mcp extra is missing, but MCPHook.get_conn() raises AirflowOptionalProviderFeatureException, the same exception SQLToolset and DataFusionToolset raise. The page now describes the three together and keeps when each one raises.
…ex-only fields

GoogleProvider accepts only api_key/base_url, so a Vertex connection with
project/location set is now rejected for a google: model. The hook docstring
and the connection docs still said that spelling works regardless of the
connection's fields, which only held through the removed env-var fallback.
…ons work

_ModelRetryPolicy._hook() built a bare PydanticAIHook, so a pydanticai_azure
connection got the base {api_key, base_url} kwargs and AzureProvider rejected
base_url. Now that a rejected provider class fails closed, every
LLMRetryPolicy / ClassifierRetryPolicy classification on such a connection
raised and fell back to fallback_rules.

- _hook() calls PydanticAIHook.get_hook(llm_conn_id, hook_params={"model_id": ...})
  so the connection type picks the hook subclass, like the operators do
- Add a test that a pydanticai_azure connection resolves to PydanticAIAzureHook
… that credentials were rejected

A provider class can reject a field that is not a credential: the Bedrock hook
forwards aws_read_timeout / aws_connect_timeout whenever they are set and
BedrockMantleProvider accepts neither, so the old message sent users hunting
for a credential problem.

- Reword the fail-closed ValueError to "Could not build provider ... from the
  fields on connection ...", keeping the fields passed and the TypeError text
- Update the hook docstring and provider_fallback.rst to match
- Rename the two tests that still described the failure as rejected credentials
…provider class

The Azure page said an openai: model bypasses Azure entirely, but the hook always
forwards Host as azure_endpoint, which OpenAIProvider does not take, so that
connection now fails get_conn(). The Vertex pages said google: requires an API
key and accepts only an API key, which is stricter than GoogleProvider: it also
takes base_url and reads GOOGLE_API_KEY when no key is set.

- Azure page: a model prefixed with another platform works only if that
  provider class accepts the forwarded fields, otherwise get_conn() raises
- Vertex page and PydanticAIVertexHook docstring: google: accepts the API key
  and Custom Endpoint URL, and rejects the Vertex-only fields
The args list can carry credentials (e.g. a Bearer header passed to mcp-remote)
and the secrets masker does not cover it, so the error text could put a token
in the task log. Report the offending item's type and index instead.
The Vertex hook's docstring and a kwargs comment attributed service_account_info,
project and location to GoogleProvider, which only accepts api_key and base_url.
…the changelog

- Say LlamaIndexHook and LangChainHook now take embed_model and llm_model by keyword only
- Say PydanticAIHook raises ValueError instead of falling back to environment-variable auth
- Say the retry module raises AirflowOptionalProviderFeatureException, so a 1.0.0-style
  except ImportError guard needs to catch both
… Codex auth comment

- Say a connection with a password or host raises ValueError when the provider rejects its fields
- Say only a connection with neither lets the ambient Codex CLI login apply
…Index and LangChain hooks

- Accept the 1.0.0 positional order again (embed_model, llm_model for LlamaIndexHook; llm_model, embed_model for LangChainHook) and emit AirflowProviderDeprecationWarning for each one
- Raise TypeError for a third positional argument or one that is also passed by keyword
- Replace the changelog's keyword-only breaking-change entry with a deprecation note
@Lee-W
Lee-W force-pushed the common-ai-hooks-hardening branch from 22919fe to b285246 Compare October 25, 2026 02:00
@Lee-W
Lee-W merged commit 6e990ea into apache:main Oct 26, 2026
77 checks passed
@Lee-W
Lee-W deleted the common-ai-hooks-hardening branch October 26, 2026 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants