Skip to content

Comments

fix(langchain): isolate middleware model calls from parent streaming …#35339

Draft
NITIN Madan (Nitin75408) wants to merge 1 commit intolangchain-ai:masterfrom
Nitin75408:fixes/34382
Draft

fix(langchain): isolate middleware model calls from parent streaming …#35339
NITIN Madan (Nitin75408) wants to merge 1 commit intolangchain-ai:masterfrom
Nitin75408:fixes/34382

Conversation

@Nitin75408
Copy link
Contributor

Summary

  • Add _get_internal_model_config() static method to AgentMiddleware
    base class that returns a RunnableConfig with callbacks=[] to
    prevent streaming callback inheritance from the parent agent context
  • Update SummarizationMiddleware to use isolated config in both
    _create_summary() and _acreate_summary()
  • Update LLMToolSelectorMiddleware to use isolated config in both
    wrap_model_call() and awrap_model_call() (previously passed no
    config at all)

Fixes #34382

Why

When astream() runs, var_child_runnable_config is set with streaming
callbacks. ensure_config() additively merges this into any config that
doesn't explicitly override callbacks. This caused internal middleware
model calls to emit tokens into the parent stream — breaking stream
correctness and isolation.

The fix passes callbacks=[] which ensure_config() merges on top of
the inherited config, clearing the streaming callbacks. This is backward
compatible: non-streaming usage already has no callbacks to clear.

Review notes

  • SummarizationMiddleware previously passed config={"metadata": {...}}
    which preserved inherited callbacks. Now uses the isolated config.
  • LLMToolSelectorMiddleware previously passed no config at all,
    fully inheriting parent callbacks. Now uses the isolated config.
  • The _get_internal_model_config() method is intentionally private
    (_-prefixed) since it's an implementation detail, but available to
    custom middleware subclasses.

@github-actions github-actions bot added fix For PRs that implement a fix langchain `langchain` package issues & PRs external and removed fix For PRs that implement a fix labels Feb 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external langchain `langchain` package issues & PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model calls inside middleware are leaking when streaming

1 participant