Repository navigation
fix(llm): retry a 200 that ends with finish_reason=error instead of reading it as no tool call (#5612) - #5751
Merged
Merged
Conversation
…eading it as no tool call (#5612) OpenRouter reports an upstream that died mid-generation as HTTP 200 with finish_reason="error". The tool path read it as an empty tool-call result, so reflect skipped forced retrieval steps instead of retrying the request. - openai-compatible: a finish_reason=error choice raises a retryable ProviderResponseError, checked where every path reads its first choice; the tool path now goes through that check and retries it with backoff. - litellm: it rewrites finish_reason "error" to "stop"; the original is read from provider_specific_fields, and both call paths retry it. - tracing: spans carry the served upstream (hindsight.provider.served_by), and each retried attempt is recorded as a failed span + metric with its usage. - tests: fake responses on the tool path set error=None (the path never checked before); the empty-choices tool test expects the clear error.
This was referenced Oct 20, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5612.
Problem
OpenRouter reports an upstream that died mid-generation as HTTP 200 with
finish_reason: "error", no tool calls, and the cause on the choice. The tool-call path readchoices[0]directly, so this became an empty tool-call result: reflect treated a transport failure as "the model chose no tool", spent its single loop-wide retry, then answered without the forcedrecall/search_observationsstep. Nothing recorded which upstream failed.Fix
finish_reason="error"choice raises a retryableProviderResponseError(with the choice's error and the served upstream), checked in the helper every path reads its first choice through.call_with_toolsnow uses that helper and retriesProviderResponseErrorwith backoff inside its existing attempt budget, likecall()already did.call()also stops accepting partial text from an errored generation."error"to"stop"but keepsnative_finish_reasonanderrorin the choice'sprovider_specific_fields; the same check reads them there, and both call paths retry it.hindsight.provider.served_by(OpenRouter's responseprovider), and every retried attempt is recorded as a failed span +success=Falsemetric with its billed usage — previously it left only a log line.Behaviour changes
choiceswith a clearProviderResponseError(empty choices used to be anIndexError).Tests
tests/test_openai_compatible_finish_reason_error.py: reporter-shaped responses — tool call retried then answered; raises naming the upstream when retries run out;call()rejects partial text; retried attempt traced with upstream + usage; OTel span attribute; LiteLLM (callandcall_with_tools) through LiteLLM's real response conversion.error=None(the path never checked it before).Not in scope (from the issue's ask 3): surfacing "a forced step was skipped" in reflect / refresh result metadata, and a per-step forced retry.