Repository navigation
fix(llm): route OpenRouter structured-output calls only to upstreams that support response_format - #5304
Merged
Conversation
…that support response_format (#3494) OpenRouter load-balances one model across upstreams. An upstream without response_format support can answer extraction/consolidation calls with empty content and finish_reason=stop. Send provider.require_parameters=true on structured-output calls (openrouter provider, or openai pointed at openrouter.ai), keeping any operator-set provider routing keys.
nicoloboschi
force-pushed
the
fix/openrouter-require-parameters-3494
branch
from
October 6, 2026 14:02
1c4f45d to
3fd56f0
Compare
This was referenced Oct 6, 2026
This was referenced Oct 6, 2026
This was referenced Oct 8, 2026
Merged
This was referenced Oct 16, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3494.
OpenRouter spreads one model across several upstreams. Some don't support
response_formatand reply with empty content andfinish_reason=stop. That matches the issue: errors only onretain_extract_factsandconsolidation(the JSON-format scopes), none on reflect. Consolidation sends no output-token cap, so the 64k-cap theory in the issue doesn't explain it.Change: on structured-output calls to OpenRouter (the
openrouterprovider, oropenaiwith anopenrouter.aibase URL), sendprovider.require_parameters=true. Anyproviderkeys set inHINDSIGHT_API_LLM_EXTRA_BODYare kept and take precedence. Free-text and tool calls are unchanged.Note: retain was not silently losing facts. A chunk that fails extraction fails the whole operation, which the worker retries.
Tests:
tests/test_openrouter_require_parameters.py.