Repository navigation
fix(reflect): work on models that reject a forced tool_choice; no 500 on an empty observation - #5917
Merged
Merged
Conversation
… on an empty observation - anthropic: Claude Opus 5.5 / Sonnet 5.5 answer a named or "any" tool_choice with a 400, so every reflect and mental-model refresh failed on them since #4470. The tool loop now resends that request with tool_choice=auto and remembers it for the provider, detected from the error rather than a model list (#5738). - reflect: an observation whose text is "" lost its text key in _prune_nulls and MemoryFact(**obs_data) failed the whole reflect with a 500 after the answer was written. based_on now fills the dropped required fields (#5840).
…pt on models that reject a forced tool (#5738) Review fixes: the strict-schema one-shot path forced a tool too, so retain and consolidation failed on the same models with HINDSIGHT_API_LLM_STRICT_SCHEMA on; it now uses the existing schema-in-prompt path once the model rejects it. Test added; comments updated.
This was referenced Oct 21, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5738
Fixes #5840
#5738: reflect fails on Claude Opus 5.5 / Sonnet 5.5
Since #4470 the
anthropicprovider maps reflect's forced steps totool_choice{"type":"tool"}/{"type":"any"}. These models answer both with400 tool_choice: type "tool" and "any" are not supported for this model. Reflect forces tools on its first steps and on the closingdonecall, so every reflect and every mental-model refresh failed.Now, when the tool loop gets that 400 for a forced choice, it resends the same request with
tool_choice: auto. It also remembers this on the provider, so later calls ask withautostraight away. It's detected from the error, not from a list of model names, so a future model that rejects forced choice works too. Reflect already copes with a model that answers in prose where a tool was forced. That's the same path the lmstudio/ollama providers use when they droprequired. Any other 400 is raised as before.Strict structured output (
HINDSIGHT_API_LLM_STRICT_SCHEMAon) also forces a tool, so retain and consolidation failed on these models too. Once the model rejects a forced tool, those calls use the existing schema-in-prompt path instead.Tests:
test_anthropic_forced_tool_choice_fallback.py. Named andrequiredare each resent asauto, and the next call asks withautoup front. Both fail without the fix. Strict structured output falls back to schema-in-prompt and stays there. An unrelated 400 is still raised.#5840: reflect 500s when it cites an observation with empty text
tool_search_observationspasses each observation through_prune_nulls, which drops"". So an observation with empty text reachesbased_onwith notextkey, andMemoryFact(**obs_data)failed validation after the answer was already generated.based_onnow fills in the required fields the pruning can drop (text="",fact_type="observation").Test:
test_reflect_empty_observation_text.py. Without the fix it fails withtext Field required, the error from the issue.