Skip to content

fix(reflect): omit temperature in structured extraction when disabled - #5163

Merged
nicoloboschi merged 3 commits into
vectorize-io:mainfrom
SebastianThiebaud:sebastian/fix-structured-refresh-temperature
Oct 6, 2026
Merged

nicoloboschi merged 3 commits into
vectorize-io:mainfrom
SebastianThiebaud:sebastian/fix-structured-refresh-temperature

Conversation

@SebastianThiebaud

Copy link
Copy Markdown
Contributor

Mental-model refreshes with a response_schema still sent temperature=0.0 during their structured extraction pass, even when HINDSIGHT_API_LLM_TEMPERATURE=none disabled that parameter for a model. Bedrock GPT-6 Luna rejects the field, so the refresh failed with structured_output_failed and preserved the previous page content.

Structured extraction now passes no temperature when the configured reflect temperature is omitted. It continues to use 0.0 for models that accept the parameter, keeping the existing deterministic extraction behavior. The regression test captures the extraction call with temperature disabled; it failed with 0.0 before this change and passes with None afterward.

Validation:

  • uv run pytest -o addopts='' -o log_cli=false tests/test_reflect_agent.py tests/test_reflect_split_synthesis.py tests/test_llm_temperature_env.py -q --tb=short -k 'not RealLLM': 114 passed, 2 skipped (local ML stack), 3 provider-backed tests deselected because this checkout has no OpenAI API key.
  • uv run pytest -o addopts='' tests/test_reflect_agent.py::TestReflectStructuredOutput tests/test_llm_temperature_env.py tests/test_llm_temperature_pipeline.py -q: 20 passed, 3 skipped (local ML stack).
  • uv run ruff check hindsight_api/engine/reflect/agent.py tests/test_reflect_agent.py and uv run ruff format --check hindsight_api/engine/reflect/agent.py tests/test_reflect_agent.py: passed.

Live Bedrock verification requires a released image containing this fix and a subsequent mental-model refresh. No deployment is included here.

@github-actions github-actions Bot added the core label Oct 2, 2026
SebastianThiebaud and others added 2 commits October 6, 2026 14:37
…uctured extraction

- structured extraction now calls the existing _deterministic_temperature()
  instead of duplicating its "0.0 unless the reflect temperature is omitted"
  expression 200 lines from the helper that already encodes it
- renamed _map_temperature/_MAP_TEMPERATURE to _deterministic_temperature/
  _DETERMINISTIC_TEMPERATURE now that both reflect mechanical passes share it,
  and folded the Bedrock structured_output_failed case into its rationale
- rebased onto origin/main (branch was 72 commits stale)
@nicoloboschi
nicoloboschi force-pushed the sebastian/fix-structured-refresh-temperature branch from c3272f7 to 9ff892f Compare October 6, 2026 12:40
test_reflect_structured_output_threads_reflect_flag stubbed get_config with
only llm_strict_schema_reflect, so the new llm_temperature_reflect read raised
AttributeError inside _generate_structured_output's except block and the LLM
call never happened (CI test-api shard 2/3).
@nicoloboschi
nicoloboschi merged commit 8cca87e into vectorize-io:main Oct 6, 2026
219 of 220 checks passed
@nicoloboschi nicoloboschi mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants