Repository navigation
fix(consolidation): prompt rules and safety nets found by the consolidation evals - #5595
Closed
nicoloboschi wants to merge 1 commit into
Closed
nicoloboschi wants to merge 1 commit into
nicoloboschi wants to merge 1 commit into
Conversation
nicoloboschi
added a commit
that referenced
this pull request
Oct 15, 2026
…5607) * feat(config): one decision-model setup with per-operation overrides Recall (#5284), reflect (#5382) and consolidation (#5595) each wire the decision model their own way. Set it up once instead: HINDSIGHT_API_DECISION_MODEL_* globally, with optional HINDSIGHT_API_{RECALL,REFLECT,CONSOLIDATION}_DECISION_MODEL_* overrides that fall back to the global value (PROVIDER=none opts an operation out). config.decision_model_config(op) resolves them; engine/decision_model.py is the shared /v1/systemone client. The typesafe reranker keeps its own settings. * fix(config): review fixes for the decision-model setup - Move the decision-model constants out of the middle of the reranker blocks. - Docs and .env.example: the setup is shared by operations that support a decision model, not used by every operation. - Type the per-setting lookup (str | None) instead of Any.
nicoloboschi
force-pushed
the
consolidation-evals
branch
2 times, most recently
from
October 15, 2026 13:09
5017fc5 to
cbc7720
Compare
nicoloboschi
force-pushed
the
consolidation-evals
branch
from
October 15, 2026 14:34
cbc7720 to
4a9852a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Prompt rules and server safety nets for consolidation, each one found by a failing scenario in #5626.
mentioned_at;current_text);merged_into), or re-queued;It also puts
retired-conventionsback in the eval's must-pass set.Results (Gemini 3.1 Flash Lite, 17 scenarios × 3)
Tests
test_consolidation_lookalike.py, plus updated delete tests.test_22_consolidation_safety_nets.py.