Skip to content

config: add consolidation_dedup_tag_match for cross-scope dedup - #2190

Closed
kevin-ho wants to merge 1 commit into
vectorize-io:mainfrom
kevin-ho:config/consolidation-dedup-tag-match
Closed

kevin-ho wants to merge 1 commit into
vectorize-io:mainfrom
kevin-ho:config/consolidation-dedup-tag-match

Conversation

@kevin-ho

@kevin-ho kevin-ho commented Jun 14, 2026 •

Copy link
Copy Markdown

Closes #2188

Adds HINDSIGHT_API_CONSOLIDATION_DEDUP_TAG_MATCH env var to make the dedup probe's tag matching mode independently configurable.

What changed:

  • config.py: new env var constant, default all_strict, validation function, config field
  • consolidator.py line 173: dedup probe uses config value instead of hardcoded all_strict
  • Consolidation recall (line 1981) stays hardcoded all_strict — security boundary intact

Validation:

  • Invalid values log a warning and fall back to all_strict
  • Valid: any, all, any_strict, all_strict, exact

Default behavior unchanged. Single-user deployments set HINDSIGHT_API_CONSOLIDATION_DEDUP_TAG_MATCH=any to enable cross-scope dedup.

…ngle-user deployments

The dedup probe hardcoded all_strict tag matching. This prevented
single-user deployments from finding near-duplicates across different
session tags, causing runaway observation duplication when agents use
per-session tags for provenance.

- Add HINDSIGHT_API_CONSOLIDATION_DEDUP_TAG_MATCH env var
- Default: all_strict (backwards compatible)
- Validate against {any, all, any_strict, all_strict, exact}
- Consolidation recall (the security boundary) stays all_strict
@kevin-ho

Copy link
Copy Markdown
Author

Closing in favor of #2189, which covers the same fix with tests and docs. Thanks @koriyoshi2041 for the quick turnaround.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dedup probe tag matching hardcoded to all_strict — causes runaway observation duplication when agents use per-session tags

1 participant