Skip to content

feat: allow for multiple metadata filters on list fields#179

Merged
emilradix merged 9 commits intomainfrom
mm-multiple_metadata_filters
Mar 2, 2026
Merged

feat: allow for multiple metadata filters on list fields#179
emilradix merged 9 commits intomainfrom
mm-multiple_metadata_filters

Conversation

@MattiaMolon
Copy link
Copy Markdown
Contributor

Summary

This PR refactors metadata filtering so search and delete consistently support multi-value filters with clear semantics, and updates self-query so it can select multiple filters per field.

What Changed

Metadata filtering implementation

  • Added a new shared helper: src/raglite/_metadata_filter.py
    • build_metadata_filter_condition(...) for SQLAlchemy query conditions.
    • build_metadata_filter_sql(...) for raw SQL query fragments + params.
  • Updated search paths to use shared metadata filter logic:
    • src/raglite/_search.py
      • vector_search() now uses shared condition builder.
      • keyword_search() now uses shared SQL builder for PostgreSQL and DuckDB.
  • Updated delete path to use shared metadata filter logic:
    • src/raglite/_delete.py
      • _get_documents_with_metadata() now builds conditions via shared helper.

Self-query improvements

  • Updated src/raglite/_search.py self-query prompt and mapping logic so a single metadata field can return multiple matched values.
  • Self-query output now keeps multiple valid values per field (with deduplication and invalid ID filtering).

Tests

  • tests/test_search.py
    • Added tests for OR semantics with multi-value filters.
    • Added tests for matching list-valued document metadata.
    • Added _self_query test to verify multiple values per field are preserved.
  • tests/test_delete.py
    • Added test for delete-by-metadata with multi-value OR semantics.

Docs

  • README.md
    • Documented list metadata storage behavior.
    • Added multi-value metadata filter example.

@emilradix emilradix changed the title Allow for multiple metadata filters on list fields feat: allow for multiple metadata filters on list fields Feb 27, 2026
@MattiaMolon
Copy link
Copy Markdown
Contributor Author

Can be merged as far as I can tell @emilradix

@emilradix emilradix merged commit d327f14 into main Mar 2, 2026
4 checks passed
@emilradix emilradix deleted the mm-multiple_metadata_filters branch March 2, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants