Skip to content

fix(resources): bind collection filter values as SQL parameters (#1304) - #1310

Open
Anai-Guo wants to merge 1 commit into
llmware-ai:mainfrom
Anai-Guo:fix/sql-injection-collection-filters
Open

fix(resources): bind collection filter values as SQL parameters (#1304)#1310
Anai-Guo wants to merge 1 commit into
llmware-ai:mainfrom
Anai-Guo:fix/sql-injection-collection-filters

Conversation

@Anai-Guo

Copy link
Copy Markdown

Fixes #1304.

Problem

llmware/resources.py builds the WHERE clause of the SQLite and Postgres collection backends by interpolating the filter value directly into the SQL string:

# SQLiteRetrieval.filter_by_key_dict
conditions_clause += f" {key} = '{value}' AND "
# PGRetrieval.filter_by_key_dict
conditions_clause += f" {key} = '{value}' AND "

The key is validated against the library's allowed-key list (retrieval.py) and the table name goes through safe_name(), but the value is never escaped or parameterized. A value containing a quote closes the literal and rewrites the predicate.

These sinks are reachable from the public API — Library.block_lookup() / Query.document_lookup() to filter_by_key_dict, and Query.text_query_with_custom_filter() / text_query_by_author_or_speaker() to text_search_with_key_value_dict_filter — where the values come from end-user query input or document metadata.

Reproduced against the SQLite filter_by_key_dict logic, with two rows belonging to different "tenants" and the filter {"author_or_speaker": "alice' OR '1'='1"}:

benign  old: [(1, 1, 1, 'alice', 'tenant-A public note')]
inject  old: [(1, 1, 1, 'alice', 'tenant-A public note'),
              (2, 2, 1, 'bob',   'tenant-B SECRET salary data')]   <-- filter neutralized

Fix

Bind the values as query parameters in the four value sinks named in the issue:

  • PGRetrieval.text_search_with_key_value_dict_filter
  • PGRetrieval.filter_by_key_dict
  • SQLiteRetrieval.text_search_with_key_value_dict_filter
  • SQLiteRetrieval.filter_by_key_dict

Postgres uses %s placeholders and SQLite uses ?, matching the parameter style already used elsewhere in this module (e.g. PGRetrieval.embedding_job_cursor, and SQLiteRetrieval.text_search_with_key_value_dict_filter which already carried an unused insert_array = ()). On Postgres the parameter tuple is only passed when it is non-empty, so a filter-less query still goes through the same single-argument execute() path as before and no % in the tsquery string gets reinterpreted.

The mongo-style {"$in": [...]} ranges are bound too. That also fixes a latent bug in those branches: list members were emitted through str(entry) with no quoting, so a list of strings produced IN (alice,bob) and raised sqlite3.OperationalError: no such column: alice. With placeholders it returns the expected rows.

After the change, same repro:

benign  new: [(1, 1, 1, 'alice', 'tenant-A public note')]
inject  new: []
$in str new: [(1, ... 'alice', ...), (2, ... 'bob', ...)]

Scope is limited to the value sinks reported in #1304 — key/table-name handling and the _prep_query full-text search string are untouched.

🤖 Generated with Claude Code

…are-ai#1304)

The SQLite and Postgres collection-database backends built WHERE clauses by
interpolating filter values straight into the SQL string, so a value carrying
SQL metacharacters escaped its quotes and rewrote the predicate. The key names
are validated against the library's allowed-key list and the table name goes
through safe_name(), but the values were never escaped.

Bind the values as query parameters instead, in the four value sinks reported
in llmware-ai#1304:

  * PGRetrieval.text_search_with_key_value_dict_filter
  * PGRetrieval.filter_by_key_dict
  * SQLiteRetrieval.text_search_with_key_value_dict_filter
  * SQLiteRetrieval.filter_by_key_dict

Postgres uses %s placeholders and SQLite uses ?, matching the parameter style
already used elsewhere in this module. The mongo-style {"$in": [...]} ranges
are bound too, which also fixes string members of those lists being emitted
unquoted (previously "IN (alice,bob)" -> "no such column: alice").
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SQL injection via unescaped metadata/filter value in the collection-database layer (cross-document disclosure)

1 participant