Skip to content

Remove redacted_because from internal unsigned.#19581

Open
erikjohnston wants to merge 10 commits intodevelopfrom
erikj/simplify_redaction
Open

Remove redacted_because from internal unsigned.#19581
erikjohnston wants to merge 10 commits intodevelopfrom
erikj/simplify_redaction

Conversation

@erikjohnston
Copy link
Member

@erikjohnston erikjohnston commented Mar 17, 2026

This is a simplification so that unsigned only includes "simple" values, to make it easier to port to Rust.

Reviewable commit-by-commit

Summary:

  1. Add recheck column to redactions table

    A new boolean recheck column (default true) is added to the redactions table. This captures whether a redaction needs its sender domain checked at read time — required for room v3+ where redactions are accepted speculatively and later validated. When persisting a new redaction, recheck is set directly from event.internal_metadata.need_to_check_redaction().

    It's fine if initially we recheck all redactions, as it only results in a little more CPU overhead (as we always pull out the redaction event regardless).

  2. Backfill recheck via background update

    A background update (redactions_recheck) backfills the new column for existing rows by reading recheck_redaction from each event's internal_metadata JSON. This avoids loading full event objects by reading event_json directly via a SQL JOIN.

  3. Don't fetch confirmed redaction events from the DB

    Previously, when loading events, Synapse recursively fetched all redaction events regardless of whether they needed domain rechecking. Now _fetch_event_rows reads the recheck column and splits redactions into two lists:

    • unconfirmed_redactions — need fetching and domain validation
    • confirmed_redactions — already validated, applied directly without fetching the event

    This avoids unnecessary DB reads for the common case of already-confirmed redactions.

  4. Move redacted_because population to EventClientSerializer

    Previously, redacted_because (the full redaction event object) was stored in event.unsigned at DB fetch time, coupling storage-layer code to client serialization concerns. This is removed from _maybe_redact_event_row and moved into EventClientSerializer.serialize_event, which fetches the redaction event on demand. The storage layer now only sets unsigned["redacted_by"] (the redaction event ID).

  5. Always use EventClientSerializer

    The standalone serialize_event function was made private (_serialize_event). All external callers — rest/client/room.py, rest/admin/events.py, appservice/api.py, and tests — were updated to use EventClientSerializer.serialize_event / serialize_events, ensuring
    redacted_because is always populated correctly via the serializer.

  6. Batch-fetch redaction events in serialize_events

    serialize_events now collects all redacted_by IDs from the event batch upfront and fetches them in a single get_events call, passing the result as a redaction_map to each serialize_event call. This reduces N individual DB round-trips to one when serializing a batch of events that includes redacted events.

@erikjohnston erikjohnston force-pushed the erikj/simplify_redaction branch from c2d73b6 to 7d90b4c Compare March 17, 2026 12:29
as_client_event=True,
# If this is an invite or a knock membership event, and we're interested
# in this user, then include any stripped state alongside the event.
include_stripped_room_state=True,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check for membership is superfluous as we only include stripped state for invites and knocks anyway, so the change here is that we include stripped state even if the appservice is not interested in the user. Which I think is fine.

The reason to batch things up is so that we more efficiently fetch redactions from the DB.

@erikjohnston erikjohnston force-pushed the erikj/simplify_redaction branch from 7d90b4c to b99fb4e Compare March 17, 2026 12:38
@erikjohnston erikjohnston marked this pull request as ready for review March 17, 2026 14:56
@erikjohnston erikjohnston requested a review from a team as a code owner March 17, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants