Repository navigation
fix(migrations): clear placeholder dates without scanning memory_units; consolidation reads them as unknown - #5922
Merged
Conversation
…olidation reads placeholder dates as unknown The migration added in #5828 cleared year-1/9999 dates with an UPDATE filtered on four date columns that no index leads with, so it read every memory_units row in every schema at startup to fix a handful of rows. It is now a no-op (the revision stays so databases that already ran it keep a valid history). Instead, consolidation's date merging (_as_dt, _min_date, _max_date) normalises each date with finite_utc, as recall already does, so a stored infinity reads as an unknown bound rather than raising 'can't compare offset-naive and offset-aware datetimes'.
Review finding: on PostgreSQL 16 and older, retain's temporal-link SQL subtracts event_date, and an old infinity there fails with 'cannot subtract infinite timestamps' - the read-side fix can't reach SQL. event_date has a standalone index (idx_memory_units_event_date), so the migration now clears just that column as two index range probes instead of scanning the table. PG only: Oracle has no infinite timestamps. - test: the migration nulls placeholder event_date and leaves the other columns - test: say why the consolidation test forges and reads rows with raw SQL
nicoloboschi
changed the title
This was referenced Oct 21, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #5828 (not released yet).
Problem. Migration
c7e2f9a4d1b8cleared year-1/9999 dates with oneUPDATE memory_units ... WHEREover four date columns.occurred_start,occurred_endandmentioned_atare only indexed behindbank_id, so it read the whole table in every tenant schema at startup — to fix a handful of rows.Fix.
event_date, which has its own index (idx_memory_units_event_date): two index range probes, no table scan. It has to stay for that column: retain's temporal-link SQL subtractsevent_date, and PostgreSQL 16 and older fail withcannot subtract infinite timestamps(checked onpostgres:16). PG only — Oracle has no infinite timestamps._as_dt,_min_date,_max_date) now runs each date throughfinite_utctoo. Before, a storedinfinitythere raisedcan't compare offset-naive and offset-aware datetimes.Tests (
test_temporal_infinite_dates.py)-infinity/infinityrows: fails with theTypeErrorwithout the fix, passes with it.event_datevalues (infinite and year-9999), keeps real ones, and leaves the other columns alone.No system story: the API refuses placeholder dates since #5828, so a client-driven test can't create the old rows.