Skip to content

perf(mobile): derive Active filter activity from the channel list, with no per-channel relay reads - #9832

Draft
brow wants to merge 8 commits into
mainfrom
perf/mobile-activity-filter-no-lookups
Draft

brow wants to merge 8 commits into
mainfrom
perf/mobile-activity-filter-no-lookups

Conversation

@brow

@brow brow commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

The Active section filter needs to know when each channel last had a message. Since #8479 (saved channel section filters) and #9796 (section filter refinements), the app has fetched that with its own relay history read per channel in every section that uses the filter. Every app resume, pull to refresh and live message in such a channel started those reads again. With many channels, the burst competes with the channel list's own reads and the read-state sync for the relay's request budget. When reads time out or get rate limited, the app shows "Some channel activity is unavailable", and the same pressure can delay other requests such as read-state sync.

The channel list already reads each channel's latest messages in one batched request when it loads. This change records the newest message time from that read (self-authored and muted messages included, unlike lastMessageAt) and keeps it current from live delivery and from the unread catch-up the list already sends after a reconnect. Where that catch-up starts after posts no read has covered yet (for example, a post that arrived while offline and was read on another device), a separate small request sent after the catch-up asks for the newest one with a single limit: 1 filter per such channel. It is sent only for channels an Active section is showing, only when the gap could hold a newer post than the one already known, and without the per-channel websocket fallback, so a failure there leaves only that channel's activity unknown and never delays unread state. A successful refresh replaces a channel's time, so a deleted latest post stops counting, while keeping any newer message that arrived during the read. Only messages whose signature verifies count, and the app verifies only a message that would change a channel's activity. When a channel's latest post is deleted, live or while offline, the app forgets that channel's activity, so it reads as unknown (and stays visible) until the next background list read re-reads it on resume, reconnect or a list change. A channel the last list read missed is asked for again the same way. The Active filter reads that value, so it sends no relay requests of its own. Because the filter can no longer fail on a lookup, the "activity couldn't load" notice and its retry are removed.

Validation

  • Network impact. Before: one history read per channel in each Active-filtered section on every resume and refresh, plus one more each time a live message arrived in such a channel. After: zero. The filter reuses the list's existing batched read.
  • Provider tests check that activity comes from the list read, live delivery and a reconnect's catch-up; ignores a message with a bad signature; includes self-authored messages; counts a future-dated post as now; leaves lastMessageAt untouched and sends no extra requests; stays unknown (the channel stays visible) when a read or catch-up fails; survives a same-community rebuild; picks up a post read on another device while offline, including after an offline rebuild resets the unread cursor; drops a deleted latest post on refresh, and forgets it until the next list read on a live deletion or a deletion the catch-up returns; skips the gap read for a channel no Active section shows; asks again for a channel whose list read had no verifiable post; and keeps a live post that arrives during a refresh.
  • Simulator run on a signed-in community: set a section to Active (7 days), then backgrounded and resumed the app and pulled to refresh. Inactive channels stayed collapsed under Show more, and no notice appeared.

Regressions

  • Signature checks run on the UI isolate, as read-state sync's do, but only for a message that would raise a channel's activity: at most one per channel per list read, plus live messages that are newer than the known activity.
  • Channels the batch doesn't cover (for example non-member or archived channels, or a batch that hits its deadline) have unknown activity. The filter fails open for them, so they stay visible, which matches how it treated a failed lookup before.
  • A catch-up that finds only newer self-authored, muted or already-read messages now republishes the list once, so the filter sees them. Before, it published nothing.
  • After a reconnect, channels an Active section shows, whose catch-up would skip posts no read has covered, get one limit: 1 filter each in one small extra request, sent after the unread catch-up. It adds at most one event per such channel, against the one request per Active channel per resume before. A rate-limit refusal of either request still pauses the session's reads, since the relay's quota covers every request this client makes.
  • A channel no Active section shows, whose catch-up would skip uncovered posts, reads as unknown until a later catch-up runs while it is shown.
  • A channel whose catch-up or gap read fails (including a catch-up that fails before sending) reads as unknown, so it shows unfiltered until a later read succeeds. A failed gap read is not retried by itself; the next list read's catch-up asks again, so it adds no retry traffic.
  • A post dated ahead of the device clock (the relay accepts up to 15 minutes) counts as activity now.

…th no per-channel relay reads

The Active section filter read each channel's latest message with its own relay history request, restarted on every resume, refresh and live message. Record the newest message time from the list's existing batched latest-message read and from live delivery instead, and drop the lookup failure notice and retry, which can no longer happen.

Signed-off-by: Tom Brow <tomb@block.xyz>
@brow

brow commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-20T22:16:46.629401Z ee7fa3c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 22cfffc8d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

brow added 3 commits November 20, 2026 13:13
…ebuilds, deletions and clock skew

A reconnect catch-up that starts after posts no read has covered asks for the newest one with a limit-1 filter in the same batch, bounded by when a read last covered the channel. A failed catch-up leaves activity unknown until one succeeds. A same-community rebuild keeps activity. A successful latest read replaces the stored time, keeping arrivals newer than the read. A future-dated post counts as now.

Signed-off-by: Tom Brow <tomb@block.xyz>
A failed or slow gap read now leaves only activity unknown instead of
discarding the unread catch-up, and a catch-up that fails during setup
marks its channels unknown. A covering latest read also clears a failed
catch-up mark.

Signed-off-by: Tom Brow <tomb@block.xyz>
A gap read that kept failing rescheduled the unread retry at the base
delay after each successful catch-up reset the backoff, so it restarted
a full list read and catch-up every ten seconds. Leave the channel's
activity unknown instead; the next list read's catch-up asks again.

Signed-off-by: Tom Brow <tomb@block.xyz>
@brow
brow marked this pull request as ready for review November 20, 2026 22:14
@brow
brow requested a review from a team as a code owner November 20, 2026 22:14
@brow
brow deployed to codex-review November 20, 2026 22:15 — with GitHub Actions Active
@github-actions github-actions Bot added the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 20, 2026
@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

🔐 Codex Security Review

Status: review required for the current range.

The current range is 8a3912f1511a46faf30b4cef774c634529e9f2f1...4ce688b34c47be03a9874c24e5e19128dfd3db85.
A new review must complete for this exact range. When manual authorization
is required, a user with write access must comment exactly
@buzz-security-review 4ce688b34c47be03a9874c24e5e19128dfd3db85 to authorize a new review.
Any previous review applies only to its recorded range.

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Star Lord — Changes needed on ee7fa3c. The shared list removes the filter’s direct requests, but the new activity paths can retain stale or unverified results and add reconnect work when Active is off. Source review only.

for (final entry in visibleMetas)
if (memberChannelIds.contains(entry.value.id) &&
!entry.value.isArchived &&
!listed.contains(entry.value.id))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 — Retry channels that missed their initial activity read. A transient non-deadline failure in one WebSocket fallback filter leaves that channel out of _known. The list still installs it, so ordinary reconnects skip its latest-message read, and a successful unread catch-up can update the timestamp without making it known. An inactive channel then stays visible until a live post or a full refresh. Include channels missing a successful latest baseline in a bounded batch retry.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 3cbc593: background list reads now ask again for any member channel without an activity baseline, including one whose only post failed verification.

);
if (recordedAtByChannel.isNotEmpty ||
appliedDeletion ||
activityAdvanced ||

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 — Reconcile activity after the latest post is deleted. If a channel’s sole recent post is deleted while the phone is offline, reconnect skips that listed channel’s latest-message read. This deletion updates unread state, while both catch-up paths can only raise activity. The channel stays in Active until expiration or a full refresh even when catch-up receives the deletion. Track the activity head’s event ID and reconcile affected channels in a bounded latest-message batch.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 3cbc593: when a deletion (live, or returned by the reconnect catch-up, with or without an h tag) targets a channel's known latest post, the app re-reads that channel's newest surviving post with one limit: 1 filter. A test step covers the offline case in d4cba5f.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Addressed differently in 7d73f30. The separate re-read we added for this raced newer refreshes and cost extra requests, so we dropped it. Now a deletion of a channel's latest post, live or returned by the catch-up, forgets that channel's activity: Active keeps showing it until the next list read re-reads it on resume, reconnect or a list change.

final newest = <String, int>{};
for (final event in events) {
if (event.channelId case final channelId?) {
final at = _clamp(event.createdAt);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 — Preserve verified timestamps in the shared activity path. The documented Active contract uses verified message timestamps, and the old provider rejected invalid signatures. The HTTP and WebSocket decoders only parse event fields, so invalid activity evidence now changes which rows the filter shows. Verify the shared activity evidence off the UI isolate and restore coverage for invalid events.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 3cbc593: only messages whose signature verifies count. A list read takes each channel's newest authentic post, and live and catch-up messages are verified only when they would raise the activity, so the cost stays at about one check per channel.

limit: _unreadCatchUpLimit,
),
];
final gapFilters = {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 P2 — Bound optional activity reads on reconnect. A recent read marker can create a gap even when every section uses All. Each qualifying channel adds a database query, and a non-deadline bridge failure falls back to one WebSocket REQ per channel. A quota refusal from this optional HTTP query also pauses foreground history and publishing through the shared request gate. Skip gaps that cannot improve activity, scope the remaining reads to channels that use Active, and give this optional batch a failure path without per-channel fallback.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Fixed in 3cbc593: the gap read now goes only to channels an Active section is showing, only when the gap could hold a newer post than the one already known, and without the per-channel websocket fallback. Other channels keep the gap for a later catch-up. I kept the rate-limit gate: the relay's quota covers every request this client sends, so a refusal should still pause the session's reads.

@jedwards27 jedwards27 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 :bot: Jude’s code review agent — changes recommended

Reviewed: 81b4054e5d6360996cafca94279ef2acc3ac14c4..ee7fa3c7904678328671228ba86ccc6da4faea1e (exact head). Risk: high for mobile channel activity/reconnect correctness. The change reuses the batched latest-message read and live/unread catch-up instead of making per-section Active-filter requests; it does not change the relay wire contract.

Blocking findings / author action:

  1. [P2] Reconcile a deleted activity head on reconnect. mobile/lib/features/channels/channels_provider.dart:517-522,993-1001 excludes already-listed channels from the reconnect latest-message read. mobile/lib/features/channels/channel_unread_catch_up.dart:124-141,174-180,312-325 receives deletions but only ever raises activity via channel_latest_activity.dart:120-132; the only lowering operation is noteRead on a latest-message result. If the newest 1h post is deleted while offline and the next surviving post is 8d old, reconnect leaves that channel falsely visible in Active(7d) until a full refresh. The previous filter provider read the latest post again on reconnect. Reconcile deletion-affected channels with a bounded, authoritative latest read (including h-less deletions) and add a production-path delete-offline → reconnect regression proving the Active result lowers without manual refresh.
  2. [P2] Make failed activity reads visible and recoverable. mobile/lib/features/channels/channel_filter/channel_filter_activity.dart:8-14 maps both pending and failed/unknown evidence to AsyncLoading, while mobile/lib/features/channels/channels_page/body.dart:369-383 shows unknown channels as if they match Active. The deleted channels_page/activity_recovery.dart previously explained failures and offered Retry. A transient failed latest-message fallback can leave a listed channel without a known baseline, which the ordinary reconnect skips (channels_provider.dart:517-522); successful unread catch-up alone does not mark it known. Users can therefore see all channels under Active indefinitely with no indication why. Preserve fail-open visibility, distinguish actual failure from initial loading, and provide a bounded retry/recovery path (including the missed baseline) with a failure → explanation/retry → success widget/production-seam regression.

Findings elsewhere: the exact-head live/catch-up ownership fence and self-authored/muted activity path looked sound in our systems pass; the source-backed defects above override that initial assessment. No separate accessibility defect was established. A prior agent review also notes verification and optional gap-query load concerns; this review does not claim they were performance-measured.

Validation: clean ee7fa3c7904678328671228ba86ccc6da4faea1e: just mobile-install PASS; just mobile-check PASS (formatter 0 changed, analyzer no issues); just mobile-test PASS in the independent full-package lane (5,710 reported, 71 skipped; cross-check package 27 passed); just file-size-check and git diff --check PASS. My duplicate just mobile-test run was intentionally terminated to avoid contending with the full run; not counted as a pass. CI at this head reports Mobile checks/tests/Android SUCCESS. Green suites do not cover the delete-offline → reconnect and failure-presentation transitions above.

Manual/native evidence: author reports a signed-in simulator background/resume and refresh smoke in the PR body; our team did not independently run a signed-in native reconnect or accessibility journey against this exact head. Verification owner: author for the two code/test fixes; reviewer/tooling for native journey after the fix, CI/release gate for candidate artifact. Residual risk: unverified native behavior and load under large channel counts; neither is presented as an additional author-actionable defect on this head.

This is a COMMENT review with a blocking code recommendation, not a GitHub Request Changes submission: repository policy reserves that action for humans. A new head requires a new review.

- Count only messages whose signature verifies. A list read picks each
  channel's newest authentic post, and a channel whose posts all fail
  stays unknown.
- Retry a channel the last list read left without a baseline: background
  list reads ask for its latest message again.
- Re-read a channel's newest post when its latest one is deleted, so the
  deleted post stops counting.
- Read a reconnect's activity gap only for channels an Active section
  shows, only when it could raise the known activity, and without the
  per-channel websocket fallback. Other channels keep the gap for a later
  catch-up and read as unknown meanwhile.

Signed-off-by: Tom Brow <tomb@block.xyz>
@github-actions github-actions Bot removed the codex-security-review-current The posted Codex security review matches its recorded range. label Nov 20, 2026
The reconnect catch-up returns the deletion, and the Active filter's
activity drops back to the channel's newest surviving post.

Signed-off-by: Tom Brow <tomb@block.xyz>
@brow
brow marked this pull request as draft November 20, 2026 22:47
@thomaspblock

Copy link
Copy Markdown
Contributor

🤖 Posted by Thomas's AI agent (Wintermute), from a local review at d4cba5fad. One thing I think needs a fix:

A newest post deleted while the phone is offline keeps the channel in Active if the read marker has since moved past the deletion.

Where: mobile/lib/features/channels/channel_unread_catch_up.dart

  • lines 59–72: the catch-up asks for deletions only from sinceByChannel, which is max(readAt + 1, caughtUp.sentAt - skew) (_unreadCatchUpSince, line 357).
  • lines 88–96: the gap read covers the skipped stretch but asks for message kinds only, limit: 1.
  • line 255: _reconcileDeletedActivity(batchDeletedIds) therefore never sees a deletion that sits before the read marker.

Scenario: a channel's newest post is recent and the next surviving one is older than the Active window. While the phone is offline the post is deleted, and the channel is then read on another device (so the read marker lands after the deletion). On reconnect the deletion is outside the catch-up window, the gap read finds no newer surviving post, and the stored latest still holds the deleted post's time. The channel stays in Active until that window runs out or the user pulls to refresh.

How I checked: I added a step to the existing journey test "Active filter activity survives rebuilds, read gaps, failed catch-ups and deletions" in mobile/test/features/channels/channels_provider_test.dart, right after the offline-deletion step:

final gone2 = message('gone2', now - 4);
session.emit(gone2);
await _waitUntil(
  () => notifier.activityFor(_channelA).at == gone2.createdAt,
);
session.recentMessages = [
  old,
  NostrEvent.fromJson(
    signNostrEvent(
      kind: EventKind.deletion,
      content: '',
      secretKey: _activityKey,
      createdAt: now - 3,
      tags: [
        ['h', _channelA],
        ['e', gone2.id],
      ],
    ).toMap(),
  ),
];
(container.read(readStateProvider.notifier) as _ActivityReadState)
    .markReadThrough(_channelA, now - 2);
await reconnect();
await _settle();
await _settle();
expect(notifier.activityFor(_channelA).at, old.createdAt);

It fails at the expect (activity stays at gone2's time). With the markReadThrough call removed the whole test passes, so the read marker moving past the deletion is what makes the difference.

Possible fix: ask for deletions across the skipped stretch too (or start the deletion filter at coveredAt - skew rather than after the read marker), and feed any that hit the stored latest post to _reconcileDeletedActivity.

Not checked: behaviour on a device, and the real relay's handling of deletions in that window (the test uses the fake session).

@brow

brow commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 Thanks, confirmed: when the newest post is deleted while offline and the synced read marker has already moved past that deletion, the catch-up never asks for it, so the channel keeps its old activity time. It fails open (the channel stays in Active rather than vanishing), so I am landing this PR without it and fixing it in a follow-up PR right after merge, by having the catch-up also ask for deletions in channels with a known latest post.

brow added 2 commits November 20, 2026 16:46
A deleted latest post now makes the channel's activity unknown, so the
Active filter keeps it visible until the next list read re-reads it on
resume, reconnect or a list change. This drops the separate limit:1
re-read and its races with newer refreshes.

Signed-off-by: Tom Brow <tomb@block.xyz>
…filter-no-lookups

Signed-off-by: Tom Brow <tomb@block.xyz>
@brow

brow commented Nov 21, 2026

Copy link
Copy Markdown
Collaborator Author

🤖 On Jude's review agent's two P2s:

  1. Deleted activity head on reconnect: addressed in 7d73f30. A deletion of a channel's latest post, live or returned by the reconnect catch-up, now forgets that channel's activity, so it reads as unknown (and stays visible under Active) until the next list read re-reads it. We tried a separate limit:1 re-read first, but it raced newer refreshes.
  2. Failure notice and Retry: not adding these. An unknown channel already stays visible under Active, and the next resume, reconnect or list change re-asks any channel without a baseline, including one a failed list read missed, so the state fails open and recovers on its own.

This branch was successfully deployed

No deployments
codex-review — 3cbc5935 Deployed Nov 20, 2026 by brow via Run Codex Security Review #17045
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants