Skip to content

fix(client): keep writing resubscribe commands on drain while the socket initiates - #3586

Closed
pujitha24 wants to merge 1 commit into
redis:masterfrom
pujitha24:auto/issue-3584
Closed

pujitha24 wants to merge 1 commit into
redis:masterfrom
pujitha24:auto/issue-3584

Conversation

@pujitha24

@pujitha24 pujitha24 commented Nov 5, 2026 •

Copy link
Copy Markdown

Description

Fixes #3584.

After a reconnect, a client with enough sharded subscriptions (about 64 KiB of SSUBSCRIBE commands, one per channel since 6.0.1) never becomes ready. RedisSocket#write stops at writableNeedDrain. The drain handler then calls #scheduleWrite(), which does nothing while the socket isn't ready. The socket only becomes ready after the resubscribe replies arrive, so the unwritten commands are never sent.

The drain handler now writes the queue directly while the socket initiator is pending. Behavior is unchanged once the client is ready.

Verification: Docker isn't available in my environment, so I couldn't run the Docker-backed suite. The added test uses a mock TCP server (2000 channels, then all server sockets destroyed). I ran it as a standalone copy: it timed out without the change and passed with it. I didn't run the issue's repro against a real Redis.


Checklist

  • Does npm test pass with this change (including linting)? (Build and lint pass; full suite needs Docker and wasn't run locally.)
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)? (No API change.)

Note

Medium Risk
Changes reconnect write scheduling on a critical path; scope is narrow (drain during socket initiator only) but incorrect flushing could affect handshake or large post-reconnect command batches.

Overview
Fixes reconnect hangs for clients with large sharded pub/sub subscriptions (issue #3584). After a drop, the socket initiator pipelines thousands of SSUBSCRIBE commands; the first chunk fills the TCP write buffer and drain used to call #maybeScheduleWrite(), which is a no-op until the socket is ready. ready only fires after those resubscribe replies arrive, so remaining commands were never sent and the client stayed stuck.

The drain handler now calls #write() directly while #initiatorWriting is set (handshake/resubscribe batch in flight) and the queue still has data to flush; normal post-ready behavior is unchanged.

A mock TCP integration test subscribes to 2000 sharded channels, forces reconnect, and asserts every SSUBSCRIBE is replayed before ready.

Reviewed by Cursor Bugbot for commit db6b6e9. Bugbot is set up for automated code reviews on this repo. Configure here.

…ket initiates

Motivation: since sharded resubscribe sends one SSUBSCRIBE per channel, a
client with many sharded subscriptions can queue more than the socket's
64 KiB write buffer during reconnect. RedisSocket#write stops at
writableNeedDrain, and the 'drain' handler only schedules writes once the
socket is ready. The socket becomes ready only after the initiator's
resubscribe replies arrive, so the unwritten commands are never sent and
the client never becomes ready (no error, no reconnect).

Approach: while the socket initiator is waiting on its promises, the
'drain' handler writes the queue directly instead of going through
#scheduleWrite (which is a no-op until ready). The flag is cleared when
the initiator settles.

Validation: Docker is not available here, so the Docker-backed suite was
not run. The new test in index.spec.ts uses a mock TCP server (2000
SSUBSCRIBE channels, then all server-side sockets destroyed). Run as a
standalone copy of that test with mocha -r tsx: it timed out without the
change and passed with it. npm run build and npm run lint pass.

Report: redis#3584
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5-5 (via Claude Code)
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-05T19:28:31.577626Z db6b6e9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db6b6e933b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

this.#write();
return Promise.all(promises);
return Promise.all(promises).finally(() => {
this.#initiatorWriting = false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep initiator write state scoped to its connection attempt

RedisSocket.#initiateWhileSocketAlive deliberately abandons a suspended initiator when its socket dies and starts the retry while the abandoned async credentials/handshake invocation can still settle. Because all invocations share this boolean, an old invocation's finally can run after the replacement has set it to true and clear the replacement's state; if that replacement still has more than one socket buffer of resubscribe commands, the next drain falls through to #maybeScheduleWrite(), which is a no-op before ready, recreating the hang this patch targets. Make the state attempt-scoped (or reference-counted) and only clear the invocation that set it.

Useful? React with 👍 / 👎.

@nkaradzhov

Copy link
Copy Markdown
Collaborator

Thanks. We fixed this in #3591 with a different approach. The ready check moves into #maybeScheduleWrite(), and drain calls #scheduleWrite() directly. This fixes the stall without a new flag, so no state is shared between connection attempts. Closing in favor of #3591.

@nkaradzhov nkaradzhov closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sharded pub/sub resubscribe after reconnect stalls forever when the SSUBSCRIBE commands exceed the socket's writableHighWaterMark

2 participants