Skip to content

fix(insights): retain only requested results while paging - #1783

Closed
zack-dev-cm wants to merge 1 commit into
rohitg00:mainfrom
zack-dev-cm:fix/1548-insight-list-payload
Closed

zack-dev-cm wants to merge 1 commit into
rohitg00:mainfrom
zack-dev-cm:fix/1548-insight-list-payload

Conversation

@zack-dev-cm

@zack-dev-cm zack-dev-cm commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Listing a small number of historical insights currently keeps the entire collection in memory before filtering and sorting. Stream the existing pages into a sorted window capped at the requested limit, preserving complete records, filtering, confidence order and backend-specific ties. The Redis ordering key and comparator are shared with the existing KV ordering helper.

Related to #1548's listing path. All keys are still scanned; the list-only compatibility fallback still materializes its supplied list.

Validation:

  • All 54 frozen checks pass, including both 4,096-record retained-heap checks: the original source exceeds the unchanged 24 MiB budget at about 135 MB, and the patch stays below it. Complete record and ordering fixtures are unchanged. The original unbounded-list guard and 89 existing KV/reflection/bounds/frame checks pass.
  • Required build, skills/docs generation and checks, and evaluation typecheck pass. Full local suite: 4,972 passed, the same 13 mesh failures, 7 skipped (4,992 total). All 4,938 original outcomes and complete errors match after the checkout-path change. Strict checking retains the original 28 source diagnostics with none added.
  • On installed Linux/Node 22/iii 0.25.1 archives, all 30 complete RPC/HTTP request, status and returned payload comparisons match the original exactly. The probe records 6 passes and the same 3 seed-versus-engine confidence-rounding assertion failures on both archives; no float rounding, field removal or tolerance was added. Both isolated runs use cached dependencies, make zero model requests and close all seven owned ports.

Hosted CI verified at 19:00 UTC on 13 November 2026, on 05fcfdd (run 43487411593): all four Ubuntu Node 20/22/24/26 jobs pass the full 4,992-case suite, all 54 new checks and 144 affected checks, with both workflow build invocations. Node 20 reports 4,983 passed and 9 existing skips; Node 22/24/26 each report 4,985 passed and 7 skips. All 38 mesh checks pass in each job. macOS remains unvalidated in this run; a passing eight-job matrix is not established.

Summary by CodeRabbit

  • Bug Fixes
    • Insight lists now sort by confidence, highest first, with insertion order used to break ties.
    • Listing applies project and confidence filters while scanning results and retains only the requested number of matches, helping limit memory use for large histories.
    • Existing deleted-insight filtering and validation behavior are preserved.

Signed-off-by: zack-dev-cm <112114097+zack-dev-cm@users.noreply.github.com>
@vercel

vercel Bot commented Nov 13, 2026

Copy link
Copy Markdown

@zack-dev-cm is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Insight listing now scans paged KV results when available and retains only the requested number of matching insights, ordering by confidence and using insertion order for Redis ties. Shared insertion-order helpers support that ordering. Tests cover backend results, validation, paging errors, fallback behavior, and memory use.

Changes: Insight Listing

Layer / File(s) Summary
Insertion-order comparison helpers
src/state/kv.ts
Exported helpers derive and compare insertion-order keys. orderLikeInsertion now uses those helpers.
Limited insight listing and validation
src/functions/reflect.ts, test/fixtures/insight-list.json, test/fixtures/insight-list-memory.ts, test/insight-list.test.ts
Insight listing filters records and retains the requested number while scanning pages when available. Tests cover file and Redis results, validation, ordering, page errors, fallback behavior, and memory use.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Refactor

Suggested reviewers: rohitg00

Merge Risk: Minimal · up to 05fcf

Insight listing now keeps only the requested top results while paging, which lowers memory use while preserving filtering and ordering. No actionable merge-blocking risk is identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "fix(insights): retain only requested results while paging" matches the main change. The PR changes the insight-list path so it keeps only the requested number of matching insights while str…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "fix(insights): retain only requested results while paging" matches the main change. The PR changes the insight-list path so it keeps only the requested number of matching insights while streaming pages, instead of holding the full collection in memory. The title names the fix, the scope (insights), and the paging behavior. It is specific and does not use vague wording.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 14536706-9c13-421e-b0a6-7b80b1f70052
Commits

Reviewing files that changed from the base of the PR and between cf9a738 and 05fcfdd.

Files selected for processing: 5

  • src/functions/reflect.ts
  • src/state/kv.ts
  • test/fixtures/insight-list-memory.ts
  • test/fixtures/insight-list.json
  • test/insight-list.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@rohitg00

Copy link
Copy Markdown
Owner

Merged as part of #1811, which kept your commits and added review fixes on top (details in its description). Thanks for the contribution.

@rohitg00 rohitg00 closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants