Skip to content

fix(consolidate): stop repeated runs from storing reworded duplicate memories - #1506

Open
Rex57 wants to merge 1 commit into
rohitg00:mainfrom
Rex57:fix/1486-consolidate-duplicates
Open

Rex57 wants to merge 1 commit into
rohitg00:mainfrom
Rex57:fix/1486-consolidate-duplicates

Conversation

@Rex57

@Rex57 Rex57 commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #1486. Also fixes the in-run case reported in #747.

POST /agentmemory/consolidate runs mem::consolidate, which regroups the same high-importance observations by concept on every run and asks the LLM for up to 10 memories. Nothing recorded that a group had already been consolidated, and the only duplicate check was an exact, case-insensitive title match against the memory list read at the start of the run. The LLM rewords titles between runs, so every run stored new copies of the same facts. On one real store, five manual runs left 80 memories that were 10 topics × 8 copies.

Changes in mem::consolidate:

  • Skip already-consolidated groups. If a current memory's sourceObservationIds already contains every top observation of a group, the group is skipped without an LLM call.
  • Evolve by shared observations, not only by title. When no title matches, the latest memory that shares at least half of the group's top observations becomes the parent: new version, supersedes it, old one isLatest: false.
  • Latest memories only, including this run's. Both checks read only latest memories (a title match against a superseded version used to fork the chain), and memories created or evolved earlier in the run are added to the list they read. That closes Same title consolidation produces duplicate and orphaned memories #747 and removes the unused existingTitles set.
  • The result and the completion log carry skipped.

The project-scope guard is unchanged; it moved up so both checks share it. The scheduled consolidate-pipeline does not call mem::consolidate; only this REST route does.

Checks

  • test/consolidate-dedup.test.ts (all three fail on main):
    • two runs with unchanged observations → second run makes no LLM call and writes nothing (skipped: 1)
    • a new observation joins the group and the LLM picks a new title → one latest memory, version 2, parent is the first memory
    • two concept groups in one run returning the same title → one latest memory with that title
  • Existing consolidate-project-scope, consolidation-pipeline and consolidation-default tests pass unchanged
  • npm run build, npm run skills:gen, npm run docs:sync (no changes), npm run skills:check
  • npm test: 2597 passed, 4 skipped

…memories

mem::consolidate regrouped the same high-importance observations on every
run and asked the LLM for new memories. Its only duplicate check was an
exact title match against the memory list read at the start of the run, and
the LLM rewords titles, so each POST /agentmemory/consolidate stored new
copies of the same facts, and two groups in one run could still produce two
memories with one title.

- Skip a concept group without an LLM call when a current memory already
  holds all of its top observations in sourceObservationIds.
- Evolve the latest memory that shares at least half of those observations
  when no title matches, so a rewording becomes a new version.
- Match only latest memories, and add memories created during the run to the
  list both checks read. This removes the unused existingTitles set.
- Report the number of skipped groups.

Signed-off-by: Rex57 <203883624+Rex57@users.noreply.github.com>
@vercel

vercel Bot commented Oct 8, 2026

Copy link
Copy Markdown

@Rex57 is attempting to deploy a commit to the rohitg00's projects Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 34 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 6f01b07b-d026-4796-846b-1c78ba1ae132
📥 Commits

Reviewing files that changed from the base of the PR and between 007a1a7 and ec55b51.

📒 Files selected for processing (2)
  • src/functions/consolidate.ts
  • test/consolidate-dedup.test.ts
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repeated POST /agentmemory/consolidate runs write reworded duplicate memories Same title consolidation produces duplicate and orphaned memories

1 participant