fix(retaindb): fix API routes, add write queue, dialectic, agent model, file tools#5461
Merged
Conversation
…l, file tools The previous implementation hit endpoints that do not exist on the RetainDB API (/v1/recall, /v1/ingest, /v1/remember, /v1/search, /v1/profile/:p/:u). Every operation was silently failing with 404. This rewrites the plugin against the real API surface and adds several new capabilities. API route fixes: - Context query: POST /v1/context/query (was /v1/recall) - Session ingest: POST /v1/memory/ingest/session (was /v1/ingest) - Memory write: POST /v1/memory with legacy fallback to /v1/memories (was /v1/remember) - Memory search: POST /v1/memory/search (was /v1/search) - User profile: GET /v1/memory/profile/:userId (was /v1/profile/:project/:userId) - Memory delete: DELETE /v1/memory/:id with fallback (was /v1/memory/:id, wrong base) Durable write-behind queue: - SQLite spool at ~/.hermes/retaindb_queue.db - Turn ingest is fully async — zero blocking on the hot path - Pending rows replay automatically on restart after a crash - Per-row error marking with retry backoff Background prefetch (fires at turn-end, ready for next turn-start): - Context: profile + semantic query, deduped overlay block - Dialectic synthesis: LLM-powered synthesis of what is known about the user for the current query, with dynamic reasoning level based on message length (low / medium / high) - Agent self-model: persona, persistent instructions, working style derived from AGENT-scoped memories - All three run in parallel daemon threads, consumed atomically at turn-start within the prefetch timeout budget Agent identity seeding: - SOUL.md content ingested as AGENT-scoped memories on startup - Enables persistent cross-session agent self-knowledge Shared file store tools (new): - retaindb_upload_file: upload local file, optional auto-ingest - retaindb_list_files: directory listing with prefix filter - retaindb_read_file: fetch and decode text content - retaindb_ingest_file: chunk + embed + extract memories from stored file - retaindb_delete_file: soft delete Built-in memory mirror: - on_memory_write() now hits the correct write endpoint
- SQLite write queue: thread-local connection pooling instead of creating+closing a new connection per operation - Prefetch threads: join previous batch before spawning new ones to prevent thread accumulation on rapid queue_prefetch() calls - Shutdown: join prefetch threads before stopping write queue - Add 73 tests covering _Client HTTP payloads, _WriteQueue crash recovery & connection reuse, _build_overlay deduplication, RetainDBMemoryProvider lifecycle/tools/prefetch/hooks, thread accumulation guard, and reasoning_level heuristic
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvage of PR #5456 by @Alinxus (original RetainDB plugin author).
The RetainDB plugin shipped in #4623 was broken end-to-end — every API route was wrong, silently 404ing. Users who activated RetainDB got zero memory.
What the contributor fixed (cherry-picked from #5456):
Follow-up improvements:
Files changed
plugins/memory/retaindb/__init__.py— rewritten plugin (302 → ~760 lines)tests/plugins/test_retaindb_plugin.py— new, 73 testsTest results
Closes #5456. Credit to @Alinxus for the original fix — their commits are cherry-picked with authorship preserved.