Repository navigation
fix(api): stop a big graph read from freezing the event loop (#5703) - #5782
Merged
Merged
Conversation
GET /banks/{id}/graph with a few thousand observations froze the API's event
loop for seconds at a time, long enough for the liveness probe to restart the pod.
- Cap the whole answer at GRAPH_MAX_EDGES (10k). Only the stored links were
capped; the edges derived on top of them (inherited from source memories,
shared source, shared entity) were not, and a dense bank derived hundreds of
thousands. Derivation now stops at the cap, keeping stored links first, then
inherited, shared-source, and shared-entity edges (rarest entity first).
- Build the answer in a worker thread, after the DB connection is released.
- Encode it with orjson instead of validating it against GraphDataResponse on
the loop; a test holds the built dict to the model's shape.
Synthetic bank, 1,000 nodes: 205k edges / 61 MB / 72s loop stall before;
10k edges / 4.3 MB after. 7,000 nodes: 13 MB, encoding ~20 ms (was ~10 s).
Review fix: with one shared cap, a well-linked bank filled it with stored links and lost every entity and shared-source edge. Stored (with inherited) and derived edges now get GRAPH_MAX_EDGES each; test covers the full-budget case.
This was referenced Oct 20, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5703.
Problem
On a bank with a few thousand observations, the Memories → Constellation view (and "Load more") made
GET /banks/{id}/graphfreeze the event loop for seconds./health/livecould not answer, and Kubernetes restarted the pod, failing recall/retain for every client.Only the stored links were capped (10k). The edges the engine derives on top of them (observations inheriting their source memories' links, observations sharing a source, memories sharing an entity) had no cap. A dense bank derived hundreds of thousands, then validated them against the response model, all on the loop.
Fix
GRAPH_MAX_EDGES, now shared by the store and the engine). Stored links (with the ones observations inherit) and derived edges each get a 10k budget, so a well-linked bank still shows entity edges. Derivation stops at the cap, shared source before shared entity (rarest entity first, since a name in every memory says nothing).GraphDataResponseon the loop. The OpenAPI schema is unchanged;tests/test_graph_assembly.pyholds the built dict to the model's shape.No node limit: the Table view uses the same endpoint, and its "Load more" grows
limitinstead of paging, so capping nodes would break it.Behaviour changes
Evidence
Synthetic bank (store mocked, machine heavily loaded, so times are inflated):
23 graph-related tests pass (new
test_graph_assembly.py, plus graph filtering, tag scope, row models, cancellation, HTTP integration).