You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
readBody corrupts multi-byte UTF-8 on bodies larger than one chunk (breaks /import and large /observe with non-ASCII content) #930
readBody() decodes each incoming TCP chunk independently with chunk.toString() and concatenates the resulting strings. When a multi-byte UTF-8 sequence (e.g. Cyrillic, CJK, emoji) straddles a chunk boundary, each half is decoded separately and the character is corrupted (replaced with U+FFFD). For request bodies larger than Node's default ~64 KB highWaterMark this is effectively guaranteed, so:
POST /agentmemory/import of a real export (>64 KB, non-ASCII content) fails with an opaque 400 and an empty body (the corrupted string fails JSON.parse).
Large POST /agentmemory/observe payloads with non-ASCII content are silently stored corrupted.
Affected code
dist/index.mjs ~L15236 and dist/src-MOg2zKJs.mjs ~L14745 (identical):
The 1 MB /import cap is self-defeating.readBody hard-rejects bodies >1e6, but /import consumes the output of /export, which exceeds 1 MB for any non-trivial memory. With no documented chunking, a user cannot round-trip their own export once it grows past ~1 MB. Consider a higher limit for /import, or a documented chunked-import path.
Opaque 400. When the body fails to parse, the response has no body. Returning {"error":"invalid JSON"} (as other validation paths already do, e.g. the handler's own {"error":"exportData is required"}) would save a lot of debugging time.
Environment
agentmemory 0.9.27 (npm latest, published 2026-06-07), Node.js 24, Windows 11, native iii-engine 0.11.2.
This doesn't need a different memory backend — it's a one-function bug in agentmemory's own readBody (src/viewer/server.ts). It does data += chunk.toString() per TCP chunk, so a multi-byte UTF-8 sequence (Cyrillic/CJK/emoji) split across a chunk boundary gets decoded in halves and corrupted (→ U+FFFD). Past Node's ~64 KB highWaterMark it's effectively guaranteed, which is exactly why POST /import of a real non-ASCII export returns an opaque 400.
Buffer the raw Buffers and decode once:
function readBody(req) {
return new Promise((resolve, reject) => {
- let data = "";+ const chunks = [];
let size = 0;
req.on("data", (chunk) => {
size += chunk.length;
if (size > 1_000_000) { req.destroy(); reject(new Error("too large")); return; }
- data += chunk.toString();+ chunks.push(chunk);
});
- req.on("end", () => resolve(data));+ req.on("end", () => resolve(Buffer.concat(chunks).toString("utf8")));
req.on("error", reject);
});
}
Verified by round-tripping a 1.7 MB Cyrillic export through export→import. @rohitg00 can lift this straight in. (Orthogonal: the same function's 1 MB cap also blocks /import of any non-trivial /export.)
Activity
woshilaohei commentedon Jun 14, 2026
Nice catch - UTF-8 corruption on chunk boundaries is a classic async read bug.
If you need a memory server that handles Unicode correctly today, MindCore Memory MCP was built with this in mind (JSONL with fsync, no async read coruption): https://github.com/woshilaohei/mindcore-memory-mcp
Won't fix agentmemory but might unblock you if you need memory working reliably now.
alexstraf1 commentedon Jun 15, 2026
This doesn't need a different memory backend — it's a one-function bug in agentmemory's own
readBody(src/viewer/server.ts). It doesdata += chunk.toString()per TCP chunk, so a multi-byte UTF-8 sequence (Cyrillic/CJK/emoji) split across a chunk boundary gets decoded in halves and corrupted (→ U+FFFD). Past Node's ~64 KB highWaterMark it's effectively guaranteed, which is exactly whyPOST /importof a real non-ASCII export returns an opaque 400.Buffer the raw Buffers and decode once:
function readBody(req) { return new Promise((resolve, reject) => { - let data = ""; + const chunks = []; let size = 0; req.on("data", (chunk) => { size += chunk.length; if (size > 1_000_000) { req.destroy(); reject(new Error("too large")); return; } - data += chunk.toString(); + chunks.push(chunk); }); - req.on("end", () => resolve(data)); + req.on("end", () => resolve(Buffer.concat(chunks).toString("utf8"))); req.on("error", reject); }); }Verified by round-tripping a 1.7 MB Cyrillic export through export→import. @rohitg00 can lift this straight in. (Orthogonal: the same function's 1 MB cap also blocks
/importof any non-trivial/export.)fix: env hydration, indexing, consolidation, connectors, hardening
fix: env hydration, indexing, consolidation lifecycle, connector acti…