Repository navigation
fix(mcp): say what recall's max_tokens counts, and return compact JSON - #4870
Merged
nicoloboschi merged 3 commits intoOct 6, 2026
Merged
nicoloboschi merged 3 commits into
nicoloboschi merged 3 commits into
Conversation
The MCP recall tool described max_tokens as the "maximum tokens to return in results". The engine counts only each fact's text, as its own docstring says, while every result also carries its id, tags, scores and dates. At the default 4096, a recall against a real bank returned about 4,500 tokens of fact text inside an 88-119k character response, so an agent sizing its context from the tool description was off by about six times. The bank-id copy of the tool also pretty-printed its JSON (indent=2), about a fifth of that response, while FastMCP already sends the session-bank copy's dict compact. It is now compact too.
…elper - mcp-server.md still said max_tokens was the maximum tokens returned - TestRecallResponseSize._result gets a return type; import moved to module top
nicoloboschi
force-pushed
the
fix/mcp-recall-max-tokens-scope
branch
from
October 6, 2026 09:08
4a9ad03 to
67a76a0
Compare
Contributor
Author
|
Thanks for fixing the docs line I missed, and for the merge. |
This was referenced Oct 8, 2026
Merged
This was referenced Oct 16, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The MCP
recalltool describesmax_tokensas the "Maximum tokens to return in results". The engine counts only each fact'stext, asMemoryEngine.recall_async's docstring says ("counts only 'text' field"). Every result also carries its id, tags, scores, dates, document and chunk ids. An agent that sizes its context from the tool description is off by a wide margin.Measured on 0.10.1 against a real bank (~900 facts), with
budget: "low"and the defaultmax_tokens: 4096, two queries returned:textindent=2whitespaceSo the budget works as the engine intends. The response is still about six times what the tool promises, and Claude Code refused both results as over its tool-output limit.
Fix
max_tokensis now described as the token budget for the facts' text. It says the per-result metadata is extra, so the response is several times larger.model_dump_json()instead ofmodel_dump_json(indent=2)). The session-bank copy returns a dict, which FastMCP already serialises compact (pydantic_core.to_jsonwith no indent), so the two copies now send the same shape. This removes about a fifth of the response. Callers alreadyjson.loadsthis string.Not changed: what
max_tokenscounts, and which fields a result carries. Either would change the engine's contract or the payload, and that is your call. For example, a separate budget covering the whole response, or leaving outscoresunless asked for, could close the rest of the gap. I'm happy to follow up with either.Test
TestRecallResponseSizeintests/test_mcp_tools.py:test_bank_id_recall_returns_compact_jsonuses a realRecallResult. It checks that the output parses to the same payload and contains no newline.test_recall_docs_say_max_tokens_counts_fact_text_onlyis parametrized over both copies.All three fail on the parent commit. With only the docstring fixed, the compact test still fails.
tests/test_mcp_*.pypass (436 on current main). The full api-slim suite (without thehs_llm_*markers): 10,020 passed. The 5 failures and 1 error are intest_fact_extraction_output_ratio.py/test_fact_extraction_analysis.py, which needHINDSIGHT_API_LLM_API_KEYand fail the same way on main. Ruff and ty are unchanged from main. The two pre-existing F841s intest_mcp_tools.pyand the 15 ty diagnostics are on main too.