Skip to content

refactor(core): unify llm event stream and tool legalization - #5086

Open
sailist wants to merge 3 commits into
mainfrom
refact-393-11-18-llm-event-stream
Open

sailist wants to merge 3 commits into
mainfrom
refact-393-11-18-llm-event-stream

Conversation

@sailist

@sailist sailist commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Requirement or Bug

使用 core、agent-core-v2 或 klient 消费 LLM 事件时,run 与 attempt 的边界混杂,外部调用者难以用同一套生命周期理解请求、重试和 recovery。与此同时,convert 会静默过滤 deferred 工具,但直接 API 实测可以接受这些工具;顶层和 system message 中的同名声明也缺少统一的最终处理。

这个 PR 统一 LLM 事件流与工具声明编码,让每次 generate、每个 attempt、重试、recovery 和重复声明都有明确且可观测的行为。

Bug Reproduction Steps

N/A

Root Cause

N/A

Code Changes

核心思路

把能力解析、合法化和工具声明规范化收归到同一条 encode pipeline;事件则按 run、attempt 和增量数据分别建模。正常 generate() 与直接 protocol.encode 现在共享同一次 capability 合并,不再需要各测试或调用方自行补齐。

flowchart LR
    Input[LlmRequest] --> Caps[with-capabilities]
    Caps --> Legalize[legalize + legalize-tools]
    Legalize --> Convert[convert]
    Convert --> Transport[send / sendStream]
    Transport --> Settle[settle]
    Settle --> Events[started / requested / delta / retry / ended]
Loading

legalize-tools 是最终兜底,外部 resolver 仍可在更早阶段解决重复声明,从而避免最后的 notice/warn。空 system message 不在这个阶段删除,交由其他 pipeline 处理。

关键决策

决策 选择 没选 理由
capability 合并位置 runPipeline 入口一次 generate 与 legalize-tools 各自合并 同时覆盖生产 generate 和直接 encode 路径
工具规范化位置 legalize-model 后的 legalize-tools 在 convert 中过滤 deferred convert 只做映射,不再承担主动过滤
同名工具优先级 顶层优先,顶层同名取最后一项;随后从最新 system message 向最旧扫描 最新 system 声明优先 与当前 resolver 的顶层视图一致;接受动态加载 token 成本上升
重复声明通知 每个重名工具聚合一条 llm.notice 和一条 log.warn 静默去重 保留工具名、保留位置和删除数量,便于外部定位
空消息 保留 system message 和 tools: [] 在工具去重时删除空消息 空消息生命周期由专门 pipeline 负责
行为测试 真实 provider 经本地 aimock 发请求 恢复手写 SDK/fetch 桩 同时覆盖真实编码、HTTP/SSE、重试和请求 journal

阅读指南

  • packages/core/src/llm/response.ts:统一后的事件类型与载荷。
  • packages/core/src/llm/generate.ts 和 policy/:run、attempt、重试与最终 outcome 的生成路径。
  • packages/core/src/llm/protocol/pipeline.ts:capability 合并、阶段 trace 和 convert 映射。
  • packages/core/src/llm/protocol/rules/tools.ts:deferred 元数据清理、同名去重和通知。
  • packages/agent-core-v2/src/agent/loop/:journal、turn projection、request observation 与 loopService 的消费迁移。
  • packages/core/test/llm/:事件生命周期、provider、错误、thinking 和真实 aimock transport 覆盖;数据 fixtures 可略读。

Behavior Changes and Affected Users

Behavior Before After Who relies on the old behavior Escape hatch
LLM 事件词汇 llm.sent、llm.streaming.*、llm.retrying 与内部 attempt.* 混合 llm.started / requested / delta.* / retry / ended;不双发旧名称 core、klient、v2 消费者及监听实时事件的外部脚本 无旧事件名兼容层;旧 session 仍通过派生步骤与消息记录恢复
Run 与 attempt 边界 started/ended 与 attempt 数据交错,部分路径有合成事件 每次 generate 恰好一次 started/ended;每个实际 attempt 一次 requested;delta 仅按真实到达发送 依赖失败、abort、空响应和重试顺序的客户端 partial history、usage、finish、timing 和 abort 保留输出均有回归覆盖
llm.requested 与 llm.ended 载荷 请求摘要和最终结果语义不统一 requested 沿用旧 wire 摘要,不含完整 messages/tools/native body;ended 只含最终 outcome,不聚合 attempts wire/vis/请求观测消费者 独立 tools snapshot 与既有归因字段保留,宿主缺省字段不伪造
deferred 顶层工具 convert 过滤 deferred: true 所有保留工具都进入 convert;非动态模型只删除 deferred 元数据 动态加载工具及依赖 token 节省的配置 顶层声明优先可能增加 prompt token;不支持动态加载的模型按普通工具发送
同名工具声明 上游只处理部分 inline 重复,直接 encode 可重复发送 顶层优先并取最后一项,再从最新 system message 向最旧去重;每个重名工具发 notice/warn 动态工具加载、显式 tools override、历史 schema 声明 外部 resolver 可提前去重,避免最终 notice/warn
空 system message 工具全部移除后可能被顺带删除 legalize-tools 不删除 system message,保留 tools: [] 依赖专门空消息处理阶段的调用方 空消息清理继续由其他 pipeline 负责
capability 与阶段观测 generate 与直接 encode 路径可能不同,trace 没有实际日志 runPipeline 统一合并一次,所有 run 阶段输出 log.debug({ pass }) 直接 encode 调用方及排查编码问题的维护者 不记录消息正文或工具 schema 正文
测试与文档 历史 LLM 测试大量删除,provider 多用 fake client 恢复并补充行为测试,真实 provider 走本地 aimock;同步中英文 LLM/MCP 文档 后续维护者与协议回归验证 未运行全 monorepo 和远程/Windows 测试

验证

  • Rebase 到 7e2c565e31fb 后:core 与 toolSelectService 共 28 个文件,331/331。
  • @moonshot-ai/core typecheck 通过。
  • 标准 pnpm lint 通过:注释检查、oxlint、import boundary、guards、ratchet、instructions 均通过。
  • pnpm install --frozen-lockfile 成功。
  • VitePress npm run build 成功;仅有既有 ES2024 target 与 chunk-size 警告。
  • Rebase 前已完成普通流式回答、多 step 工具、429/500 重试和 session 恢复的 TUI 对照验证,8/8 通过。
  • 只读敏感信息审计通过;测试密钥均为占位值,未发现真实内部标识或不应提交的产物。

Checklist

  • I have read the CONTRIBUTING document.
  • I have linked a related issue (external PRs: issue must have a maintainer's /approve).
  • I have added tests that prove my feature works.
  • The behavior-change table above is complete, and every removed behavior or flipped default is named in the changeset and either has an escape hatch or was explicitly approved by a maintainer in this PR.(行为变化已列明;本次按要求不写 changeset,旧事件不双发与 token tradeoff 请维护者审阅。)
  • Ran gen-changesets skill, or this PR needs no changeset.(任务所有者明确要求本次不写 changeset。)
  • Ran gen-docs skill, or this PR needs no doc update.(已手动同步中英文文档,未运行 gen-docs。)

@changeset-bot

changeset-bot Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: cffb84f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T07:36:54.017529Z cffb84f New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@pkg-pr-new

pkg-pr-new Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
pnpm dlx https://pkg.pr.new/@moonshot-ai/kimi-code@cffb84f
npx https://pkg.pr.new/@moonshot-ai/kimi-code@cffb84f

commit: cffb84f

@github-actions

Copy link
Copy Markdown
Contributor

❌ Nix build failed

Hash mismatch in pnpmDeps:

Hash
specified sha256-JiwS/wfwbQxWy0sR/AauFxE5aYOOCINqONAux2fU++8=
got sha256-AFS+3QIuvVPuEHmSIi6m9oEdjlW/6+tAJAndDDXWVrM=

Please update flake.nix with the got hash.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 39841d6b7b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +164 to +165
export type LlmDeltaEvent =
| { readonly type: 'llm.delta.response_header'; readonly headers: Record<string, string> }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add release metadata for the breaking event vocabulary

This commit removes the old exported event types and changes every runtime discriminator from llm.streaming.*/llm.retrying to llm.delta.*/llm.retry, but adds no .changeset entry. This reaches external consumers because packages/klient/src/core/facade/kosong-types.ts publicly aliases GenerateEvent to LlmRunEvent; upgrading will therefore break both compiled consumers and scripts matching the old event names without the required major-version signal. Obtain approval for the breaking bump and add the corresponding changeset before release.

AGENTS.md reference: AGENTS.md:L87-L87

Useful? React with 👍 / 👎.

system: module.profile.systemPrompt,
messages: module.content.messages.flatMap((message) => pipeline.message(lowerableMessage(message), module)),
tools: (module.profile.tools ?? []).filter((tool) => tool.deferred !== true).map(pipeline.tool),
tools: (module.profile.tools ?? []).map(pipeline.tool),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Include newly transmitted deferred tools in request hashes

When a top-level tool has deferred: true, this line now transmits it to the provider, but llmStartedEvent/llmRequestedEvent in policy.ts:109 and llmToolsSnapshotPayload still remove every deferred tool before computing the tools hash and durable snapshot. Consequently llm.requested.toolsHash, llm.tools_snapshot, request logs, and cache-break telemetry describe a different tool set from the actual provider request; for non-dynamic models this occurs whenever the new legalization path promotes a deferred tool to an ordinary declaration. Compute observability hashes from the legalized/transmitted tool set rather than the old filtered view.

Useful? React with 👍 / 👎.

system: module.profile.systemPrompt,
messages: module.content.messages.flatMap((message) => pipeline.message(lowerableMessage(message), module)),
tools: (module.profile.tools ?? []).filter((tool) => tool.deferred !== true).map(pipeline.tool),
tools: (module.profile.tools ?? []).map(pipeline.tool),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep deferred tools out of dynamic models' top-level list

With ToolSelect enabled for a model declaring dynamically_loaded_tools, the Loop seeds its profile with initialTools(), which marks every deferred MCP tool, and requestTools(..., disclosure=true, dynamic=true) returns that base array unchanged. legalizeTools also preserves those entries for a dynamic model, so this unconditional mapping now serializes every deferred schema into the ordinary top-level provider tools list. This defeats the flag's on-demand-loading contract and restores the full context cost for configurations with many deferred MCP tools; filter unloaded deferred entries in the ToolSelect resolver or otherwise keep them out of this conversion while still allowing explicitly retained direct-API tools.

Useful? React with 👍 / 👎.

Comment on lines 30 to 33
model: {
...request.model,
spec: withProtocolCapabilities(request.model.spec, request.protocol.base),
spec: request.model.spec,
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve inferred capabilities for custom protocols

When a Protocol supplies capabilities through its capability(modelName) callback but the request model omits explicit capabilities, generate() previously passed the merged spec to both encode and decoder. The merge now happens only inside runPipeline, so a custom Protocol.encode that does not use that helper—and even a defineProtocol decoder or patched decoder that inspects module.model.spec.capability—receives the unmerged model and can silently choose the wrong decoding or feature behavior. Build the capability-enriched module in generate() as before; runPipeline can retain its idempotent merge for direct encode calls.

Useful? React with 👍 / 👎.

- unify run boundaries, request summaries, deltas, retries, and outcomes
- normalize deferred and duplicate tool declarations with notices and debug tracing
- migrate consumers and add aimock-backed LLM behavior coverage
@sailist
sailist force-pushed the refact-393-11-18-llm-event-stream branch from 39841d6 to cffb84f Compare November 19, 2026 07:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant