hotfix(ai-diagnosis): correct misclassifications and surface provider error JSON#15241
hotfix(ai-diagnosis): correct misclassifications and surface provider error JSON#15241SiinXu wants to merge 3 commits into
Conversation
…ror JSON The rule layer conflated rate-limit with quota, treated 403 geo-blocks as auth, and let generic network rules steal MCP/OCR timeouts; the AI layer ignored the actual provider responseBody/data/finishReason that carry the diagnostic signal, and the UI showed raw exception text when diagnosis itself failed. This rewrites the classification rules, pipes the missing fields into the AI prompt, and adds a rule-based fallback in the UI on diagnosis failure. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Siin Xu <31815270+SiinXu@users.noreply.github.com>
… fallback
Tighten this hotfix to only the misclassifications confirmed in real provider
responses. Defensive keywords without a known triggering case ('exceeds the
maximum', 'tls', 'ssl handshake', 'geographic', 'geo block') are removed; the
diagnosis-failed UI fallback (rule-based text + ai_failed i18n key) is reverted
because it is a UX improvement orthogonal to this hotfix and belongs in a
separate PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Siin Xu <31815270+SiinXu@users.noreply.github.com>
There was a problem hiding this comment.
Note
This issue/comment/review was translated by Claude.
This code review primarily found one inconsistency between the PR description and the final code implementation (see inline comments for details). Other than that, the refactoring of rule categorization for newly added geo-detection and rate limiting is very clear, and the unit tests are also very comprehensive.
| | 'rate_limit' | ||
| | 'context_length' | ||
| | 'payload' | ||
| | 'network' |
There was a problem hiding this comment.
Note
This comment was translated by Claude.
[B7] The PR description mentions: "On diagnosis failure, the UI shows the rule-based classification text first, then the raw error below as a small detail, with the retry button kept." However, in the final commit of this branch, this logic and the i18n translation key "ai_failed" were removed in commit "2e1f28c8a". The current code does not include this fallback UI logic, which conflicts with the PR description. I suggest updating the PR description or implementing this feature.
Original Content
[B7] PR 描述中提及:"On diagnosis failure, the UI shows the rule-based classification text first, then the raw error below as a small detail, with the retry button kept."。但在分支的最终提交中,该部分逻辑和 i18n 翻译键 "ai_failed" 在 commit "2e1f28c8a" 被移除了。目前代码中并未包含此兜底 UI 逻辑,与 PR 描述存在冲突,建议更新 PR 描述或补充该功能实现。
|
Note This comment was translated by Claude. Why are all PRs submitted by AI now? Original Content怎么现在都是全AI提的PR了 |
There was a problem hiding this comment.
Note
This review was translated by Claude.
Two review points: complete the quota categorization/diagnostic context/tests for 402 Payment Required, and complete non-core language translations for the newly added diagnostic text.
Original Content
留下两点 review:补齐 402 Payment Required 的 quota 分类/诊断上下文/测试,以及为新增诊断文案补全非核心语言翻译。
| } | ||
|
|
||
| // Quota / rate limit (429) | ||
| // Quota / balance exhausted — check first so "429 + insufficient_balance" routes here, not to rate_limit |
There was a problem hiding this comment.
[A1] This split still misses plain HTTP 402 Payment Required, which several providers use for billing/quota failures. If a provider returns status 402 with a generic message, it can fall through to unknown instead of quota. Please route numStatus === 402 to quota, mirror that in ErrorDiagnosisService so the AI prompt uses quota/billing context for 402, and add unit coverage for the classifier and diagnosis hint.
| "payload": "Nội dung yêu cầu quá lớn, vui lòng giảm kích thước tệp hoặc văn bản", | ||
| "proxy": "Lỗi proxy hoặc chứng chỉ SSL, hãy kiểm tra cài đặt proxy và mạng", | ||
| "quota": "Hạn mức tài khoản đã cạn kiệt, vui lòng nạp tiền hoặc chuyển đổi nhà cung cấp", | ||
| "rate_limit": "[to be translated]:Too many requests in a short time. Wait a moment and retry, or switch to a model with a higher rate limit", |
There was a problem hiding this comment.
[C1] The new rate_limit / region diagnosis strings are user-visible, but the translate files still contain [to be translated]:... English placeholders across the non-core locales. Please provide actual translations for all touched locale files, and include the new error.http.402 translation as well if 402 handling is added.
…ations Address review feedback from #15241: - Add `numStatus === 402` to the quota branch in `errorClassifier.ts`. HTTP 402 Payment Required is the canonical billing-failure status used by several providers and gateways. Previously these fell through to `unknown` and missed the quota navTarget / context. - Mirror the 402 check in `ErrorDiagnosisService` so the AI prompt receives billing/quota context for 402, not the rate-limit context (which would advise the user to "wait and retry" — wrong for a billing failure). - Add `error.http.402` text to all 12 locale files. - Translate the previously-placeholder `rate_limit` and `region` strings across the 9 non-core locales (de, el, es, fr, ja, pt, ro, ru, vi). - Add 3 unit tests covering the 402 routing path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Siin Xu <31815270+SiinXu@users.noreply.github.com>
|
Thanks for the review — addressed all three points in A1 (402 → quota) — done. C1 (translations) — done. B7 (PR description / Also added 402 to the "After this PR" section in the description so it's discoverable from the PR summary. Ready for re-review when you have a moment. @ousugo @zhibisora |
What this PR does
Before this PR:
The AI diagnosis feature gave misleading results in several common scenarios:
unsupported_country_region_territory, Anthropic geo-blocks) was diagnosed as "API key invalid, please reconfigure" — sending users to re-check a key that was fine./settings/generalinstead of the relevant feature settings.responseBody,data) and Gemini/Anthropic's structuredfinishReason(SAFETY/RECITATION), so the AI had to guess from the wrapper message.After this PR:
429 + insufficient_balancestill routes to quota.402 Payment Requiredroutes toquotaand is mirrored in the AI prompt context, so providers that signal billing failure via 402 get billing-language guidance instead of falling through tounknown.regioncategory for geo-blocks, with its own i18n text and navigation to general/proxy settings.stream,parse, anddeprecatedkeyword matches to remove false positives (e.g.max_tokens must be a valid JSON numberno longer flags as a parse error; "parameter is deprecated" no longer triggers "model deprecated").finishReason).responseBody,data,finishReason, and auto-fillsprovider/modelIdfrom the error itself when the caller did not provide context.Fixes #
Why we need it and why it was done in this way
The misclassifications were already shipping wrong advice to users (e.g. telling them to top up balance when they just need to slow down requests, or to check their API key when their region is blocked). This is a correctness issue, not an enhancement.
The following tradeoffs were made:
rate_limit,quota) and slightly more rules. This raises per-call token usage modestly. Diagnosis runs on the CherryAI free model, so the cost impact is effectively zero, and the examples meaningfully steer the model away from the most common wrong answers.unknown, which still triggers the AI second layer, so user experience does not degrade.The following alternatives were considered:
rate_limitsplit and stopping there — rejected because the same root pattern (overly broad rules and lost context) caused several other misdiagnoses surfaced during the audit.Links to places where the discussion took place: N/A
Breaking changes
None. The change is purely a correction to error-diagnosis logic. No public APIs, data schemas, or persistence formats change. Persisted diagnosis results in
block.metadatacontinue to render unchanged.Special notes for your reviewer
regioncategory navigates to/settings/general(proxy config) rather than/settings/provider, because geo-block fixes typically require a proxy, not a new API key.causefield onSerializedAiSdkErroris intentionally read as a string (it is serialized byserializeErrorinsrc/renderer/src/utils/error.ts). The newresponseBody/data/finishReasonextraction relies on the top-level fields already flattened by the same serializer; no nested status-code extraction is needed.Checklist
/gh-pr-review,gh pr diff, or GitHub UI) before requesting review from othersRelease note
Related issues