Skip to content

bug(session): finish_reason 'unknown' triggers an assistant-prefill continuation; SAP AI Core (and other prefill-forbidding providers) 400 after every successful turn #46415

Closed as not planned

Description

@ton618to

bug(session): finish_reason: unknown triggers an assistant-prefill continuation, so providers that forbid prefill (SAP AI Core) return 400 after every successful turn

Summary

With SAP AI Core (sap-ai-core, @jerome-benoit/sap-ai-provider-v2, e.g. anthropic--claude-4.8-opus), every turn renders the assistant response correctly and is then immediately followed by an extra error:

400 - LLM Module: This model does not support assistant message prefill. The conversation must end with a user message.
Error location: LLM Module

Root cause: the provider's finish reason normalizes to "unknown", which the run loop treats as "turn incomplete, keep going". The loop then re-issues a model request whose message array ends with the assistant message (an assistant prefill / continuation). SAP's gateway rejects prefill, so a spurious 400 appears after each otherwise-successful turn.

Evidence

Querying the local session DB for recent sap-ai-core assistant messages, finish is "unknown" on every completed turn:

finish histogram: { "unknown": 10, "None": 2 }   // "None" == the earlier auth-error turns
model: anthropic--claude-4.8-opus, finish: "unknown", error: false, completed: true   // x10

Server log confirms SAP runs through the AI-SDK path:

llm.native_unsupported_reason="provider is not openai, opencode, or anthropic"

Root cause (code path)

  1. Finish-reason enum is narrow. packages/llm/src/schema/ids.ts:39

    export const FinishReason = Schema.Literals(["stop", "length", "tool-calls", "content-filter", "error", "unknown"])

    Note it does not include the AI SDK's "other".

  2. Unrecognized reasons collapse to "unknown". packages/opencode/src/session/llm/ai-sdk.ts:22-24

    function finishReason(value: string | undefined): FinishReason {
      return Schema.is(FinishReason)(value) ? value : "unknown"
    }

    @jerome-benoit/sap-ai-provider-v2 reports a finish reason that is not one of the six literals (AI SDK "other" or a raw Claude stop reason), so opencode records finish = "unknown".

  3. The run loop keeps looping on "unknown". packages/opencode/src/session/prompt.ts:1111-1116

    if (
      lastAssistant?.finish &&
      !["tool-calls", "unknown"].includes(lastAssistant.finish) &&
      !hasToolCalls &&
      lastAssistant.parentID === lastUser.id
    ) {
      // exit loop
      break
    }

    Because "unknown" is excluded from the exit condition, a completed turn (text produced, no tool calls) does not exit; step++ and the model is called again.

  4. The continuation request is an assistant prefill. With no new user message and no tool results appended, the message array sent on the next iteration ends with the assistant message. Native Anthropic/OpenAI accept this (prefill/continuation), but the SAP AI Core gateway forbids it and returns 400 ... must end with a user message.

Reproduction

  1. Connect sap-ai-core and select anthropic--claude-4.8-opus.
  2. Send any prompt (e.g. "what is 1+1").
  3. The answer streams correctly, then an extra 400 ... does not support assistant message prefill error appears.
  4. Inspect the assistant message: finish === "unknown".

Suggested fixes

  • opencode (robust): do not issue a prefill continuation when finish === "unknown" but the assistant already produced content and there are no pending/unexecuted tool calls — treat it as terminal. Alternatively, gate prefill continuation on a provider capability flag, since many gateways forbid trailing-assistant requests.
  • Finish-reason mapping: map the AI SDK's "other" explicitly (and, for sap-ai-provider-v2, map Claude's end_turn/stop_sequence to "stop") so a clean turn exits the loop instead of collapsing to "unknown".
  • Provider package: @jerome-benoit/sap-ai-provider-v2 should emit a standard "stop" finish reason for completed Claude turns.

Environment

  • opencode dev branch (package version 1.18.23)
  • macOS (Apple Silicon), Bun runtime
  • Provider sap-ai-core via @jerome-benoit/sap-ai-provider-v2, model anthropic--claude-4.8-opus

Related

Activity

  1. github-actions commented on Aug 31, 2026

    @github-actions
    Contributor

    This issue might be a duplicate of existing issues. Please check:

  2. lgarceau768 commented on Sep 2, 2026

    @lgarceau768

    Same root cause on a second provider, and this time with the recorded finish value rather than a histogram.

    opencode 1.18.26, amazon-bedrock/anthropic.claude-opus-5 — so not gateway-specific, and not a provider that misreports its finish reason.

    Last three messages of the session, out of opencode.db:

    role finish
    assistant tool-calls
    assistant unknown
    assistant (APIError)

    The error on that last one:

    This model does not support assistant message prefill.
    The conversation must end with a user message.
    

    The unknown message matches your suggested predicate exactly: time.completed set, a 1192-character text part, and no tool part — its parts are step-start, reasoning, text, step-finish. Content produced, nothing pending, and the loop continued into a trailing-assistant request anyway.

    ["tool-calls","unknown"].includes(...) is still the exit condition in 1.18.26, at three call sites.

    The user-visible symptom is different enough from yours to be worth noting: a long reply appears to stop mid-sentence, then the 400 lands. Nothing is retryable in-session.

    One thing for prioritisation: on Bedrock there is no model choice that avoids this. Assistant prefill 400s across the current Claude family, so every Anthropic model in that catalogue hits the same path.

  3. amelvil2-ford commented on Sep 3, 2026

    @amelvil2-ford

    I haven't used this provider, but does this fix help you? #47079

    Let me know how it goes

  4. ton618to commented on Sep 5, 2026

    @ton618to
    Author

    Thanks for the pointer @amelvil2-ford.

    I don't think #47079 resolves this specific case, and for SAP AI Core / Bedrock it would trade the 400 for a per-turn hard error.

    The difference is subtle:

    For sap-ai-core and amazon-bedrock Anthropic models, "unknown" is the value on every successful turn (see the DB evidence above and @lgarceau768's Bedrock confirmation). #47079's default path treats anything outside stop | tool-calls | length | error | content-filter as ok: false and throws unrecognized finish reason "unknown". So out of the box it would turn every completed SAP/Bedrock turn into an error, and you'd need a finish.chunk plugin that flips ok = true for "unknown" just to get back to a working state. The finish.chunk hook itself is a nice addition, but as a default it swaps one per-turn failure for another on these providers.

    The proximate cause is the loop-exit predicate !["tool-calls", "unknown"].includes(finish) (still present at the three call sites @lgarceau768 noted). The narrow fix that avoids both the prefill 400 and a spurious error is to treat an "unknown" finish as terminal when the turn already produced text and has no pending tool calls, while keeping the existing continue-on-"unknown" behavior for empty/truncated turns (so #45315 stays covered). Concretely:

    const hasText =
      lastAssistantMsg?.parts.some((part) => part.type === "text" && part.text.trim().length > 0) ?? false
    
    if (
      lastAssistant?.finish &&
      lastAssistant.finish !== "tool-calls" &&
      (lastAssistant.finish !== "unknown" || hasText) &&
      !hasToolCalls &&
      lastAssistant.parentID === lastUser.id
    ) {
      // exit loop
    }

    I've been running exactly this as a local patch against 1.18.23 and it cleanly stops the prefill 400 on SAP anthropic--claude-4.8-opus with no error line and no regression in the prompt-loop tests. Mapping the provider's raw finish (end_turn/stop → stop) would be the more complete fix, but the predicate above is a safe minimal guard in the meantime.

  5. Dawncoral commented on Oct 28, 2026

    @Dawncoral

    Confirming the prefill path on 1.18.35, plus a version boundary.

    Harness: a local OpenAI-compatible mock that returns finish_reason: "unknown" and rejects any request whose last message is assistant, with the prefill 400.

    1.18.35 request sequence:

    system, user, user        (title call)
    system, user              -> 200, finish_reason "unknown"
    system, user, assistant   -> 400 -> CLI prints the prefill error
    

    On 2.2.0 the same harness no longer produces a trailing assistant message: the client appends a synthetic user turn ("The previous response was interrupted...") and keeps auto-continuing (12 requests in ~90s here). That removes the 400 but the loop remains; #56295 covers the unmarked synthetic message.

    The steps path (#56250) has an open fix in #56257. Happy to share the ~100-line Python mock if it helps.

  6. wayzha commented on Oct 28, 2026

    @wayzha

    v2.2.0: prefill loop fixed ✅, but Schema validation failed still fires on every SAP AI Core turn
    The prefill continuation loop described in this issue no longer occurs on v2.2.0 — the model responds correctly and no spurious 400 is produced. However, every successful SAP AI Core response still triggers:
    level=ERROR message="Failed to drain Session"
    cause="Error: Schema validation failed"
    [cause]: LLM.Event.StepFinish → reason → normalized: AnyOf failed
    The underlying root cause identified here remains unfixed: FinishReason in packages/llm/src/schema/ids.ts:39 does not include "other", and the adapter in packages/opencode/src/session/llm/ai-sdk.ts:22-24 maps it to "unknown". While the run loop now handles "unknown" without looping, the schema validation on StepFinish still rejects it because the normalized value fails the strict 6-literal union check.
    The fix should either add "other" to the FinishReason schema, or ensure the mapped "unknown" value passes through schema validation during drain.
    Environment: v2.2.0 macOS, aisdk:@jerome-benoit/sap-ai-provider-v2@4.4.11, tested with sap-ai-core/gpt-6-astra and other SAP AI Core models.

  7. opencodejr commented on Oct 28, 2026

    @opencodejr

    Thanks for the detailed root-cause analysis!

    The version you're running (1.18.23) is OpenCode v1, and the code paths you traced (session/prompt.ts, session/llm/ai-sdk.ts) are v1 code. v1 now only gets fixes for critical bugs such as data loss, security problems, or breakage with no workaround. Here the answer is still delivered and the extra 400 comes after it, so this is unlikely to be fixed in v1.

    We'd encourage you to try OpenCode v2 (@opencode/cli, the opencode command). It's a separate implementation, and it doesn't send a prefill continuation when the finish reason is unknown. It treats that case as a provider error instead, so you shouldn't get this exact 400 there. We haven't tested that with SAP AI Core, though. Related v2 issues you may want to follow:

    If you think this is critical, or you see the problem on v2, please reply here with your opencode --version and what you see.

  8. wayzha commented on Oct 29, 2026

    @wayzha

    Thanks for the detailed root-cause analysis!

    The version you're running (1.18.23) is OpenCode v1, and the code paths you traced (session/prompt.ts, session/llm/ai-sdk.ts) are v1 code. v1 now only gets fixes for critical bugs such as data loss, security problems, or breakage with no workaround. Here the answer is still delivered and the extra 400 comes after it, so this is unlikely to be fixed in v1.

    We'd encourage you to try OpenCode v2 (@opencode/cli, the opencode command). It's a separate implementation, and it doesn't send a prefill continuation when the finish reason is unknown. It treats that case as a provider error instead, so you shouldn't get this exact 400 there. We haven't tested that with SAP AI Core, though. Related v2 issues you may want to follow:

    If you think this is critical, or you see the problem on v2, please reply here with your opencode --version and what you see.

    I'm already on v2.2.0 (opencode --version → v2.2.0). The prefill 400 is indeed gone ✅.

    However, every SAP AI Core response still triggers Schema validation failed during SessionRunner.drain — the LLM.Event.StepFinish.reason.normalized field fails validation because the AI SDK's "other" finish reason is not in v2's FinishReason enum either. See my earlier comment above for the full error.

    This is non-blocking (the response does render), but it fires on every single turn with any aisdk: provider that returns finishReason: "other".

  9. opencodejr commented on Oct 29, 2026

    @opencodejr

    Closing this as not planned. It was reported on v1 (1.18.23), where only critical bugs get fixed. The extra 400 after each turn is annoying, but the answer is still delivered, so it doesn't meet that bar. We waited a day for a reply in case you saw this on v2 or thought it was critical, and haven't heard back.

    OpenCode v2 (@opencode/cli) doesn't send a prefill continuation on an unknown finish reason. If you see the same 400 with SAP AI Core on v2, please open a new issue with your opencode --version and we'll look into it. Thanks for the detailed analysis!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions