Skip to content

v2 (opencode2): max_tokens never sent for @ai-sdk/openai-compatible models, limit.output ignored, thinking turns truncate at provider default #47398

Closed

Description

@TheFerragamo

Description

On the v2 CLI (opencode2, @opencode-ai/cli beta) requests to @ai-sdk/openai-compatible models go out without max_tokens, regardless of limit.output in the provider config. Stable opencode 1.18.20 with the same config sends max_tokens: 32000.

Because the field is missing, the upstream applies its own default budget. In my setup (DeepSeek V4 with thinking enabled, behind an OpenAI-compatible proxy) the effective cap was 4096 tokens: long-thinking turns spend the whole budget on reasoning and finish with finish_reason: length and empty content, at exactly 4096 output tokens. Same symptom as #46595 (Bedrock), different route.

Captured outbound body from the beta (sanitized, messages/tools omitted):

{
  "model": "combo-deepseek-v4-flash",
  "stream": true,
  "stream_options": { "include_usage": true },
  "store": false,
  "prompt_cache_key": "ses_…"
}

Same session on opencode/1.18.20:

{ "model": "combo-deepseek-v4-flash", "stream": true, "max_tokens": 32000, … }

Where it goes wrong, as far as I can tell:

  • packages/core/src/session/runner/llm.ts builds LLM.request({ model, http, providerOptions, system, messages, tools, toolChoice }) with no generation block.
  • packages/llm/src/protocols/openai-chat.ts fromRequest emits max_tokens: generation?.maxTokens, which is undefined, so the key is dropped.
  • packages/core/src/session/runner/model.ts (withDefaults) does put limit.output into route.defaults.limits, but nothing projects it into generation.maxTokens. v1 did this via ProviderTransform.maxOutputTokens (min(limit.output, 32000) || 32000).

The Anthropic route (anthropic-messages.ts) is built from the same request, so it is likely affected too, but I only verified the openai-compatible path.

Related: #46595 (same root cause on Bedrock), #29363 (v1 32K cap of limit.output).

Plugins

none

OpenCode version

@opencode-ai/cli 0.0.0-beta-19086 (opencode2). Not reproducible on opencode 1.18.20.

Steps to reproduce

  1. Config:
    "provider": {
      "my-proxy": {
        "npm": "@ai-sdk/openai-compatible",
        "options": { "baseURL": "https://…/v1", "apiKey": "…" },
        "models": {
          "combo-deepseek-v4-flash": { "limit": { "context": 1000000, "output": 131072 } }
        }
      }
    }
  2. Point the base URL at anything that logs request bodies (a local proxy is enough).
  3. Send a prompt from opencode2: the body has no max_tokens. Send the same prompt from opencode 1.18.20: max_tokens: 32000 is present.
  4. With a thinking model, long turns end at the provider's default output cap, reasoning only, no text.

Screenshot and/or share link

n/a (request bodies above)

Operating System

macOS 15.7.9

Terminal

n/a (not terminal-specific)

Activity

  1. clopca commented on Sep 5, 2026

    @clopca

    Opened #47532 for the shared code site (SessionModelRequest.prepare in packages/core/src/session/model-request.ts, which is where the runner's request gets its generation block). It projects resolved.limit.output into generation.maxTokens with hook → model defaults → route defaults → catalog limit precedence, so it covers the openai-compatible max_tokens path as well as Bedrock inferenceConfig.maxTokens. If you can verify it against DeepSeek V4 with thinking enabled, that would be a useful second data point.

  2. famewolf commented on Sep 24, 2026

    @famewolf

    Additional evidence from a live v2.0.12→v2.0.15 desktop installation (2026-09-23):

    Confirmed the output cap is completely dropped in both v2.0.12 and v2.0.15. With limit.output: 4096 in config, the v2 sidecar sends max_tokens: -1 (uncapped) to the backend.

    Observed responses from a local llama-server (Qwen3-27B with thinking enabled via --reasoning-format deepseek):

    • 1.65MB response body over 7m38s
    • 1.63MB over 4m9s
    • 586KB over 8m15s
    • Multiple 499 client aborts at ~5-6min (opencode client timeout)

    For comparison, v1 (1.18.31) with the same config sends max_tokens: 4096 and completes in seconds.

    The impact with a thinking model behind an OpenAI-compatible proxy is severe: the model spends the entire (uncapped) budget on reasoning tokens, producing multi-MB responses that take minutes, then the client aborts and retries in a loop. This is the root cause of the "hang" behavior — the model IS working, just with no output cap.

    Related: #51020 (part persistence wedge — a separate but compounding v2 bug where the responses are never persisted to DB).

  3. opencode-agent commented on Sep 27, 2026

    @opencode-agent
    Contributor

    Fix merged into v2 in #51271. It landed after v2.0.18 and will be included in the next V2 release, v2.0.19. Primary requests now send a context-fitted output-token limit to OpenAI-compatible providers, instead of omitting max_tokens and leaving the provider to use its own default.

  4. peterxh7 commented on Oct 26, 2026

    @peterxh7

    I have the same issue.
    OpenCode v2.1.4

  5. rekram1-node commented on Oct 27, 2026

    @rekram1-node
    Contributor

    @peterxh7 max tokens is always sent by default so either ur using a provider that doesnt support them or ur config is missing some settings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions