Skip to content

Genuine 429 rate limits whose message contains "limit exhausted" are misclassified as non-retryable QuotaExceeded, permanently halting agent runs #59752

Open

Description

@Lolleka

Description

Summary

A provider's genuine, transient 429 rate-limit response — one that sets retry-after and x-ratelimit-* correctly — is misclassified as QuotaExceededError instead of RateLimitError when the error message contains the substring "limit exhausted". Because QuotaExceededError is non-retryable (isRetryable → false), the client aborts instead of honoring retry-after and waiting out the cooldown — permanently halting agent runs under concurrency.

This is the false-positive counterpart of the same classifier issue discussed in #32120 (which routes real quota 429s into the non-retryable bucket to avoid burning user quota windows). The message-text heuristic meant to catch subscription/quota 429s is collateral-damaging legitimate rate limits that happen to share a phrase.

Evidence

Provider (gwdg SAIA, OpenAI-compatible) returns a fully spec-compliant 429, with correct status, type, retry-after, and x-ratelimit-*:

HTTP/2 429
retry-after: 4
x-ratelimit-limit-requests-10s: 3
x-ratelimit-remaining-requests-10s: 0
x-ratelimit-reset-requests-10s: 14

{"error":{"message":"User 10s request rate limit exhausted (3 of 3).",
            "type":"rate_limit_exceeded","param":null,"code":null}}

Root cause

In provider-error.ts, the 429 disambiguation checks QUOTA_TEXT before the rate-limit branch:

// ~line 169
const QUOTA_TEXT =
  /insufficient[-_\s]?(?:quota|balance)|quota[-_\s]?exceeded|budget exceeded|usage limit|limit exhausted|package has expired|plan does not yet include/i

// ~lines 262-266 — evaluated FIRST
if (
  input.status === 402 ||
  codes.some((code) => QUOTA_CODES.has(code)) ||
  (input.status === 429 && QUOTA_TEXT.test(text))   // 'limit exhausted' matches here
)
  return new QuotaExceededError(details)            // non-retryable

// ~lines 272-281 — evaluated SECOND, never reached for this 429
if (
  input.status === 429 ||                           // a 429 alone SHOULD hit this
  codes.some((code) => code.includes("rate_limit") || /* ... */) ||
  RATE_LIMIT_TEXT.test(text)
)
  return new RateLimitError({ ... retryAfterMs ... })  // retryable, honors retry-after

QUOTA_TEXT's alternative limit exhausted (intended for billing/quota messages) catches the generic rate-limit message "request rate limit exhausted". Then isRetryable() (model-retry.ts, case "QuotaExceeded": return false) aborts instead of honoring retry-after.

The regression is triggered purely by incidental provider message wording: this provider previously worded it "rate limit exceeded" (→ RATE_LIMIT_TEXT, correctly RateLimit); a server-side wording change to "rate limit exhausted" now trips QUOTA_TEXT. Client behavior should not depend on free-text phrasing.

Why it matters (vs. #32120)

#32120 argues real quota-429s shouldn't be retried. But that design relies on a message-text heuristic to distinguish quota from rate limit, and a legitimate rate limit with retry-after set has no business in the quota bucket. Every authoritative source keys rate-limit vs quota off status/code/type/retry-after, not message wording. OpenAI's own docs: "Check the error message and error.code". This response even sets type: "rate_limit_exceeded", which the classifier ignores in favor of the text.

Suggested fix

  1. Respect retry-after/retry-after-ms: if a 429 sets retry-after (or carries x-ratelimit-*, or a structured code/type indicating rate limit), classify as RateLimit. A quota exhaustion never advertises a retry-after. Only fall through to QuotaExceeded for 429s that are clearly quota (a quota-specific code, or billing phrasing without retry-after).
  2. Tighten QUOTA_TEXT: limit exhausted is too broad. Bind it to a quota context ("monthly limit", "usage limit", "plan", "billing", "credit") or drop it and rely on QUOTA_CODES + billing-specific phrases.
  3. Prefer the structured error.type/error.code fields over message text where they're authoritative.

Plugins

OMO-Slim

OpenCode version

v2.0.32

Steps to reproduce

Interpose the headers/body above on any 429 (or point a scratch provider at a server returning it), then:

opencode run -m <model> "hi"   # → Error, no retry

Deterministic on classification; confirm via opencode run headless (a proxy returning exactly the above 429 reproduces it).

Screenshot

No response

Operating System

No response

Terminal

No response

Activity

  1. opencodejr commented on Oct 27, 2026

    @opencodejr

    Thanks for the detailed report. I reproduced this on 2.0.32, and it still happens on the latest dev (fd8268ea5).

    I pointed a custom @ai-sdk/openai-compatible provider at a mock server that returns your 429 (type: "rate_limit_exceeded", retry-after and x-ratelimit-* headers), then ran opencode run -m mock/m "hi":

    • With the message User 10s request rate limit exhausted (3 of 3)., the run fails at once with that error and doesn't wait for retry-after.
    • With the same response worded rate limit exceeded, OpenCode retries with backoff as expected.

    This matches your analysis: in packages/ai/src/provider-error.ts, the limit exhausted alternative in QUOTA_TEXT is checked before the rate-limit branch, so the error is classified as a non-retryable quota error. The script below reproduces both cases.

    Related: #47685 (quota/402 classification relying on message text, the opposite direction).

    Reproduction script (repro.sh)
    #!/usr/bin/env bash
    # Reproduces #59752: a transient 429 whose message says "rate limit exhausted"
    # is classified as QuotaExceeded and never retried, while the same 429 with
    # "rate limit exceeded" is retried honoring retry-after.
    # Usage: ./repro.sh [opencode command...]   (defaults to `opencode`)
    # From source: BUN_OPTIONS="--conditions=browser --preload=<repo>/packages/tui/node_modules/@opentui/solid/scripts/preload.js" \
    #   ./repro.sh bun <repo>/packages/cli/src/index.ts
    set -u
    OC=("${@:-opencode}")
    DIR=$(mktemp -d); cd "$DIR"; git init -q
    cat > server.ts <<'TS'
    let hits = 0
    const word = process.env.WORD!
    Bun.serve({ port: Number(process.env.PORT), fetch() {
      hits++; console.error(`[mock] request #${hits} at ${new Date().toISOString()}`)
      return new Response(JSON.stringify({ error: { message: `User 10s request rate limit ${word} (3 of 3).`, type: "rate_limit_exceeded", param: null, code: null } }), {
        status: 429,
        headers: { "content-type": "application/json", "retry-after": "1",
          "x-ratelimit-limit-requests-10s": "3", "x-ratelimit-remaining-requests-10s": "0", "x-ratelimit-reset-requests-10s": "1" } })
    } })
    TS
    run() {
      local word=$1 port=$2
      cat > opencode.json <<JSON
    { "\$schema": "https://opencode.ai/config.json",
      "provider": { "mock": { "npm": "@ai-sdk/openai-compatible", "name": "Mock",
        "options": { "baseURL": "http://127.0.0.1:$port/v1", "apiKey": "x" },
        "models": { "m": { "name": "m" } } } } }
    JSON
      WORD=$word PORT=$port bun server.ts 2> "mock-$word.log" & local pid=$!
      sleep 1
      echo "===== message wording: 'rate limit $word' ====="
      timeout 12 "${OC[@]}" run -m mock/m "hi" 2>&1 | tail -n 15
      echo "exit: ${PIPESTATUS[0]} (124 = still retrying when killed after 12s)"
      kill $pid; cat "mock-$word.log"; echo
    }
    run exhausted 18431
    run exceeded 18432

    Run it: bash repro.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions