[FEATURE]: Auto-switch to vision model when images enter the agent loop (incl. tool-produced) #46794

• I have searched for existing issues and confirmed this is not a duplicate
• I have read the contributing guidelines

Problem

Many teams pair a strong text-only coding model (which cannot read images) with a companion vision model. Currently there is no automatic way to use the vision model exactly when an image needs to be understood, and the text model otherwise.

The painful case is autonomous work inside the agent loop: tools such as Playwright (screenshots), Figma, or the filesystem read tool return image attachments mid-task. The active model stays text-only for the rest of that turn and simply cannot see the image — forcing the user to manually swap the model, which (per #46345) then leaves the images unusable anyway.

Expected behavior

Automatically switch the active model to a configured vision model whenever an image enters the conversation context, and switch back to the primary text model when no images remain:

1. Image attached in a new user message → switch to vision before the first provider call.
2. Image produced by a tool (Playwright screenshot, Figma export, filesystem read of an image) mid-loop → switch to vision before the next provider turn.
3. When no images are present in the current turn → switch back to the text model.

The key technical gap (why existing issues don't cover this)

In the classic agent loop (packages/opencode/src/session/prompt.ts, runLoop), the model is pinned to the last user message and re-resolved on every iteration:

const model = yield* getModel(lastUser.model.providerID, lastUser.model.modelID, sessionID)

switchModel() / POST /api/session/{sessionID}/model only updates session.model. Because lastUser.model stays unchanged, the in-flight loop keeps the original text-only model for the remaining turns of that user message. So a screenshot produced by a tool in turn N is still consumed by the text-only model in turn N+1, regardless of any model switch.

The newer runner (core/session/runner/llm.ts) already resolves the model fresh per turn via models.resolve(session) — which would make an automatic switch apply immediately. This feature would benefit from making model resolution per provider turn in the stable path, matching that behavior.

Proposed API

A small config option, mirroring the shape requested in #31936:

{
  "model": "<text-only-model>",   // primary text model
  "vision_model": "<vision-model>" // used when images are in context
}

Optionally also expose a plugin hook (e.g. a tool.result / turn event carrying whether the context now contains an image) so plugins can drive the switch programmatically and detect the specific "image to be sent next turn" condition.

Detection

Reuse the existing modalities/capabilities.input.image field on models to decide whether the target model supports vision, and match image FileParts by mime starting with image/ (including ToolPart attachments).

Related issues (for maintainer triage — not duplicates)

• [Bug]: switching from a text-only model to a vision model leaves images unusable #46345 — Bug: switching text-only → vision leaves images unusable. This is the failure the auto-switch must avoid; fixing [Bug]: switching from a text-only model to a vision model leaves images unusable #46345 is complementary.
• Image input is rejected after switching from a non-vision model within the same session #33199 / PR fix(session): sync model switch to server immediately #33471 — Fixed UI-driven model switch server sync, but not mid-loop continuation.
• feat: vision sub-model fallback for non-multimodal models #31936, [FEATURE]: Auto image-to-text transcription via multimodal model for non-multimodal providers #22828, [FEATURE]: Auto-describe images via vision fallback when active model lacks vision support #24948 — Propose a "vision sub-model transcription fallback" (image → text, continue on main model). This request differs: it switches the model/persona of the whole session to the vision model, rather than transcribing into text.
• [FEATURE] Multi-model orchestration: intent-based routing, vision delegation, turn-level advisor #34370 — Broader multi-model orchestration (intent routing, advisor). This is the focused slice: automatic vision switching for image-bearing turns.