Skip to content

[BUG] Post-processing silently falls back to thinking mode when the endpoint rejects reasoning_effort "none" #2362

Open

Description

@HomerHm

Trying to turn thinking off for post-processing, but the retry turns it back on.

Bug Description

On main (7bfb2e2), post-processing asks the endpoint to skip reasoning with reasoning_effort: "none" (src-tauri/src/llm_client.rs:79-84). When the endpoint answers 400/422 because it does not accept that value, the retry added in #1809 resends the same request with no reasoning field at all, and remembers the rejection so no reasoning field is ever sent to that endpoint again.

The issue is what "no reasoning field" means. DeepSeek's API reference documents thinking as an object with possible values enabled / disabled and a default of enabled, and reasoning_effort as none / low / high / max where none disables thinking:

So the successful retry does not mean reasoning is off, it means the endpoint falls back to its default, which is thinking on. Every dictation after the first one then runs in thinking mode. The rejection is cached in reasoning_rejections() for the life of the process (llm_client.rs:87-110) and is only logged at info!, so nothing in the app shows that post-processing is slower than intended.

Handy already knows a second documented spelling, thinking: {type: "disabled"}, but it only sends that when the base URL contains api.deepseek.com (llm_client.rs:62-68). Any OpenAI compatible gateway or proxy in front of DeepSeek takes the generic path and never gets it.

Measured by replaying Handy's request body (system prompt plus transcript as the user message, stream: false), 90 word dictation, median of 2 to 3 runs:

endpoint reasoning field sent result
gateway none, after the cached rejection 200, thinking on, 3.55 s, 133 to 209 reasoning tokens
gateway reasoning_effort: "none", what Handy sends 400 Invalid option: expected one of "off"|"low"|"medium"|"high"|"xhigh"|"max"
gateway reasoning_effort: "off", for reference 200, thinking off, 3.12 s, 0 reasoning tokens
gateway thinking: {type: "disabled"} 200, field ignored, thinking on
api.deepseek.com none 200, thinking on, 1.60 s
api.deepseek.com reasoning_effort: "none" 200, thinking off, 1.16 s
api.deepseek.com thinking: {type: "disabled"} 200, thinking off, 1.12 s

Two smaller things I noticed while measuring. Neither needs action in this report:

Expected behavior

When a reasoning-disable field is rejected, try the other documented spelling before giving up on disabling reasoning, and remember which one worked so later dictations go straight to it. If none work, log at warn! rather than info! so it is visible that this endpoint will run with reasoning enabled.

Actual behavior

First dictation: one 400 plus one silent retry. Every later dictation: thinking mode, extra latency, nothing above info!, and no hint anywhere in the app.

Logs

INFO Endpoint rejected request with reasoning disabled (status 400): {"error":{"message":"Invalid option: expected one of \"off\"|\"low\"|\"medium\"|\"high\"|\"xhigh\"|\"max\""}}. Retrying without reasoning fields
INFO Retry without reasoning fields succeeded; '<base_url>' (model '<model>') will skip them from now on

System Information

App Version: main at 7bfb2e2. Same code path in v0.9.5 through v0.9.8, since #1809 landed in v0.9.5.

Operating System: Linux

CPU / GPU: n/a. I measured by replaying Handy's request body against the endpoints with a small script instead of going through the GUI, so this is not a GUI level repro. The request bodies and the code path are what I am reporting.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions