Repository navigation
[BUG] Post-processing silently falls back to thinking mode when the endpoint rejects reasoning_effort "none" #2362
Description
Trying to turn thinking off for post-processing, but the retry turns it back on.
Bug Description
On main (7bfb2e2), post-processing asks the endpoint to skip reasoning with reasoning_effort: "none" (src-tauri/src/llm_client.rs:79-84). When the endpoint answers 400/422 because it does not accept that value, the retry added in #1809 resends the same request with no reasoning field at all, and remembers the rejection so no reasoning field is ever sent to that endpoint again.
The issue is what "no reasoning field" means. DeepSeek's API reference documents thinking as an object with possible values enabled / disabled and a default of enabled, and reasoning_effort as none / low / high / max where none disables thinking:
- https://api-docs.deepseek.com/api/create-chat-completion
- https://api-docs.deepseek.com/guides/thinking_mode
So the successful retry does not mean reasoning is off, it means the endpoint falls back to its default, which is thinking on. Every dictation after the first one then runs in thinking mode. The rejection is cached in reasoning_rejections() for the life of the process (llm_client.rs:87-110) and is only logged at info!, so nothing in the app shows that post-processing is slower than intended.
Handy already knows a second documented spelling, thinking: {type: "disabled"}, but it only sends that when the base URL contains api.deepseek.com (llm_client.rs:62-68). Any OpenAI compatible gateway or proxy in front of DeepSeek takes the generic path and never gets it.
Measured by replaying Handy's request body (system prompt plus transcript as the user message, stream: false), 90 word dictation, median of 2 to 3 runs:
| endpoint | reasoning field sent | result |
|---|---|---|
| gateway | none, after the cached rejection | 200, thinking on, 3.55 s, 133 to 209 reasoning tokens |
| gateway | reasoning_effort: "none", what Handy sends |
400 Invalid option: expected one of "off"|"low"|"medium"|"high"|"xhigh"|"max" |
| gateway | reasoning_effort: "off", for reference |
200, thinking off, 3.12 s, 0 reasoning tokens |
| gateway | thinking: {type: "disabled"} |
200, field ignored, thinking on |
api.deepseek.com |
none | 200, thinking on, 1.60 s |
api.deepseek.com |
reasoning_effort: "none" |
200, thinking off, 1.16 s |
api.deepseek.com |
thinking: {type: "disabled"} |
200, thinking off, 1.12 s |
Two smaller things I noticed while measuring. Neither needs action in this report:
- The comment at
llm_client.rs:63says DeepSeek rejectsreasoning_effort: "none". That matched the error in Custom post-processing fails with DeepSeek because reasoning_effort:none is rejected #1342 in April, but it no longer matches the API reference or a live request today. - A gateway that passes parameters through is fine, since both fields work on the official endpoint. The failure only shows up when the gateway validates the enum.
Expected behavior
When a reasoning-disable field is rejected, try the other documented spelling before giving up on disabling reasoning, and remember which one worked so later dictations go straight to it. If none work, log at warn! rather than info! so it is visible that this endpoint will run with reasoning enabled.
Actual behavior
First dictation: one 400 plus one silent retry. Every later dictation: thinking mode, extra latency, nothing above info!, and no hint anywhere in the app.
Logs
INFO Endpoint rejected request with reasoning disabled (status 400): {"error":{"message":"Invalid option: expected one of \"off\"|\"low\"|\"medium\"|\"high\"|\"xhigh\"|\"max\""}}. Retrying without reasoning fields
INFO Retry without reasoning fields succeeded; '<base_url>' (model '<model>') will skip them from now on
System Information
App Version: main at 7bfb2e2. Same code path in v0.9.5 through v0.9.8, since #1809 landed in v0.9.5.
Operating System: Linux
CPU / GPU: n/a. I measured by replaying Handy's request body against the endpoints with a small script instead of going through the GUI, so this is not a GUI level repro. The request bodies and the code path are what I am reporting.
Activity