[shahar1]

Summary

• JevProvider could never get an answer from the Jev API. Every choice, score and noul call returned HTTP 400, because it sent operation, prompt, options and scale to POST /v1/systemone, which requires model, state and questions. It also read label, value and probability from the top level of the response, while the API returns them under answers.
• The pinned model systemone-2026-06-01 is not a model the API documents. It is now jev-1.14.0.
• The failure stayed hidden on both sides. The tests mocked responses in the same invented shape, and the contract is fail-open, so the pr-triage shadow pre-filter would only have logged fell_through.

The public choice, score and noul contract does not change.

Type of change

• Skill change (.claude/skills/<name>/) — eval fixtures updated below
• Tool / bridge contract (tools/<system>/*.md)
• Python package (tools/*/ with pyproject.toml)
• Groovy reference impl
• Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
• Documentation (docs/, README.md, CONTRIBUTING.md)
• Project template (projects/_template/)
• CI / dev loop (prek, workflows, validators)
• Other:

Test plan

• prek run --all-files passes, through the pre-commit hook on this commit
• For Python packages touched: uv run pytest / ruff check / mypy passes for tools/typed-decision, and test_typed_decision_prefilter.py passes
• For Groovy bridges touched: command-line invocation tested end-to-end
• For skill changes: eval suite passes for the affected skill
• For skill behaviour changes: a new or updated eval fixture is included in this PR
• Other: live calls against api.typesafe.ai, using made-up text only

The mocks now follow the published OpenAPI schema, so I also ran the provider against the real API. enforce_privacy_gate was patched out in a throwaway script for these calls, because I have no approved privacy-llm.md entry. With the real gate left in place, the same call was still denied with no request sent.

Call	main	This PR
choice on a crash report, options bug/feature/question	HTTP 400	bug, confidence 1.0
noul "Is the sky usually blue on a clear day?"	HTTP 400	0.95
noul "Is the sun made of cheese?"	HTTP 400	0.02
score full production outage, scale (1, 5)	HTTP 400	4.71, confidence 0.74
score typo in a comment, scale 10	HTTP 400	0.81, confidence 0.67

typed_decision_prefilter.py also ran end to end on a synthetic PR, enabled through MAGPIE_ENABLE_TYPED_DECISION_PREFILTER, with the gate patched the same way. It logged outcome: high_confidence, predicted_label: passing, confidence: 0.98.

The updated tests fail against the old provider.

RFC-AI-0004 compliance

• HITL — any new mutation is gated on explicit user confirmation
• Sandbox — no new unrestricted host access; network reach declared in the adapter
• Vendor neutrality — placeholders (<PROJECT>, <tracker>, <upstream>, <security-list>) used in all skill / tool prose (the check-placeholders prek hook is the mechanical gate)
• Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
• Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
• Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed. The gate call and its deny-by-default behaviour are unchanged.

Linked issues

Follows #1402 and #1403.

Notes for reviewers (optional)

• score mapping: the API rates against ordered level descriptions, at most 10, and returns a position starting at zero. The contract takes a numeric scale, so the provider sends numeric levels and maps the position back. Scales wider than 9 steps, such as (0, 10), or with fractional bounds use 10 evenly spaced levels, so the labels are not all whole numbers.
• Prompt placement: the whole prompt goes in state, with short generic instructions, so fenced contributor text stays data.
• Removed behaviour: the result and decision envelope handling is gone, because the API does not return those envelopes.

Not in this PR:

• typed-decision-prefilter.md calls the script at <framework>/skills/pr-management-triage/scripts/. The skills/ symlink exists in a framework checkout, but the one marketplace <framework> copy I checked held only tools/, and it predates fix(marketplace): ship every family's framework tools, not just security's #1937. Other skills use the same <framework>/skills/... form, so this looks like a separate fix.
• typed_decision_confidence_threshold defaults to 0.85, which was never measured against real Jev answers. This fix makes that measurement possible.

Generated-by: Claude Code (Opus 5.5), reviewed by shahar1 before submission

[onlyarnav]

LGTM! thanks for catching and fixing the mismatch with the live Typespace System One API.
I didnt cross-check the system against an actual JEV API, while finalizing, which is where I missed it. Really appreciate your help!

[onlyarnav]

the docs example shows criteria for choice as a list of labels, but this sends a dict {option: option}. can you confirm the dict form is documented?

[onlyarnav]

i suggest adding one sentence here: score criteria are bare numbers, so callers must define what high and low mean in the prompt.

[onlyarnav]

minor suggestion: rename to test_choice_missing_choice_in_response(...) since the field is now choice