You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SEP authors testing not-yet-released spec changes hit two blockers when running the suite with protocolVersion: "draft":
initialize scenario rejects "draft" — src/checks/client.ts hard-codes VALID_PROTOCOL_VERSIONS = ['2025-06-18', '2025-11-25'], so a client sending "draft" fails the mcp-client-initialization check with "Version mismatch".
--spec-version draft only runs draft-tagged scenarios — the initialize scenario (and most of the core suite) is tagged ['2025-06-18', '2025-11-25'], so --spec-version draft skips it entirely. Tagging it [..., 'draft'] is blocked by the spec-version.test.ts isolation check.
Net effect: there's no way to run the full suite against an SDK that's tracking the draft spec without forking the conformance repo.
Raised by @mikekistler in #conformance-testing-wg (thread, Apr 10–21) while testing SEP-2243. He proposed a long-lived draft branch as a workaround; preference here is to make draft a first-class spec-version target instead.
Proposed fix
Add LATEST_SPEC_VERSION and DATED_SPEC_VERSIONS constants so the next spec release is a one-line change.
Treat "draft" as a valid protocol version in the initialize check and mock-server response (echoes back "draft" if the client sends it).
Make --spec-version draft resolve to latest-dated ∪ draft-tagged scenarios — i.e., draft is a superset of the current release plus draft-only additions. The spec-version.test.ts isolation check is updated to match (a scenario is tagged draft-only, but selected by --spec-version draft if it's in the latest release).
Pass the resolved spec version to the client process via MCP_CONFORMANCE_SPEC_VERSION so SDK examples can pick the right protocolVersion without hard-coding.
Out of scope
Migrating the ~30 hard-coded '2025-11-25' literals across server scenarios — separate cleanup.
Problem
SEP authors testing not-yet-released spec changes hit two blockers when running the suite with
protocolVersion: "draft":initializescenario rejects"draft"—src/checks/client.tshard-codesVALID_PROTOCOL_VERSIONS = ['2025-06-18', '2025-11-25'], so a client sending"draft"fails themcp-client-initializationcheck with "Version mismatch".--spec-version draftonly runs draft-tagged scenarios — theinitializescenario (and most of the core suite) is tagged['2025-06-18', '2025-11-25'], so--spec-version draftskips it entirely. Tagging it[..., 'draft']is blocked by thespec-version.test.tsisolation check.Net effect: there's no way to run the full suite against an SDK that's tracking the draft spec without forking the conformance repo.
Raised by @mikekistler in #conformance-testing-wg (thread, Apr 10–21) while testing SEP-2243. He proposed a long-lived
draftbranch as a workaround; preference here is to makedrafta first-class spec-version target instead.Proposed fix
LATEST_SPEC_VERSIONandDATED_SPEC_VERSIONSconstants so the next spec release is a one-line change."draft"as a valid protocol version in theinitializecheck and mock-server response (echoes back"draft"if the client sends it).--spec-version draftresolve to latest-dated ∪ draft-tagged scenarios — i.e., draft is a superset of the current release plus draft-only additions. Thespec-version.test.tsisolation check is updated to match (a scenario is tagged draft-only, but selected by--spec-version draftif it's in the latest release).MCP_CONFORMANCE_SPEC_VERSIONso SDK examples can pick the rightprotocolVersionwithout hard-coding.Out of scope
'2025-11-25'literals across server scenarios — separate cleanup.experimental/folder proposal (Proposal: Tasks conformance scenarios (spec 2025-11-25) #244 comment thread) — that's about where scenarios live, this is about which version they target.