Skip to content

CLI --load: multi-connection mode (--load-connections) to test many sessions, not one #2746

Closed

Description

@cliffhall

Follow-up to #2439 / #2743.

Problem

--load runs every call over the single connection the CLI already opened, so it measures concurrency within one session (documented in the CLI README). It cannot show how a server behaves with many clients at once: per-session state, session creation cost, connection limits, or contention that only shows up across sessions. For stdio this means one server process. For Streamable HTTP it means one Mcp-Session-Id.

Proposal

An opt-in --load-connections <n> (default 1, so today's behavior is unchanged). It opens n independent InspectorClient connections using the same resolved transport, headers and OAuth, spreads the workers across them, and reports one combined summary. It could optionally add a per-connection breakdown, or report connect failures separately from call errors.

Things to settle:

  • stdio: n connections means n spawned server processes. That is a different test from HTTP; decide whether to allow it or reject it for stdio.
  • OAuth: every connection reuses the stored token, so no extra prompts. Confirm that token refresh under concurrency doesn't race.
  • How --load-concurrency relates to it: total across all connections, or per connection.
  • Setup and teardown cost should stay out of the latency figures.

The engine (core/cli/handlers/load-test.ts, runLoad) already takes a call function, so the change is mostly in runToolLoad.

Priority score

Severity 3 (a real capability is missing; the workaround of running several CLI processes and merging their summaries by hand is partial) + urgency 3 (wanted this milestone) + 1 (milestoned) = 7 → Medium.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature requestv2Issues and PRs for v2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions