Add embedded MCP server for in-process UI introspection

[tilladam]

Summary

• Add an in-process MCP (Model Context Protocol) server to the Slint testing backend, allowing MCP clients (e.g. coding assistants like Claude) to inspect and interact with a running Slint application's UI directly — no separate bridge process needed
• When the mcp feature is enabled and SLINT_MCP_PORT=<port> is set at runtime, an HTTP server starts on 127.0.0.1:<port> implementing the MCP Streamable HTTP transport with 11 tools: list_windows, get_window_properties, find_elements_by_id, get_element_properties, query_element_descendants, get_element_tree, take_screenshot, click_element, invoke_accessibility_action, set_element_value, dispatch_key_event
• Refactors the existing systest module to share an IntrospectionState with the new MCP transport, avoiding code duplication for window tracking, element handle management, property extraction, and query execution

Architecture

slint/mcp → selector/mcp → testing/mcp
                                ├── introspection.rs  (shared state: window/element arenas, properties, queries)
                                ├── mcp_server.rs     (HTTP server, JSON-RPC 2.0, 11 MCP tools)
                                └── systest.rs        (refactored to use shared IntrospectionState)

Key design decisions

• Feature-gated: mcp feature is fully opt-in, zero cost when disabled
• Localhost-only: Binds to 127.0.0.1, validates Origin header against localhost addresses for DNS rebinding protection
• Arena eviction: FIFO eviction with 10k handle cap prevents unbounded memory growth; root element handles are never evicted; circuit breaker prevents infinite loops
• HTTP/1.1 keep-alive: Carry-over buffer handles pipelined requests; Connection: close respected
• Serde for parameters: Typed Deserialize structs for all tool parameters with proper error messages
• 63 unit tests covering JSON-RPC protocol, parameter deserialization, tool error paths, origin validation, eviction logic, and schema validation

Test plan

• cargo test -p i-slint-backend-testing --features mcp --lib — 63 tests pass
• cargo check -p i-slint-backend-testing --features system-testing — systest refactor compiles
• Manual integration test against gallery example with live MCP client (14 interaction scenarios verified)
• CI with --features mcp enabled

[Sewer56]

I didn't think someone would do it 😅
I've actually been interested in such a thing myself; but my backlog's been so big I haven't even gotten around to using Slint proper.

[tronical]

I've been thinking about this back and forth and my current thinking is this:

I think the "inspector" side of this should live entirely inside Slint. I'm not sure this has to be MCP as protocol, but fundamentally I think we want the ability to set certain feature flags / etc. to activate an API that permits inspecting the element tree, take screenshots, etc. . We've got this mode of the testing backend connecting to a given address if an environment variable is set, but I think for this agent interaction, we want the agent to specify a port and expect the application to listen at that given port. If the port can't be listened on, the process should abort in a way that tells the agent that it should try again with a different port. There's still a race though if two agents try to launch an app with the same port, one might end up talking to the wrong one. So a cookie might be worth adding to the protocol here - just to catch the error.

I think the agent side should be the result of the llm learning about how to connect to the inspector via SKILL.md file. We can submit skills to the various market places for Claude, etc. It can even be versioned: "Minimum Slint version is 1.16, in version 1.19 this and that changed, etc.).

[Sewer56]

I'm not sure this has to be MCP as protocol, but fundamentally I think we want the ability to set certain feature flags / etc. to activate an API that permits inspecting the element tree, take screenshots, etc.

While the internals of Slint itself are foreign to me; the (general purpose) engineer says in me that you should just write the underlying code in a generic way. i.e. Have one part of the code implementing all of the functionality needed for debugging; and then just expose that using simple wrappers. Be it MCP; HTTP; Pipes, local sockets, or whatever IPC protocol you wish to use.

Under normal conventional means, you'd just have 1 protocol you expose, and other code can then link up and connect to that protocol. With MCP, I think it may be a bit trickier; because you want ideally a hassle free setup experience; so embedded is good; you could in theory just point your MCP-enabled client at a localhost URL.

Asking people to install docker for instance, or other software; which you would otherwise get, may be a bit more burdensome.

The difficult part will always be multiple instances at a time though.

[tronical]

(Looking at the PR now; somehow thought this was the previous one :)

[tronical]

From what I can tell this is awesome. Pretty much what I'm looking for. This way we can teach Claude etc quite easily how to introspect and interact.

I have just one concern, but it's a big one: The implementation is huge and duplicates a lot of functionality. Adding more properties, etc sounds like one requires touching a lot of code.

I'd like to explore if it's possible to move to a different protobuf implementation and then, instead of mcp, expose protobuf messages directly as json, along with a generated schema.

One issue here is that protobuf beyond the crate we're currently using requires a protoc binary, which sucks. But there is now https://crates.io/crates/protoc-bin-vendored , which might just be good enough to generate the code in build.rs.

I'd be happy to discuss this further - plan level.