Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,9 @@ jobs:
- run: npm run test:dependency-isolation

e2e:
runs-on: ubuntu-latest
# Pinned: the Playwright version in package.json cannot install its
# browser on ubuntu-26.04, which ubuntu-latest now resolves to.
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6

Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/update-snapshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ jobs:
(github.event.issue.pull_request &&
contains(github.event.comment.body, '/update-snapshots') &&
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association))
runs-on: ubuntu-latest
# Pinned to the same image as the e2e job in ci.yml
runs-on: ubuntu-24.04
steps:
- name: Get PR branch
if: github.event_name == 'issue_comment'
Expand Down
3 changes: 3 additions & 0 deletions docs/migrate_from_openai_apps.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,9 @@ Client-side migration involves replacing the implicit `window.openai` global wit
| -------------------------------------------------- | --------------------------------------------------------- | ----------------------------------- |
| `await window.openai.requestDisplayMode({ mode })` | `await app.requestDisplayMode({ mode })` | Same API |
| — | Check `app.getHostContext()?.availableDisplayModes` first | MCP lets you check what's available |
| `_meta["openai/ui"].preferredDisplayMode` | `preferredDisplayMode` in the `App` capabilities | A hint; see below |

`_meta["openai/ui"].preferredDisplayMode` comes from OpenAI's [MCP extensions](https://github.com/openai/mcp-extensions/blob/main/docs/spec.md#display-modes), where it is set on the UI resource. In MCP Apps the View declares the preference itself when it connects (`new App(appInfo, { availableDisplayModes, preferredDisplayMode })`), and the host may ignore it.

### Size Reporting

Expand Down
19 changes: 19 additions & 0 deletions docs/patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,25 @@ In fullscreen mode, remove the container's border radius so content extends to t
}
```

If your View is mainly useful on a large surface, declare `fullscreen` as its preferred display mode. This is a hint for how the host first shows the View. The host may ignore it, or apply it just after the View is first shown, so keep the `onhostcontextchanged` handler above:

<!-- prettier-ignore -->
```ts source="../src/app.examples.ts#App_constructor_preferredDisplayMode"
const app = new App(
{ name: "MyApp", version: "1.0.0" },
{
availableDisplayModes: ["inline", "fullscreen"],
preferredDisplayMode: "fullscreen",
},
);
await app.connect();

// The host may ignore the preference: read the actual mode.
const container = document.getElementById("main")!;
const isFullscreen = app.getHostContext()?.displayMode === "fullscreen";
container.classList.toggle("fullscreen", isFullscreen);
```

> [!NOTE]
> For full examples that implement this pattern, see: [`examples/shadertoy-server/`](https://github.com/modelcontextprotocol/ext-apps/tree/main/examples/shadertoy-server), [`examples/pdf-server/`](https://github.com/modelcontextprotocol/ext-apps/tree/main/examples/pdf-server), and [`examples/map-server/`](https://github.com/modelcontextprotocol/ext-apps/tree/main/examples/map-server).

Expand Down
20 changes: 20 additions & 0 deletions examples/basic-host/src/implementation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -409,5 +409,25 @@ export function newAppBridge(
return { mode: newMode };
};

// Honor `appCapabilities.preferredDisplayMode`. It is a hint for how the
// app is first shown, so apply it once, as soon as the app has initialized
// (a repeated `ui/initialize` does not renew it). This host starts every
// app inline and offers only inline and fullscreen, so fullscreen is the
// one preference to act on, and only if the app also lists it as available.
const applyPreferredDisplayMode = () => {
appBridge.removeEventListener("initialized", applyPreferredDisplayMode);
const caps = appBridge.getAppCapabilities();
if (
caps?.preferredDisplayMode === "fullscreen" &&
caps.availableDisplayModes?.includes("fullscreen") &&
(options?.displayMode ?? "inline") === "inline"
) {
log.info("Applying the app's preferred display mode: fullscreen");
appBridge.sendHostContextChange({ displayMode: "fullscreen" });
callbacks?.onDisplayModeChange?.("fullscreen");
}
};
appBridge.addEventListener("initialized", applyPreferredDisplayMode);

return appBridge;
}
10 changes: 10 additions & 0 deletions examples/basic-host/src/index.module.css
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,16 @@
display: flex;
flex-direction: column;

/* Host controls stay visible above the app: its name and a way out */
.fullscreenBar {
display: flex;
align-items: center;
justify-content: space-between;
gap: 0.5rem;
margin-bottom: 0.5rem;
font-family: monospace;
}

/* Hide collapsible panels in fullscreen */
.collapsiblePanel {
display: none;
Expand Down
13 changes: 13 additions & 0 deletions examples/basic-host/src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -514,8 +514,21 @@ function AppIFramePanel({ toolCallInfo, isDestroying, onTeardownComplete }: AppI
? `${styles.appIframePanel} ${styles.fullscreen}`
: styles.appIframePanel;

// Host-owned way out of fullscreen: the app may have been started there by
// its preferred display mode, before the user has seen it.
const exitFullscreen = () => {
appBridgeRef.current?.sendHostContextChange({ displayMode: "inline" });
setDisplayMode("inline");
};

return (
<div className={panelClassName}>
{displayMode === "fullscreen" && (
<div className={styles.fullscreenBar}>
<span>{toolCallInfo.serverInfo.name}:{toolCallInfo.tool.name}</span>
<button onClick={exitFullscreen}>Exit fullscreen</button>
</div>
)}
<iframe ref={iframeRef} />
{messages.length > 0 && (
<CollapsiblePanel
Expand Down
38 changes: 38 additions & 0 deletions specification/draft/apps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -560,6 +560,12 @@ interface McpUiAppCapabilities {
* @example ["inline", "fullscreen"]
*/
availableDisplayModes?: Array<"inline" | "fullscreen" | "pip">;
/**
* Display mode in which the app prefers to be first shown. A hint: the host MAY ignore it.
* See Display Modes section for details.
* @example "fullscreen"
*/
preferredDisplayMode?: "inline" | "fullscreen" | "pip";
}
```

Expand Down Expand Up @@ -835,6 +841,30 @@ Views declare which display modes they support in the `ui/initialize` request vi

Hosts declare which display modes they support in `HostContext.availableDisplayModes`. Views should check this before requesting a mode change.

#### Preferred Display Mode

A View can name the display mode in which it prefers to be first shown, in `appCapabilities.preferredDisplayMode`. This suits Views that are mainly useful on a large surface, such as an editor, a dashboard or a map.

```typescript
// Example: View supports inline and fullscreen, and prefers to start in fullscreen
{
method: "ui/initialize",
params: {
appCapabilities: {
availableDisplayModes: ["inline", "fullscreen"],
preferredDisplayMode: "fullscreen"
},
// ...
}
}
```

The preference is a hint. The Host chooses how it first shows the View and how much weight it gives to the preference, for example when several Views prefer `fullscreen` at once. The View sends the preference before it has seen `HostContext.availableDisplayModes`, so checking that the mode is available falls to the Host.

A Host that applies the preference should do so before it first shows the View, so that the user does not see the View move. It tells the View either in the `hostContext.displayMode` of its `ui/initialize` result, or in a `ui/notifications/host-context-changed` sent after `ui/notifications/initialized`. A Host that has already shown the View, for example inline while the tool runs, may instead move it once, no later than when it sends the first `ui/notifications/tool-result`. After that, a Host should not use the preference to move the View without a user action: to change mode later, Views use `ui/request-display-mode`.

Note (non-normative): the name and the meaning of this field follow OpenAI's [MCP extensions](https://github.com/openai/mcp-extensions/blob/15e0f4745e7c95865ee2195de5b79f4148b21418/docs/spec.md#display-modes), which define `_meta["openai/ui"].preferredDisplayMode` as the preferred initial display mode. There it is on the UI resource content item rather than in `ui/initialize`; this specification does not define that key.

#### Requesting Changes

Views request display mode changes via `ui/request-display-mode`. See the Requests section for message format.
Expand All @@ -849,12 +879,19 @@ Hosts notify views of display mode changes via `ui/notifications/host-context-ch
- View MUST declare all display modes it supports in `appCapabilities.availableDisplayModes` during initialization.
- View MUST check if the requested mode is in `availableDisplayModes` from host context before requesting a mode change.
- View MUST handle the response mode differing from the requested mode.
- View that sets `appCapabilities.preferredDisplayMode` MUST also list that mode in `appCapabilities.availableDisplayModes`.
- View MUST handle being shown in a display mode other than `appCapabilities.preferredDisplayMode`.

**Host behavior:**
- Host MUST NOT switch the View to a display mode that does not appear in its `appCapabilities.availableDisplayModes`, if set.
- Host MUST return the resulting mode (whether updated or not) in the response to `ui/request-display-mode`.
- If the requested mode is not available, Host SHOULD return the current display mode in the response.
- Host MAY decline display mode requests from Views that did not declare said modes in their capabilities.
- Host MAY ignore `appCapabilities.preferredDisplayMode`, for example in favor of a choice the user has made, or by declining to start a View in `fullscreen` or `pip` without a user action.
- Host MUST NOT refuse `ui/initialize` because it does not offer the preferred mode (one of the display modes defined above), or because `appCapabilities.availableDisplayModes`, if set, lacks it; it ignores the preference instead.
- Host that applies `appCapabilities.preferredDisplayMode` SHOULD do so before it first shows the View. A Host that has already shown the View MAY instead apply it once, no later than when it sends the first `ui/notifications/tool-result`, unless the user has changed the View's display mode in the meantime.
- After that, Host SHOULD NOT use the preference to change the View's display mode without a user action. A repeated `ui/initialize` from a View that is already shown does not renew the preference.
- Host that applies `appCapabilities.preferredDisplayMode` SHOULD report the resulting mode in `hostContext.displayMode`.

### Theming

Expand Down Expand Up @@ -2663,6 +2700,7 @@ App tools MUST be tied to the app's lifecycle:

- **Social engineering:** UI can still display misleading content. Hosts should clearly indicate sandboxed UI boundaries.
- **Resource consumption:** Malicious View can consume CPU/memory. Hosts should implement resource limits.
- **Preferred display mode:** A View that starts in `fullscreen` or `pip` gets a large or floating surface before the user has seen it in the conversation, which can help it imitate Host UI. A Host that starts a View in `fullscreen` or `pip` should keep its own controls visible, naming the app and offering a way out.

## Reservations in MCP

Expand Down
30 changes: 29 additions & 1 deletion src/app-bridge.examples.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import type {
CreateMessageResult,
} from "@modelcontextprotocol/server";
import { AppBridge, PostMessageTransport } from "./app-bridge.js";
import type { McpUiDisplayMode } from "./types.js";
import type { McpUiDisplayMode, McpUiHostContext } from "./types.js";

/**
* Example: Basic usage of the AppBridge class with PostMessageTransport.
Expand Down Expand Up @@ -95,6 +95,34 @@ function AppBridge_getAppCapabilities_checkAfterInit(bridge: AppBridge) {
//#endregion AppBridge_getAppCapabilities_checkAfterInit
}

/**
* Example: Apply the View's preferred display mode.
*/
function AppBridge_getAppCapabilities_preferredDisplayMode(
Comment thread
ochafik marked this conversation as resolved.
Resolved
bridge: AppBridge,
hostContext: McpUiHostContext,
) {
//#region AppBridge_getAppCapabilities_preferredDisplayMode
let applied = false;
bridge.addEventListener("initialized", () => {
// The preference is for how the View is first shown: apply it once.
if (applied) return;
applied = true;

const caps = bridge.getAppCapabilities();
const preferred = caps?.preferredDisplayMode;
// A hint: only for a mode that both the View and the host support.
if (
preferred &&
caps.availableDisplayModes?.includes(preferred) &&
hostContext.availableDisplayModes?.includes(preferred)
) {
bridge.setHostContext({ ...hostContext, displayMode: preferred });
}
});
//#endregion AppBridge_getAppCapabilities_preferredDisplayMode
}

/**
* Example: Log View information after initialization.
*/
Expand Down
69 changes: 69 additions & 0 deletions src/app-bridge.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,75 @@ describe("App <-> AppBridge integration", () => {
expect(appCaps).toEqual(appCapabilities);
});

it("Bridge receives the app's preferred display mode", async () => {
const appCapabilities = {
availableDisplayModes: ["inline" as const, "fullscreen" as const],
preferredDisplayMode: "fullscreen" as const,
};
app = new App(testAppInfo, appCapabilities, { autoResize: false });

await bridge.connect(bridgeTransport);
await app.connect(appTransport);

expect(bridge.getAppCapabilities()).toEqual(appCapabilities);
});

it("App learns from host context that its preferred display mode was applied", async () => {
app = new App(
testAppInfo,
{
availableDisplayModes: ["inline", "fullscreen"],
preferredDisplayMode: "fullscreen",
},
{ autoResize: false },
);
const hostContext: ReturnType<App["getHostContext"]> = {
theme: "dark",
displayMode: "inline",
availableDisplayModes: ["inline", "fullscreen"],
};
bridge = new AppBridge(
createMockClient() as Client,
testHostInfo,
testHostCapabilities,
{ hostContext },
);
bridge.addEventListener("initialized", () => {
const caps = bridge.getAppCapabilities();
const preferred = caps?.preferredDisplayMode;
if (
preferred &&
caps?.availableDisplayModes?.includes(preferred) &&
hostContext.availableDisplayModes?.includes(preferred)
) {
bridge.setHostContext({ ...hostContext, displayMode: preferred });
}
});

await bridge.connect(bridgeTransport);
await app.connect(appTransport);
await flush();

expect(app.getHostContext()).toEqual({
...hostContext,
displayMode: "fullscreen",
});
});

it("Bridge refuses ui/initialize when the preferred display mode is not a known mode", async () => {
app = new App(
testAppInfo,
{ preferredDisplayMode: "not-a-mode" as never },
{ autoResize: false },
);

await bridge.connect(bridgeTransport);
await expect(app.connect(appTransport)).rejects.toMatchObject({
code: -32602,
});
expect(bridge.getAppCapabilities()).toBeUndefined();
});

it("App receives initial hostContext after connect", async () => {
// Need fresh transports for new bridge
const [newAppTransport, newBridgeTransport] =
Expand Down
21 changes: 21 additions & 0 deletions src/app-bridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -618,6 +618,27 @@ export class AppBridge extends Protocol<BaseContext> {
* };
* ```
*
* @example Apply the view's preferred display mode
* ```ts source="./app-bridge.examples.ts#AppBridge_getAppCapabilities_preferredDisplayMode"
* let applied = false;
* bridge.addEventListener("initialized", () => {
* // The preference is for how the View is first shown: apply it once.
* if (applied) return;
* applied = true;
*
* const caps = bridge.getAppCapabilities();
* const preferred = caps?.preferredDisplayMode;
* // A hint: only for a mode that both the View and the host support.
* if (
* preferred &&
* caps.availableDisplayModes?.includes(preferred) &&
* hostContext.availableDisplayModes?.includes(preferred)
* ) {
* bridge.setHostContext({ ...hostContext, displayMode: preferred });
* }
* });
* ```
*
* @see {@link McpUiAppCapabilities `McpUiAppCapabilities`} for the capabilities structure
*/
getAppCapabilities(): McpUiAppCapabilities | undefined {
Expand Down
21 changes: 21 additions & 0 deletions src/app.examples.ts
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,27 @@ async function App_requestDisplayMode_toggle(app: App) {
//#endregion App_requestDisplayMode_toggle
}

/**
* Example: Prefer to be first shown in fullscreen.
*/
async function App_constructor_preferredDisplayMode() {
Comment thread
ochafik marked this conversation as resolved.
Resolved
//#region App_constructor_preferredDisplayMode
const app = new App(
{ name: "MyApp", version: "1.0.0" },
{
availableDisplayModes: ["inline", "fullscreen"],
preferredDisplayMode: "fullscreen",
},
);
await app.connect();

// The host may ignore the preference: read the actual mode.
const container = document.getElementById("main")!;
const isFullscreen = app.getHostContext()?.displayMode === "fullscreen";
container.classList.toggle("fullscreen", isFullscreen);
//#endregion App_constructor_preferredDisplayMode
}

/**
* Example: Manually notify host of size change.
*/
Expand Down
17 changes: 17 additions & 0 deletions src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -577,6 +577,23 @@ export class App extends Protocol<BaseContext> {
* { autoResize: true }, // options
* );
* ```
*
* @example Prefer to be first shown in fullscreen
* ```ts source="./app.examples.ts#App_constructor_preferredDisplayMode"
* const app = new App(
* { name: "MyApp", version: "1.0.0" },
* {
* availableDisplayModes: ["inline", "fullscreen"],
* preferredDisplayMode: "fullscreen",
* },
* );
* await app.connect();
*
* // The host may ignore the preference: read the actual mode.
* const container = document.getElementById("main")!;
* const isFullscreen = app.getHostContext()?.displayMode === "fullscreen";
* container.classList.toggle("fullscreen", isFullscreen);
* ```
*/
constructor(
private _appInfo: Implementation,
Expand Down
Loading
Loading