feat(browser): add @mastra/browser-viewer package with Playwright-based CLI provider#15415
Merged
Conversation
…ed CLI provider
- Add PlaywrightViewer class that launches Chrome via Playwright
- Add providerType ('sdk' | 'cli') to MastraBrowser for runtime enforcement
- Add runtime check in Agent constructor to ensure SDK providers only
- SDK providers (AgentBrowser, StagehandBrowser) set providerType='sdk'
- PlaywrightViewer sets providerType='cli' for workspace usage
This enables reliable screencast and input injection for CLI tools
(agent-browser, browser-use, browse-cli) by using direct page-level
CDP sessions instead of browser-level flattened protocol.
Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
🦋 Changeset detectedLatest commit: b75af5e The changes in this PR will be included in the next version bump. This PR includes changesets to release 22 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
WalkthroughAdds a new Changes
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
- Add example showing 5 different browser automation approaches - CLI agents use PlaywrightViewer (integration pending) - SDK agents use AgentBrowser and StagehandBrowser directly - Include setup instructions for all CLI tools and skills Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
…nt variables - Add addEnv() to WorkspaceSandbox interface (optional) - Implement addEnv() in LocalSandbox - Export commandExists utility from detect.ts This enables workspace integrations (like browser providers) to inject environment variables into the sandbox after creation. Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
- Add browser config to WorkspaceConfig that accepts MastraBrowser instances - Add browser getter and initialization to Workspace class - Inject CDP URL into execute-command for CLI browser commands - Wire BrowserContextProcessor to get context from workspace browser - Add getCdpUrl() method to MastraBrowser base class - Update browser-workspace example agents to use workspace browser config Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
Cleaner name since there's no collision with any existing BrowserViewer class. Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
NikAiyer
force-pushed
the
feat/browser-viewer-cli
branch
from
1c3c317 to
4a7ae00
Compare
When getWorkspace() is called, if the agent doesn't have its own browser configured but the workspace does (CLI approach with BrowserViewer), copy the workspace's browser to the agent's #browser field. This allows server-side code that uses agent.browser to work for both: - SDK providers (AgentBrowser, StagehandBrowser) set directly on agent - CLI providers (BrowserViewer) set on workspace Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
- Add BrowserViewerThreadManager for per-thread Chrome instances - Update BrowserViewer.launch() and getCdpUrl() to accept threadId - Add thread-aware browser launch in execute-command tool - Inject CDP URL into CLI commands for workspace browsers - MastraBrowser base class now supports threadId in launch/getCdpUrl/isBrowserRunning Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
chromium.launch() doesn't expose wsEndpoint(), only launchServer() does. Updated to use launchServer() and connect() pattern for both thread and shared browser sessions.
Playwright's wsEndpoint() returns a Playwright-protocol WebSocket URL, not a raw Chrome CDP URL. External tools like agent-browser, browser-use, and browse-cli need the real CDP WebSocket URL to connect. Chrome writes the debugging port and WebSocket path to a DevToolsActivePort file in its user data directory. We now read this file to construct the correct CDP URL (ws://127.0.0.1:PORT/devtools/browser/GUID). Also unified CDP injection in execute-command.ts: - All CLIs accept either port or full WebSocket URL - Updated browse flag from --cdp-url to --ws (per its help output) - Use full URL for all CLIs for consistency
…ctToCdp helper Extract shared CDP connection logic from createSharedSessionFromCdp and connectToExternalCdp into a single connectToCdp private method. Both methods were doing the same work: - Connect via chromium.connectOverCDP - Get or create context and page - Set up CDP session - Register disconnection handler - Build session and store it - Notify callbacks Now createSharedSessionFromCdp is a one-liner that delegates to connectToCdp, and connectToExternalCdp handles pre-close logic then delegates. Reduces thread-manager.ts from 641 to 592 lines (~8% reduction).
…r logic BrowserCliHandler tests (50 tests): - CLI detection (agent-browser, browser-use, browse) - External CDP flag detection and URL extraction - CDP injection into single and chained commands - Shell escaping for URLs and thread IDs - Warmup state management (isolation by browser/CLI/thread) - Warmup command generation and deduplication execute-command browser tests (10 tests): - Browser launch when CLI detected and not running - CDP injection into commands - External CDP detection and connection - Graceful handling of external CDP connection failure - Warmup command execution and skip when already warmed Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
The #setBrowserFromWorkspace method was incorrectly removed while addressing CodeRabbit feedback. This method is needed for agents to pick up BrowserViewer from workspace, which is required for screencast to work with CLI browser providers. The method only sets this.#browser if: 1. Agent doesn't already have a browser (SDK takes precedence) 2. Workspace has a browser configured Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
browser/browser-viewer/src/thread-manager.ts (1)
356-359: Avoid one-shotDevToolsActivePortdiscovery to reduce startup flakiness.At Line 356, returning
nullimmediately if the file is not present can intermittently miss a valid CDP URL during Chrome startup. A short bounded retry window would make discovery more reliable.🔧 Proposed fix
- if (!existsSync(portFilePath)) { - this.logger?.warn?.('DevToolsActivePort file not found'); - return null; - } + const deadline = Date.now() + 1500; + while (!existsSync(portFilePath) && Date.now() < deadline) { + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 50); + } + if (!existsSync(portFilePath)) { + this.logger?.warn?.('DevToolsActivePort file not found'); + return null; + }Also applies to: 361-369
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@browser/browser-viewer/src/thread-manager.ts` around lines 356 - 359, The current DevToolsActivePort discovery in thread-manager.ts returns null immediately when existsSync(portFilePath) is false, causing flakiness; change this to a short bounded retry loop: repeatedly check existsSync(portFilePath) (and then the subsequent read/parse logic) with a small delay (e.g., 50–200ms) until either the file appears or a total timeout (e.g., 2–5s) elapses, then proceed to read the file or return null; update both the initial existsSync check and the follow-up block around lines 361–369 that parses the file so they use the same retry-with-timeout helper (or inline loop) and preserve logging via this.logger for each failure/timeout.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@browser/browser-viewer/src/thread-manager.ts`:
- Around line 491-493: The 'disconnected' listener currently closes over raw
threadId which can be wrong for shared-slot sessions; change the callback to use
the computed effectiveThreadId (the same value used when storing sessions, e.g.,
the variable/logic that derives effectiveThreadId from
threadId/DEFAULT_THREAD_ID) before calling this.handleBrowserDisconnected so
lifecycle callbacks always reference the shared/session key; update the
browser.on('disconnected', ...) handler to capture and pass effectiveThreadId
into handleBrowserDisconnected instead of the original threadId.
In `@packages/core/src/agent/agent.ts`:
- Around line 1175-1185: The bug is that `#setBrowserFromWorkspace` assigns
workspace.browser into the agent field this.#browser once and that cached
browser is then preferred in `#execute`, causing stale/incorrect browsers for
dynamic/factory workspaces; fix by removing the persistent assignment and
instead read the browser from the incoming workspace at execution time (use
earlyWorkspace?.browser) or, if you must cache, key the cache by workspace
identity and clear/update this.#browser whenever the current workspace
differs—change `#setBrowserFromWorkspace` to not unconditionally set this.#browser
(or add workspace-aware cache invalidation) and update `#execute` to prefer the
current earlyWorkspace?.browser over any cached this.#browser.
---
Nitpick comments:
In `@browser/browser-viewer/src/thread-manager.ts`:
- Around line 356-359: The current DevToolsActivePort discovery in
thread-manager.ts returns null immediately when existsSync(portFilePath) is
false, causing flakiness; change this to a short bounded retry loop: repeatedly
check existsSync(portFilePath) (and then the subsequent read/parse logic) with a
small delay (e.g., 50–200ms) until either the file appears or a total timeout
(e.g., 2–5s) elapses, then proceed to read the file or return null; update both
the initial existsSync check and the follow-up block around lines 361–369 that
parses the file so they use the same retry-with-timeout helper (or inline loop)
and preserve logging via this.logger for each failure/timeout.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 918a6e4b-e4c3-47b0-acce-94af60d5c8fc
📒 Files selected for processing (4)
browser/browser-viewer/src/thread-manager.tspackages/core/src/agent/agent.tspackages/core/src/browser/__tests__/cli-handler.test.tspackages/core/src/workspace/tools/__tests__/execute-command.test.ts
thread-manager.ts: - Add retry loop for DevToolsActivePort file discovery during Chrome startup - Use effectiveThreadId in connectToCdp disconnect handler for consistent lifecycle agent.ts: - Track explicit vs workspace-derived browser with #hasExplicitBrowser flag - Keep workspace browser in sync per-request instead of caching stale reference - hasOwnBrowser() now returns true only for explicitly configured SDK browsers Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
abhiaiyer91
approved these changes
Apr 22, 2026
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
packages/core/src/agent/agent.ts (2)
1142-1168:⚠️ Potential issue | 🟠 MajorClear the derived browser when no workspace resolves.
This still leaves
this.#browsersticky when the current request has no workspace at all. If a factory returnsundefinedhere, or there is no global workspace on a later request, the previously-synced workspace browser survives and#execute()can keep injecting that stale session into browser tools /requestContext.browser.Suggested fix
const resolvedWorkspace = await Promise.resolve(result); if (!resolvedWorkspace) { + if (!this.#hasExplicitBrowser) { + this.#browser = undefined; + } return undefined; } @@ const globalWorkspace = this.#mastra?.getWorkspace(); + if (!globalWorkspace && !this.#hasExplicitBrowser) { + this.#browser = undefined; + } if (globalWorkspace) { this.#setBrowserFromWorkspace(globalWorkspace); } return globalWorkspace;Also applies to: 1177-1185
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/core/src/agent/agent.ts` around lines 1142 - 1168, When no workspace resolves (resolvedWorkspace is undefined and Mastra's getWorkspace() returns undefined), clear the agent's derived browser to avoid carrying a stale session: update the branch after the factory-resolution block and the fallback block (where `#setBrowserFromWorkspace` is called) to explicitly clear this.#browser (or call a `#clearBrowser` or similar helper) when there is no workspace to set; ensure the change is applied in both places referenced around resolvedWorkspace/#setBrowserFromWorkspace and the later fallback (also at the region noted around 1177-1185) so that requestContext.browser cannot retain a previous session.
197-197:⚠️ Potential issue | 🟠 MajorKeep
setBrowser()aligned with the new browser invariant.The SDK-only check now exists only in the constructor, while the public setter still accepts any
MastraBrowserand never updates#hasExplicitBrowser. That meansagent.setBrowser(browserViewer)bypassesAGENT_INVALID_BROWSER_PROVIDER, andagent.setBrowser(sdkBrowser)can still be treated as non-explicit and later overwritten byworkspace.browser.Also applies to: 330-348, 488-497
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/core/src/agent/agent.ts` at line 197, Update setBrowser to mirror the constructor's browser invariant: validate the incoming MastraBrowser the same way the constructor does (reject SDK-only providers by throwing AGENT_INVALID_BROWSER_PROVIDER), assign the browser to this.#browser, and set the private flag this.#hasExplicitBrowser to true for explicit (non-SDK) browsers (and false for SDK/browser-from-workspace cases) so calling agent.setBrowser(...) cannot bypass the invariant or leave `#hasExplicitBrowser` stale; apply the same change to the other setBrowser-like paths referenced around the other occurrences so all places (including where workspace.browser may overwrite) maintain the same validation and flag update.
🧹 Nitpick comments (1)
packages/core/src/browser/__tests__/cli-handler.test.ts (1)
238-244: Consider strengthening the URL escaping assertion.The current assertion only verifies the beginning of the escaped string. Since the URL contains embedded single quotes (
'test'), the test should verify the complete escaped output to ensure these are properly handled.Suggested improvement
it('escapes special characters in URLs', () => { const specialUrl = "ws://localhost:9222/devtools?foo=bar&baz='test'"; const result = handler.injectCdpUrl('agent-browser open https://google.com', specialUrl, threadId); - // Should be shell-escaped - expect(result).toContain("'ws://localhost:9222/devtools?foo=bar&baz="); + // Verify the URL is properly shell-escaped + // The exact escaping format depends on the implementation (e.g., single quotes with escaped inner quotes) + expect(result).toContain('--cdp'); + expect(result).toMatch(/ws:\/\/localhost:9222\/devtools\?foo=bar&baz=/); + // Verify the result doesn't allow shell interpretation of special chars + expect(result).not.toMatch(/[^'"]&[^'"]/); // Unquoted ampersand would be interpreted by shell });🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/core/src/browser/__tests__/cli-handler.test.ts` around lines 238 - 244, Update the test "escapes special characters in URLs" to assert the full escaped output from handler.injectCdpUrl rather than only the prefix: call handler.injectCdpUrl with the specialUrl and verify the returned string exactly contains the correctly shell-escaped form of the entire URL (including the embedded single-quoted segment "'test'") so that the escaping logic in injectCdpUrl is fully validated; locate the test and the injectCdpUrl invocation to replace the partial expect(result).toContain(...) with an assertion that compares or matches the complete expected escaped string.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/core/src/workspace/tools/__tests__/execute-command.test.ts`:
- Around line 551-554: The tests call vi.clearAllMocks() in the beforeEach but
do not clear the singleton warmup state held by browserCliHandler, causing test
leakage; update the beforeEach to also call browserCliHandler.resetWarmup() (or
the module's explicit clear/reset method) to reset the CLI warmup cache between
tests so warmup assertions remain isolated and order-independent.
---
Duplicate comments:
In `@packages/core/src/agent/agent.ts`:
- Around line 1142-1168: When no workspace resolves (resolvedWorkspace is
undefined and Mastra's getWorkspace() returns undefined), clear the agent's
derived browser to avoid carrying a stale session: update the branch after the
factory-resolution block and the fallback block (where `#setBrowserFromWorkspace`
is called) to explicitly clear this.#browser (or call a `#clearBrowser` or similar
helper) when there is no workspace to set; ensure the change is applied in both
places referenced around resolvedWorkspace/#setBrowserFromWorkspace and the
later fallback (also at the region noted around 1177-1185) so that
requestContext.browser cannot retain a previous session.
- Line 197: Update setBrowser to mirror the constructor's browser invariant:
validate the incoming MastraBrowser the same way the constructor does (reject
SDK-only providers by throwing AGENT_INVALID_BROWSER_PROVIDER), assign the
browser to this.#browser, and set the private flag this.#hasExplicitBrowser to
true for explicit (non-SDK) browsers (and false for SDK/browser-from-workspace
cases) so calling agent.setBrowser(...) cannot bypass the invariant or leave
`#hasExplicitBrowser` stale; apply the same change to the other setBrowser-like
paths referenced around the other occurrences so all places (including where
workspace.browser may overwrite) maintain the same validation and flag update.
---
Nitpick comments:
In `@packages/core/src/browser/__tests__/cli-handler.test.ts`:
- Around line 238-244: Update the test "escapes special characters in URLs" to
assert the full escaped output from handler.injectCdpUrl rather than only the
prefix: call handler.injectCdpUrl with the specialUrl and verify the returned
string exactly contains the correctly shell-escaped form of the entire URL
(including the embedded single-quoted segment "'test'") so that the escaping
logic in injectCdpUrl is fully validated; locate the test and the injectCdpUrl
invocation to replace the partial expect(result).toContain(...) with an
assertion that compares or matches the complete expected escaped string.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 3a0cb793-260d-45a8-890a-d0c3ab27ce6f
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (4)
browser/browser-viewer/src/thread-manager.tspackages/core/src/agent/agent.tspackages/core/src/browser/__tests__/cli-handler.test.tspackages/core/src/workspace/tools/__tests__/execute-command.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- browser/browser-viewer/src/thread-manager.ts
Previously the ViewerRegistry only logged screencast errors but didn't notify the UI, leaving the screencast panel showing 'Live' after browser close. Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@browser/browser-viewer/src/browser-viewer.ts`:
- Around line 241-250: When constructing BrowserState in browser-viewer.ts,
don't hard-code the active tab to index 0; use
BrowserViewerThreadManager.getActivePageForThread() to find the active Page and
compute activeTabIndex and each tab's isActive accordingly: call
getActivePageForThread() to obtain the active Page (or its GUID/identity),
iterate context.pages() to build tabs using page.url() and set isActive = (page
=== activePage) (or compare identities), and set activeTabIndex to the index of
that active page (fallback to 0 only if no active page found). Update the tabs
creation code and the returned activeTabIndex to reflect that computed index.
- Around line 264-293: The provider.getCdpSession implementation re-selects
pages and calls context.newCDPSession directly, bypassing the thread manager's
existing active-page resolution and race handling; replace the manual selection
with the thread-manager helper (e.g., call
this.threadManager.getCdpSession(threadId) or the thread-manager method that
returns an existing/ready CDP session for a thread) and return a wrapper around
that session implementing send/on/off, throwing if the thread-manager helper
returns null/throws so the same disconnect/page-close cleanup logic is used as
in thread-manager.ts.
- Around line 377-392: The injectMouseEvent and injectKeyboardEvent
implementations call this.threadManager.getCdpSessionForThread(...) which opens
a new CDP session per call; change these methods to reuse a thread-scoped
persistent session instead of creating a session per event (or, if the thread
manager provides a persistent accessor, call that), e.g. obtain and cache a
single CDP session per thread (or request a long-lived session from
BrowserViewerThreadManager) and reuse it for subsequent dispatches in
injectMouseEvent and injectKeyboardEvent, or ensure any transient session
returned by getCdpSessionForThread is explicitly closed/retired after dispatch
to avoid accumulating sessions.
In `@packages/core/src/agent/agent.ts`:
- Around line 488-493: The setBrowser method currently only sets this.#browser
and flips this.#hasExplicitBrowser when a truthy browser is passed, so calling
setBrowser(undefined) fails to persist an explicit "disabled" override and
getWorkspace() can repopulate `#browser`; change setBrowser(browser: MastraBrowser
| undefined) so that whenever it is called with undefined it still sets
this.#hasExplicitBrowser = true (i.e., treat an explicit undefined as an
explicit override), while preserving the existing behavior for non-undefined
browsers; update any comments/tests relying on explicit-ness of
hasOwnBrowser()/#hasExplicitBrowser accordingly to reflect that an explicit
disable is recorded.
In `@packages/core/src/browser/cli-handler.ts`:
- Around line 44-46: The external CDP detection only matches --cdp values that
start with ws:// or wss://; update the regexes externalCdpPattern and
externalCdpExtractor to also recognize numeric ports and host:port forms (e.g.,
9222, localhost:9222, 127.0.0.1:9222) so analyzeCommand treats these as external
CDP targets; change externalCdpPattern to allow --cdp followed by either wss?://
or an IPv4/hostname with optional :port or a bare port number, and change
externalCdpExtractor to capture the full host[:port] or bare port into the same
capture group (preserving existing wss?:// capture), ensuring downstream code
that uses the extractor (analyzeCommand and any code paths around the
launch/warmup logic) can handle the extracted host/port string.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3e02d3f8-cdd7-4fde-99dc-73f63542f334
📒 Files selected for processing (6)
browser/browser-viewer/src/browser-viewer.tsbrowser/browser-viewer/src/thread-manager.tspackages/core/src/agent/agent.tspackages/core/src/browser/__tests__/cli-handler.test.tspackages/core/src/browser/cli-handler.tspackages/core/src/workspace/tools/__tests__/execute-command.test.ts
✅ Files skipped from review due to trivial changes (1)
- browser/browser-viewer/src/thread-manager.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/core/src/browser/tests/cli-handler.test.ts
- packages/core/src/workspace/tools/tests/execute-command.test.ts
Comment on lines
+241
to
+250
| const pages = context.pages(); | ||
| const tabs: BrowserTabState[] = pages.map((page, index) => ({ | ||
| url: page.url(), | ||
| title: '', // Would need async call to get title | ||
| isActive: index === 0, | ||
| })); | ||
|
|
||
| return { | ||
| tabs, | ||
| activeTabIndex: 0, |
Contributor
There was a problem hiding this comment.
Use the actual active page when building BrowserState.
isActive and activeTabIndex are hard-coded to the first page, so multi-tab sessions will report the wrong active tab after a tab switch. BrowserViewerThreadManager.getActivePageForThread() is already available and should drive both values here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@browser/browser-viewer/src/browser-viewer.ts` around lines 241 - 250, When
constructing BrowserState in browser-viewer.ts, don't hard-code the active tab
to index 0; use BrowserViewerThreadManager.getActivePageForThread() to find the
active Page and compute activeTabIndex and each tab's isActive accordingly: call
getActivePageForThread() to obtain the active Page (or its GUID/identity),
iterate context.pages() to build tabs using page.url() and set isActive = (page
=== activePage) (or compare identities), and set activeTabIndex to the index of
that active page (fallback to 0 only if no active page found). Update the tabs
creation code and the returned activeTabIndex to reflect that computed index.
Comment on lines
+377
to
+392
| override async injectMouseEvent(params: MouseEventParams, threadId?: string): Promise<void> { | ||
| const cdpSession = await this.threadManager.getCdpSessionForThread(threadId ?? this.getCurrentThread()); | ||
| if (!cdpSession) { | ||
| throw new Error('CDP session not available for mouse injection'); | ||
| } | ||
|
|
||
| await cdpSession.send('Input.dispatchMouseEvent', params); | ||
| } | ||
|
|
||
| override async injectKeyboardEvent(params: KeyboardEventParams, threadId?: string): Promise<void> { | ||
| const cdpSession = await this.threadManager.getCdpSessionForThread(threadId ?? this.getCurrentThread()); | ||
| if (!cdpSession) { | ||
| throw new Error('CDP session not available for keyboard injection'); | ||
| } | ||
|
|
||
| await cdpSession.send('Input.dispatchKeyEvent', params); |
Contributor
There was a problem hiding this comment.
Don’t open a fresh CDP session for every input event.
BrowserViewerThreadManager.getCdpSessionForThread() creates a new CDP session on each call, so mouse moves and keyboard input turn into a session-per-event hot path here. That is going to accumulate avoidable CDP sessions and add a lot of overhead under live control; these methods should reuse a thread-scoped session or explicitly retire the transient session after dispatch.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@browser/browser-viewer/src/browser-viewer.ts` around lines 377 - 392, The
injectMouseEvent and injectKeyboardEvent implementations call
this.threadManager.getCdpSessionForThread(...) which opens a new CDP session per
call; change these methods to reuse a thread-scoped persistent session instead
of creating a session per event (or, if the thread manager provides a persistent
accessor, call that), e.g. obtain and cache a single CDP session per thread (or
request a long-lived session from BrowserViewerThreadManager) and reuse it for
subsequent dispatches in injectMouseEvent and injectKeyboardEvent, or ensure any
transient session returned by getCdpSessionForThread is explicitly
closed/retired after dispatch to avoid accumulating sessions.
Comment on lines
+44
to
+46
| externalCdpPattern: /(?:\bconnect\s+["']?wss?:\/\/|--cdp\s+["']?wss?:\/\/)/, | ||
| // Extract URL from: connect "wss://..." or --cdp "wss://..." | ||
| externalCdpExtractor: /(?:\bconnect|--cdp)\s+["']?(wss?:\/\/[^\s"']+)["']?/, |
Contributor
There was a problem hiding this comment.
agent-browser --cdp <port> is not recognized as external CDP and can trigger the wrong execution path.
Line 44 only treats --cdp as external when the value starts with ws:///wss://. That misses valid port-style usage (e.g., --cdp 9222), so analyzeCommand may classify the command as non-external and unnecessarily run local launch/warmup logic.
💡 Proposed fix
'agent-browser': {
@@
- externalCdpPattern: /(?:\bconnect\s+["']?wss?:\/\/|--cdp\s+["']?wss?:\/\/)/,
+ externalCdpPattern: /(?:\bconnect\s+["']?wss?:\/\/|--cdp\s+["']?\S+)/,Also applies to: 127-145, 293-295
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/core/src/browser/cli-handler.ts` around lines 44 - 46, The external
CDP detection only matches --cdp values that start with ws:// or wss://; update
the regexes externalCdpPattern and externalCdpExtractor to also recognize
numeric ports and host:port forms (e.g., 9222, localhost:9222, 127.0.0.1:9222)
so analyzeCommand treats these as external CDP targets; change
externalCdpPattern to allow --cdp followed by either wss?:// or an IPv4/hostname
with optional :port or a bare port number, and change externalCdpExtractor to
capture the full host[:port] or bare port into the same capture group
(preserving existing wss?:// capture), ensuring downstream code that uses the
extractor (analyzeCommand and any code paths around the launch/warmup logic) can
handle the extracted host/port string.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/server/src/server/browser-stream/viewer-registry.ts`:
- Around line 402-409: The error handler on stream doesn't explicitly stop the
errored stream, risking leaked CDP/session resources; inside the
stream.on('error', ...) block (where you check this.screencasts.get(viewerKey)
!== currentStream, log the error, then this.screencasts.delete(viewerKey) and
this.broadcastStatus(viewerKey, { status: 'browser_closed' })), explicitly tear
down the currentStream before deleting it: invoke the stream's cleanup API (e.g.
call a stop/close/dispose method if available on currentStream), remove any
remaining listeners (e.g. removeAllListeners or equivalent), then proceed to
delete from this.screencasts and call this.broadcastStatus to ensure no orphaned
resources or handlers remain.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 752c31b6-c7be-468c-af46-894016d05e29
📒 Files selected for processing (2)
.changeset/warm-dots-flow.mdpackages/server/src/server/browser-stream/viewer-registry.ts
✅ Files skipped from review due to trivial changes (1)
- .changeset/warm-dots-flow.md
- Fix BrowserState active tab logic to use actual page count - Cache CDP sessions for input injection to reduce overhead - Use dedicated method for screencast CDP session creation - Update setBrowser(undefined) to mark browser as explicitly set - Expand external CDP detection to recognize --cdp <port> pattern - Fix viewer-registry error handler to explicitly stop stream - Add browser disconnect detection via Target.targetDestroyed - Propagate browser close events from ThreadManager to ViewerRegistry Co-Authored-By: Mastra Code (anthropic/claude-opus-4-5) <noreply@mastra.ai>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
browser/browser-viewer/src/thread-manager.ts (1)
326-367: CDP session caching is a reasonable approach but has an edge case.The caching keyed by
pageUrl(line 349) addresses the previous concern about stale sessions. However, if multiple tabs have the same URL, the cache may return a session for the wrong tab. This is a minor edge case since the session will still target a valid page with the expected URL.Consider using a more robust page identity (like Playwright's internal page ID) if this becomes an issue in practice.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@browser/browser-viewer/src/thread-manager.ts` around lines 326 - 367, The cache currently keys CDP sessions by pageUrl in getCdpSessionForThread, which can return the wrong tab when multiple pages share the same URL; update inputCdpSessions to store and compare a stable unique page identifier from the Playwright Page (e.g., the page's internal GUID/id accessible on the Page object or main frame) instead of pageUrl, fall back to URL if no id is available, and update all uses in getCdpSessionForThread (where activePage, cached, pageUrl, and newCDPSession are referenced) to check cached.pageId === currentPageId before reusing the session and to cache { session, pageId } when creating a new CDP session.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@browser/browser-viewer/src/thread-manager.ts`:
- Around line 326-367: The cache currently keys CDP sessions by pageUrl in
getCdpSessionForThread, which can return the wrong tab when multiple pages share
the same URL; update inputCdpSessions to store and compare a stable unique page
identifier from the Playwright Page (e.g., the page's internal GUID/id
accessible on the Page object or main frame) instead of pageUrl, fall back to
URL if no id is available, and update all uses in getCdpSessionForThread (where
activePage, cached, pageUrl, and newCDPSession are referenced) to check
cached.pageId === currentPageId before reusing the session and to cache {
session, pageId } when creating a new CDP session.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: c69d64c1-a5ec-4201-9e48-9b798051e4a4
📒 Files selected for processing (6)
browser/browser-viewer/src/browser-viewer.tsbrowser/browser-viewer/src/thread-manager.tspackages/core/src/agent/agent.tspackages/core/src/browser/__tests__/cli-handler.test.tspackages/core/src/browser/cli-handler.tspackages/server/src/server/browser-stream/viewer-registry.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/server/src/server/browser-stream/viewer-registry.ts
- packages/core/src/agent/agent.ts
abhiaiyer91
pushed a commit
that referenced
this pull request
Apr 22, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ `main` is currently in **pre mode** so this branch has prereleases rather than normal releases. If you want to exit prereleases, run `changeset pre exit` on `main`.⚠️ ⚠️ ⚠️ ⚠️ ⚠️ ⚠️ # Releases ## @mastra/browser-viewer@0.1.0-alpha.0 ### Minor Changes - Initial release of @mastra/browser-viewer ([#15415](#15415)) Playwright-based browser viewer for CLI providers that enables screencast visualization in Studio. Supports thread-isolated browser sessions and automatic CDP connection management. ```typescript import { BrowserViewer } from '@mastra/browser-viewer'; const workspace = new Workspace({ sandbox: new LocalSandbox({ cwd: './workspace' }), browser: new BrowserViewer({ cli: 'agent-browser', headless: false, }), }); ``` ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## @mastra/core@1.27.0-alpha.1 ### Minor Changes - Added support for CLI-driven browser automation with screencast support in `@mastra/core`, including automatic CDP injection for browser CLIs. ([#15415](#15415)) Fixed local process spawning so workspace-relative `cwd` values no longer get duplicated. ### Patch Changes - Added opt-in temporal-gap markers for observational memory. When enabled via `observationalMemory.temporalMarkers: true`, the agent receives a `<system-reminder type="temporal-gap">` before any user message that arrives more than 10 minutes after the previous one, so it can anchor responses in real elapsed time. Markers are persisted, surfaced to the observer, and rendered by the MastraCode TUI on reload. ([#15605](#15605)) ## @mastra/memory@1.17.0-alpha.0 ### Minor Changes - Added opt-in temporal-gap markers for observational memory. When enabled via `observationalMemory.temporalMarkers: true`, the agent receives a `<system-reminder type="temporal-gap">` before any user message that arrives more than 10 minutes after the previous one, so it can anchor responses in real elapsed time. Markers are persisted, surfaced to the observer, and rendered by the MastraCode TUI on reload. ([#15605](#15605)) ### Patch Changes - Fixed observer agent truncation that could cut UTF-16 surrogate pairs in half when formatting messages, tool results, or observation lines with emoji or other astral-plane characters. This produced lone surrogates that strict JSON parsers (including Anthropic's) reject with errors like `no low surrogate in string`, causing observer runs to fail. ([#15634](#15634)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## @mastra/client-js@1.14.1-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## @mastra/react@0.2.28-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/client-js@1.14.1-alpha.1 ## @mastra/deployer-cloud@1.27.0-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/deployer@1.27.0-alpha.1 ## @mastra/deployer-cloudflare@1.1.25-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/deployer@1.27.0-alpha.1 ## @mastra/deployer-netlify@1.1.1-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/deployer@1.27.0-alpha.1 ## @mastra/deployer-vercel@1.1.19-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/deployer@1.27.0-alpha.1 ## @mastra/longmemeval@1.0.30-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`6e9ab07`](6e9ab07)]: - @mastra/core@1.27.0-alpha.1 - @mastra/memory@1.17.0-alpha.0 ## @mastra/opencode@0.0.27-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`6e9ab07`](6e9ab07)]: - @mastra/core@1.27.0-alpha.1 - @mastra/memory@1.17.0-alpha.0 ## @mastra/tavily@1.0.1-alpha.0 ### Patch Changes - Fixed runtime `ERR_MODULE_NOT_FOUND` for `@tavily/core` by making it a direct dependency. Consumers no longer need to install `@tavily/core` manually. ([#15628](#15628)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## mastracode@0.15.1-alpha.1 ### Patch Changes - Added opt-in temporal-gap markers for observational memory. When enabled via `observationalMemory.temporalMarkers: true`, the agent receives a `<system-reminder type="temporal-gap">` before any user message that arrives more than 10 minutes after the previous one, so it can anchor responses in real elapsed time. Markers are persisted, surfaced to the observer, and rendered by the MastraCode TUI on reload. ([#15605](#15605)) - Improved model ID display in the Mastra Code TUI status line. Fireworks model IDs are now shown in compact form (e.g. fireworks/kimi-k2.6 instead of the full fireworks-ai/accounts/fireworks/models/... path). Version separators in model names are also normalized (e.g. kimi-k2p6 is displayed as kimi-k2.6). ([#15631](#15631)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`96f6fb2`](96f6fb2), [`01a7d51`](01a7d51), [`6e9ab07`](6e9ab07)]: - @mastra/core@1.27.0-alpha.1 - @mastra/tavily@1.0.1-alpha.0 - @mastra/memory@1.17.0-alpha.0 ## @mastra/agent-builder@1.0.28-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`6e9ab07`](6e9ab07)]: - @mastra/core@1.27.0-alpha.1 - @mastra/memory@1.17.0-alpha.0 ## mastra@1.6.2-alpha.1 ### Patch Changes - Improved server deploy errors: the CLI now shows the platform API message (for example billing or payment errors) instead of only a generic status code. ([#15459](#15459)) - Improved env file handling for `mastra server deploy` and `mastra studio deploy`. Deploy now selects a single env file instead of silently merging multiple files. When multiple `.env` files exist, you are prompted to choose one. Added `--env-file` to specify the file directly (e.g. `--env-file .env.staging`), which is required in non-interactive mode when multiple env files are present. ([#15641](#15641)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/deployer@1.27.0-alpha.1 ## @mastra/deployer@1.27.0-alpha.1 ### Patch Changes - Updated dependencies \[[`2a87046`](2a87046), [`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`0a0aa94`](0a0aa94)]: - @mastra/server@1.27.0-alpha.1 - @mastra/core@1.27.0-alpha.1 ## @mastra/editor@0.7.18-alpha.0 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`6e9ab07`](6e9ab07)]: - @mastra/core@1.27.0-alpha.1 - @mastra/memory@1.17.0-alpha.0 ## @mastra/mcp-docs-server@1.1.27-alpha.2 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## @mastra/playground-ui@23.0.1-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/client-js@1.14.1-alpha.1 - @mastra/react@0.2.28-alpha.1 ## @mastra/server@1.27.0-alpha.1 ### Patch Changes - Forward `requestContext` from the `/approve-tool-call`, `/decline-tool-call`, `/approve-tool-call-generate` and `/decline-tool-call-generate` REST handlers to `agent.approveToolCall(...)` / `declineToolCall(...)` / `approveToolCallGenerate(...)` / `declineToolCallGenerate(...)`. ([#15620](#15620)) Previously `requestContext` was destructured from the handler arguments but never passed through. On resume, `dynamicInstructions` ran with `requestContext: undefined`, so any value placed on the per-request `RequestContext` by upstream middleware (or by `body.requestContext` auto-merge) was lost for the rest of the turn. Agents whose prompt assembly depends on request-scoped data (e.g. read-only state from the frontend) produced blank or placeholder responses after the user approved a HITL tool call. Other agent entry points (`stream`, `generate`) already forwarded `requestContext` correctly; this brings the approval routes in line. - Fixed screencast panel staying "Live" after browser closes due to an error. The `ViewerRegistry` now broadcasts `browser_closed` status when a screencast stream emits an error, not just when it stops cleanly. ([#15415](#15415)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## @mastra/express@1.3.11-alpha.1 ### Patch Changes - Updated dependencies \[[`2a87046`](2a87046), [`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`0a0aa94`](0a0aa94)]: - @mastra/server@1.27.0-alpha.1 - @mastra/core@1.27.0-alpha.1 ## @mastra/fastify@1.3.11-alpha.1 ### Patch Changes - Fix custom route handlers on the Fastify adapter silently overwriting request-body fields named `tools` (e.g. `POST /stored/agents`, `POST /stored/workspaces`). The adapter now exposes registered tools as `registeredTools` in handler params, matching the Express and Hono adapters and the `@mastra/server` handler contract. ([#15635](#15635)) - Updated dependencies \[[`2a87046`](2a87046), [`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`0a0aa94`](0a0aa94)]: - @mastra/server@1.27.0-alpha.1 - @mastra/core@1.27.0-alpha.1 ## @mastra/hono@1.4.6-alpha.1 ### Patch Changes - Updated dependencies \[[`2a87046`](2a87046), [`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`0a0aa94`](0a0aa94)]: - @mastra/server@1.27.0-alpha.1 - @mastra/core@1.27.0-alpha.1 ## @mastra/koa@1.4.11-alpha.1 ### Patch Changes - Updated dependencies \[[`2a87046`](2a87046), [`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51), [`0a0aa94`](0a0aa94)]: - @mastra/server@1.27.0-alpha.1 - @mastra/core@1.27.0-alpha.1 ## @mastra/clickhouse@1.5.1-alpha.1 ### Patch Changes - Improved ClickHouse v-next observability initialization errors to include the underlying ClickHouse message in the standard error text. This makes init failures actionable in loggers that only print `error.message`. ([#15588](#15588)) - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 ## create-mastra@1.6.2-alpha.1 ## @internal/playground@1.6.2-alpha.1 ### Patch Changes - Updated dependencies \[[`0a0aa94`](0a0aa94), [`01a7d51`](01a7d51)]: - @mastra/core@1.27.0-alpha.1 - @mastra/client-js@1.14.1-alpha.1 - @mastra/ai-sdk@1.4.1 - @mastra/react@0.2.28-alpha.1 - @mastra/playground-ui@23.0.1-alpha.1 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds package for Playwright-based browser management with CLI providers (agent-browser, browser-use, browse-cli), enabling screencast visualization in Studio.
What's New
@mastra/browser-viewer (new package)
BrowserViewerclass that launches Chrome via Playwright with--remote-debugging-port@mastra/core changes
Workspace.browserconfiguration for CLI provider integrationBrowserCliHandlerfor automatic CDP URL injection into browser CLI commandsMastraBrowser.providerTypeproperty ('sdk'|'cli') andconnectToExternalCdp()methodHow It Works
When an agent uses a browser CLI command:
BrowserViewerlazily launches Chrome with remote debugging enabledBrowserCliHandlerautomatically injects the CDP URL into CLI commandsSupported CLIs:
agent-browser— via--cdp <port>and warmupconnectcommandbrowser-use— via--cdp-url <url>browse— via--ws <url>External CDP support: If an agent provides their own CDP URL (e.g., browser-use cloud), the system detects it, skips injection, and connects
BrowserViewerto the external browser for screencast.Testing Notes
npm i -g agent-browser,pipx install browser-use, etc.)Related
ELI5: This PR adds a small tool that can launch or connect to Chrome and share a WebSocket CDP URL so CLI browser tools can talk to it—letting Studio show live browser screencasts and interact with those CLIs. It also wires core workspace/agent logic so CLI browser commands get auto-configured, warmed up, and lifecycle-managed.
New package: @mastra/browser-viewer
Core integrations (packages/core)
Browser-viewer internals
Commits / runtime fixes and refinements
Bug fixes
Tests
Docs & release
Scope & reviewer notes