Embedded browser preview with screenshot-to-composer and agent-operated verification #155

Activity

1. currentbits

   added

   enhancementNew feature or requestNew feature or request

   plan:todoPlanboard statusPlanboard status

   on Aug 16, 2026
2. currentbits commented on Aug 17, 2026

   

   currentbits

   on Aug 17, 2026

   OwnerAuthor

   More actions

   Research note (Onlook, Apache-2.0, documented in-repo): the strong version of preview integration is element-to-code mapping via instrumentation — instrument the previewed app so every DOM element maps back to its source location; click an element in the preview and the agent gets the exact file/line. Bigger bet than screenshot annotations, but it is the category's standout differentiator and the implementation approach is open source.
3. currentbits commented on Aug 18, 2026

   

   currentbits

   on Aug 18, 2026

   OwnerAuthor

   More actions

   Round-19: Orca Design Mode is the sharpest version of the annotation half of this ticket (https://www.onorca.dev/).

   A real Chromium window per worktree. Click any UI element → the agent receives that element's HTML, CSS, and a cropped screenshot. Not a verbal description of "the blue button", not a full-page screenshot the model has to search.

   Synara 0.6.4/0.6.6 already specified the transport (versioned, count- and field-bounded, exact-page affinity kept local, document-only metadata stripped before provider injection). Combine: Orca's payload (HTML + CSS + crop) + Synara's bounds. That is the annotation step. Live agent-operated navigation stays step two, as written.
4. currentbits commented on Aug 19, 2026

   

   currentbits

   on Aug 19, 2026

   OwnerAuthor

   More actions

   Claude Code Desktop's Browser pane is now the most complete version of this feature shipped anywhere. Details worth lifting (docs):

   • The dev server is declarative, in the repo: .claude/launch.json with name / runtimeExecutable / runtimeArgs / port, multiple configurations per project (frontend + API), auto-detected on first run, editable from a server dropdown in the session toolbar. This is the answer to Per-project setup script auto-run on worktree creation, plus named quick actions #153 and Per-worktree dev-server isolation: port injection, data dirs, app identity #250 as well — a checked-in file beats a per-project setting in Solenta's store, and it survives worktrees for free.
   • autoVerify: true by default: after every edit the agent screenshots, checks for errors, inspects the DOM, clicks elements, fills forms and fixes what it finds — with a per-project override in the same file and a toggle in the dropdown.
   • Persist sessions: cookies and localStorage survive server restarts so the agent does not have to re-login on every iteration. This is the small thing that decides whether agent-driven verification works past the login screen at all.
   • It is a tabbed browser, not a preview iframe: docs and issue trackers open next to the running app, Cmd+Shift+B toggles, Cmd+Shift+S selects an element (→ composer, which is this issue's screenshot-to-composer).
   • Backends too: "Claude can test API endpoints, view server logs, and iterate" — the pane is not a frontend-only feature.
   • Clean profile by design: the pane uses a browser profile with none of the user's logins, explicitly distinct from the Chrome extension that acts as you (Signed-in browser bridge: agents drive the user's real browser session #267). Good split for Solenta: embedded pane = build-and-test, signed-in bridge = act-as-me.

   The .claude/launch.json shape is the highest-value piece and is independently useful without the pane.
5. currentbits

   added

   plan:doingPlanboard statusPlanboard status

   and removed

   plan:todoPlanboard statusPlanboard status

   on Aug 23, 2026
6. currentbits commented on Aug 23, 2026

   

   currentbits

   on Aug 23, 2026

   OwnerAuthor

   More actions

   Shipped on this branch (coder/fork-embedded-browser-preview-with-scree-a06432).

   Step 1: preview pane + screenshot-to-composer

   • Views → Browser is a real pane (no longer a placeholder).
   • It loads the thread's captured dev-server URL (or a typed localhost URL).
   • Add screenshot captures the visible page and pins it on the composer as an image attachment.

   Step 2: agent-operated navigation

   • coder-threads MCP tool preview (info / navigate / reload / back / forward / screenshot / click / type) drives the same visible guest.
   • The pane must be open on that thread. Screenshot returns a PNG the agent can see.

   Boundary

   • App window policy in electron/links.js is unchanged (still never navigates to the web).
   • The guest is loopback-only (localhost / 127.0.0.1 / ::1). Other http(s) links open in the OS browser. webviewTag guests must use partition solenta-preview:<threadId> or they are refused.

   Not in this slice: element annotations (click a node, attach a redacted callout). That is the remaining Synara gap.
7. currentbits

   added

   plan:donePlanboard statusPlanboard status

   and removed

   plan:doingPlanboard statusPlanboard status

   on Aug 23, 2026
8. currentbits

   added a commit that references this issue on Aug 23, 2026

   Embedded browser preview pane with screenshot-to-composer (#155) (#679)

   ...

   79cab12
9. currentbits

   added

   plan:doingPlanboard statusPlanboard status

   plan:donePlanboard statusPlanboard status

   and removed

   plan:donePlanboard statusPlanboard status

   plan:doingPlanboard statusPlanboard status

   on Oct 3, 2026
10. currentbits commented on Oct 3, 2026

    

    currentbits

    on Oct 3, 2026

    OwnerAuthor

    More actions

    Landed on main as 79cab12 "Embedded browser preview pane with screenshot-to-composer (#155) (#679)". The issue was left open with plan:done; closing during the 2026-10-03 board cleanup.
11. currentbits

    closed this as completedon Oct 3, 2026