Repository navigation
fix(browser): keep the Annotate note box out of the page's reach (ATO-1543) - #1608
Draft
sosidudku1 wants to merge 1 commit into
Draft
sosidudku1 wants to merge 1 commit into
sosidudku1 wants to merge 1 commit into
Conversation
…-1543) The annotate overlay had a comment box inside the page. Key and input events in the page's document are composed, so a page script listening on window in the capture phase saw every key the user typed there, even with the overlay in an isolated world and a closed shadow root. The overlay now only picks elements: a click drops a numbered pin at once, nothing on the page takes focus or listens to keys. The user says what to change for each pin number in the agent's chat, and the tool results tell the model to pair pins with those chat messages (and to ask when a pin has no words yet). Pins no longer carry a comment field.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: ATO-1543 (follow-up to ATO-518 / #1510)
What leaked
Annotate mode put a small comment box on the page itself. The overlay runs in an isolated world inside a closed shadow root, so the page cannot read the saved pins. But typing is different: key and input events in the page's document are composed, and a page script listening on
windowin the capture phase runs before anything in the overlay can stop them. So any site could read every key the user typed into the note box. #1510 only added a "don't type passwords here" line.What changed
browser.annotateresults say so:starttells the model to ask the user to write per-pin notes in the chat, not on the page;readreturns pins without text and tells the model to pair pin numbers with the user's chat messages and to ask when a pin has no words yet.commentfield; anything a script sends back under that name is dropped.Precedent
Same split as Playwright codegen's element picker and Claude in Chrome: pick the element in the page, take the text in the host's UI. I looked at keeping the box in the page with an isolated world plus stopping propagation, and it does not work: a page capture listener on
windowregistered earlier always runs first.Tests
annotate-in-page.test.ts: the start script has no textarea, input, contentEditable,.focus(or key listeners, and builds no such element; a real click on a page element drops a numbered pin with no comment; a repeat click on the same element and a synthetic (untrusted) click add nothing; read returns pins withoutcomment.annotate-pins.test.ts: pins without a comment are kept; acommentfield from a script is dropped and never rendered.annotate.test.ts:startsends the user to the chat;readrenders pins as[n] <tag> "text"and points the model at the chat messages.Not yet checked on the second Mac.