Browser

Enabling it

Browser access is off by default. Open the employee, go to Settings → Browser, and flip on Browser access. The same card holds the two shaping controls: the allow list (which hosts the employee may open) and require approval for form submits (a human-in-the-loop gate on anything that sends data). A Routine can override the toggle per schedule — useful for an employee who may browse during a nightly research run but not in ad-hoc chat.

The tools

When enabled, the employee's tool list grows by the browser set. Every action returns a fresh snapshot of the page, so the employee always acts on current state:

browser_open

Navigate to a URL (gated by the allow list) and snapshot the loaded page.

browser_snapshot

Re-read the current page without acting — e.g. at the start of a new turn.

browser_click

Click an element. If the click opens a new tab, the browser follows it automatically.

browser_fill

Type into an input or textarea, replacing its contents.

browser_select

Choose an option in a native dropdown by value or visible label.

browser_press

Press a key — Enter, Tab, Escape, arrows.

browser_hover

Hover to reveal menus or tooltips; the hover holds so a follow-up click works.

browser_scroll

Scroll by a viewport (fires real wheel events, so lazy-loaded and infinite-scroll pages load more content) or bring a specific element into view.

browser_back

Go back one page in history — the recovery move after a misclick.

browser_wait

Wait for a selector to appear (up to 15s) or pause a fixed time, instead of polling with snapshots.

browser_screenshot

A JPEG of the viewport, for when layout or imagery matters.

browser_submit

Submit a form. With approval mode on, queues an Approval instead of firing.

browser_resume

Re-fire an approved submit — in the same turn or any later one.

browser_close

Shut the browser down (skipped while a human is watching the live view).

Snapshots and refs

A snapshot is a YAML outline of the page in which every interactive element carries a stable marker like [ref=e12] — including elements inside iframes. The employee acts on a ref directly by passing aria-ref=e12 as the selector, which resolves instantly and unambiguously; CSS and text selectors work too as fallbacks. The outline covers the whole page, not just the viewport; on very large pages it is capped with a note saying how many elements were omitted, and the employee narrows down by interacting with a section or navigating to a more specific URL.

Events the employee could not otherwise see — a JavaScript dialog that was auto-dismissed, a popup tab that was adopted, a selector that matched more than one element — are surfaced as NOTE: lines at the top of the next snapshot.

The allow list

One host pattern per line; blank means unrestricted. Lines starting with # are comments. Matching rules:

• mail.google.com — that exact host, and nothing else. Use this to pin a single host.
• *.github.com — the apex github.com and every subdomain (www.github.com, gist.github.com, …).
• app.*.example.com — a glob; each * spans a single label and never crosses a dot.

The list is enforced server-side on every browser_open, so edits apply immediately — no restart needed.

Approval-gated submits

With require approval for form submits on, a browser_submit does not fire. It queues an Approval — visible in the company Approvals inbox with the page URL and a one-line summary of what the employee is trying to do — and the employee is told the submission is pending. Once you approve, the employee re-fires it with browser_resume, in the same turn or a later one. The approval is bound to the page it was raised on and fires exactly once: if the browser has moved to a different page (or was reclaimed while idle) the employee is asked to submit again rather than firing blindly against whatever is now loaded. Rejecting writes the decision to the employee's journal.

Live view and takeover

While the employee browses, the chat panel shows the page live. Click Take over to drive it yourself — your mouse and keyboard go straight to the same Chromium. That is the intended flow for credentials, captchas, and 2FA: the employee navigates to the login page, you type the secret, the employee carries on. The browser is never torn down while someone is watching.

What persists

The browser outlives individual chat turns — "I'll wait while you sign in" genuinely works, and an idle browser is reclaimed after five minutes once nobody is using it. Cookies and local storage are snapshotted per employee under the company data directory, so a login survives new conversations and container restarts. Model credentials are never involved; see Configuration for where data lives on disk.