Use Browser

---
name: use-browser
description: Do something on a website in a real browser on the box — click, type, fill a form, search, check a page, add to a cart, star a repo, draft a reply. Use for "go to <site> and …", "buy/book/fill/submit/check … on <site>", "open <url> and …", or anything a normal fetch cannot do (JavaScript pages, clicks, forms, logged-in pages). Works with NOTHING connected; connecting a site (the connect-site skill) is only for pages where it must be the owner's account.
---

# use-browser — snapshot, act by ref, verify; the owner says yes to pay/post/send/delete

`5dive browser` drives a real Chrome on this box. Give any page verb a **URL** and it picks
the profile itself: the host's one connected login, or the **public profile** (nothing logged
in) when there is none. A box with zero connected sites can still browse and act on any public
page. Connecting a site is only for the owner's own accounts — their inbox, their cart, their
repo — and that is the `connect-site` skill, not this one.

## The loop: ONE snapshot per decision

```bash
5dive browser snapshot <url> --interactive     # the refs you can act on
5dive browser act <url> --steps='[{"op":"click","selector":"ref=button/Star"}]'
5dive browser act <url> --steps='[{"op":"fill","selector":"ref=textbox/Title","value":"Bug: …"},
                                  {"op":"click","selector":"ref=button/Save draft"}]' --expect='Draft saved'
```

1. **Snapshot** the page with `--interactive` and pick refs from what it printed. Never invent a
   ref; a page you have not snapshotted is a page you are guessing about. A web app (Gmail) shows
   a loading screen first: add `--wait-for='[role=main]'` (or `text=<words>` the loaded page
   has). **Exit 76 is not the page** — a loading screen or a `--wait-for` that never came; run it
   again with a `--wait-for`, and do not ask anyone to log in.
2. **Act** on those refs. Steps are `goto fill type click wait_for select press`, run in order,
   in one tab under one lease. Without a URL, `act` continues on the page a served browser holds.
   Use `type` for search boxes and autocompletes that react to keystrokes, and `fill` for plain
   inputs: `fill` sets the value with no key presses, so a suggestion list never opens. `type`
   goes key by key (`"delay_ms"`, default 50); pick the suggestion with a `wait_for` and a
   `click`.
3. **Verify.** `--expect=<regex>` grades the page as the steps left it, re-read for up to 5 s so
   a toast counts. Without it, `act` only says the steps ran — open the `page.png` it wrote
   before you tell anyone it worked.

`read`, `links` and `shot` take a URL the same way when you only need the text, the links or a
picture of a page.

## Doing it again: record once, replay without snapshots

A flow you will repeat (a daily post, a check, a clicker) costs a snapshot and a decision per
page every time. Record it the first time, then replay it in one call:

```bash
5dive browser act <url> --steps='[…]' --expect='…' --record=daily-post   # each act of run 1
5dive browser replay <site> daily-post --values='["today'"'"'s text"]'     # run 2, 3, …: no snapshots
5dive browser routine ls            # what is recorded; `routine show <site> <name>` for the steps
```

- `--record=<name>` keeps an act's steps **only when it succeeded**, and **never the values you
  typed**: each fill/type/select value becomes a numbered slot that `replay --values` fills in
  order. Add `--expect` when you record, so the replay re-checks each page.
- `replay` runs every act from the cache with no snapshot to read. If the site renamed a button,
  it re-picks it once and rewrites the routine. If it cannot, it stops (exit 1), names the act,
  and tells you to `routine forget <site> <name> --from=<k>` and re-record from there with a
  snapshot. A pay step stops a replay for the owner exactly as it stops an act (exit 73).
- Read the last `page.png` a replay leaves before you report success, as after any act.

**Re-snapshotting the same page?** Add `--delta` after the first full snapshot: it prints only
the refs and text that changed since your last snapshot of that site, and does not write
`page.png` when under 1% of it moved (the picture you have is current). A new page, or a change
bigger than the page, comes back full.

## Input mode: sites that refuse an automated browser (tiktok.com)

Some sites will not render for a browser under automation control. For those, the site's
adapter says `"drive": "input"`, and the browser is **plain Chrome, driven the way a person
drives it**: you read the screen and act through a real keyboard and mouse. There is no
automation channel, so to the page it is indistinguishable from the owner's own window.
tiktok.com is the first such site. A box can also make input mode the **default** for every
logged-in site (`5dive browser config` shows it; root sets `drive=input`), and then this is how
you work on all of them, unless a site's adapter says `"drive": "cdp"`. The verbs tell you when a
site is in input mode: `read`, `tree` or `run` refused "INPUT mode" means use the screen.

```bash
5dive browser shot tiktok.com https://www.tiktok.com/@someone   # a PNG of the live window (1280x800) + its title
5dive browser shot tiktok.com                                     # the window as it is now, no navigation
5dive browser act tiktok.com --steps='[{"op":"click","x":922,"y":674},
                                       {"op":"type","value":"hello"},{"op":"press","key":"Enter"}]'
```

1. **Look first.** Every decision starts from a `shot` (or the `page.png` the last `act` left).
   The coordinates are pixels of that PNG. The address bar is in the picture, so you can see
   which page you are on.
2. **Act in small steps**: `click{x,y[,button,count]}`, `move{x,y}`, `type{value}` (into
   whatever has focus), `press{key}` (`Enter`, `Tab`, `Escape`, `PageDown`, `ctrl+l`, …),
   `scroll{dy}` (wheel notches, + is down), `goto{url}` (typed into the address bar),
   `wait{ms}`. Then read the `page.png` it wrote before the next act. A click that opens a new
   page ends that act: type into the new page in the next one, after you have seen it. A click
   on a spot another window covers, or keys that cannot reach the browser, fail the step (exit
   1) with nothing sent past it. Read the error and the screenshot; do not repeat the same step
   blind.
3. **There is no DOM here.** `tree`, `read`, `links`, adapter `run` actions and `selector`/`ref`
   steps are refused, by name. Do not ask for them; use the screen.
4. **Say what a step does when it matters.** Nothing here can read a button's label, so a step
   that pays, publishes, sends or deletes must carry `"kind":"pay"` (or `publish`, `send`,
   `delete`). The owner's policy applies to what you declare: a kind they set to `ask` (pay, by
   default) stops with exit 73 before anything runs. Never leave the kind off to get past that.
   An undeclared purchase is still a purchase.

**Hand the window to the owner whenever a person is needed.** A captcha or slider puzzle, a
verification code, an "unusual activity" page, or a judgement call that is theirs:

```bash
5dive browser handoff tiktok.com --reason="solve the slider puzzle"   # they get a Connect button
5dive browser handoff tiktok.com --wait                               # returns when they are done
```

They open the **same window** you were driving. Your input is refused while they have it, and
when they tap Done you carry on from the page they left. Take a `shot` before your next act.
A challenge whose title says so stops an act by itself (exit 75). TikTok's puzzle does not change
the title, so when you see one in a screenshot, hand the window over. Never try to solve it.

## Pages that need the owner's login

A page verb refuses a page that is visibly a sign-in form or a challenge, and never hands one
back as evidence. On a site with no adapter it proceeds otherwise, and says that no adapter
confirmed the login. If you hit the sign-in refusal on a page that must be the owner's account,
switch to the `connect-site` skill. Never type the owner's password yourself, and never try to
get past a CAPTCHA or other challenge.
An authenticator-app 2FA prompt is the one exception, and only on a site whose owner saved its
seed on the box: `run` types the code itself, and `5dive browser totp fill <site> [<url>]` does
it on demand. Neither shows you the code. With no seed saved, it stops like any challenge.

**Blocked from a server IP** ("Request blocked by network security", "suspicious network"):
some sites block server IPs; `5dive browser proxy set <url>` sends this box's browser through
your own proxy. That is the owner's proxy and the owner's call — tell them, and use only a URL
they give you. Switching the proxy mid-session can end a site's login (the site sees a new IP).

**Two accounts on one site** (`github.com_work`, `github.com_personal`): a URL alone is refused
and the refusal names both. **Ask the owner which one, never guess**, then name it:
`5dive browser act github.com_work <url> --steps=…`.

## Stuck on a captcha: ask the owner to clear it, then carry on (DIVE-5200)

A CAPTCHA, a "verify you are human" or a "Just a moment…" check is a hard stop. You never
solve it and never look for a way round it. But you do not give up the task either: ask the
person, wait, and finish the job yourself.

```bash
5dive browser connect-request <site> --challenge --url=<the page you were stopped on> \
  --reason="<one line: what you were doing>"
```

`<site>` is the host (`futuretools.io`), even if you were browsing with nothing connected.
A page verb that hits a check prints this exact command for you. Your paired owner gets
"I'm stuck on a captcha on <site>" with an **Open <site>** button; their tap opens the box
browser on that page and sends them a one-time link as code, from root, never through you.
They clear the check and tap **Done**. Done closes their view and **hands the same browser
back to you, on that page**, so the check stays cleared for you.

After the `[browser challenge] … tapped Done` message, carry on **without asking again**:

1. First check what already happened. If the step you were on may have gone through,
   read the result (the listing, the sent folder, the confirmation page) before you redo it.
   Nothing is sent twice.
2. Re-read the page you were stopped on (`5dive browser snapshot <url>`). It goes through
   the browser profile the owner just used.
3. Clear: continue from that step. Still a check: say so once and stop.

Do not poll while you wait. Do not open the link. A flat block with nothing to click
("access denied", a 403 from a server IP) is not a check a person can clear; see
"Blocked from a server IP".

## Paying, posting, sending and deleting are the owner's call

**The owner's default is standard** (DIVE-5148): `act` runs post, send and delete steps without
stopping, says `ALLOWED (default standard)`, and logs each one for the owner with what it sent and
a screenshot — but a **pay step stops and asks them** (exit 73), because a post or a delete can be
undone and money cannot. `5dive browser approvals policy` shows the policy and its `mode`
(`standard`, `yolo`, `careful` or `custom`). Changing it is the owner's, not yours:
`sudo 5dive browser approvals policy set careful` puts the stop on all four,
`set yolo` takes it off all four, `set standard` is this default,
`set <kind>=ask|allow` changes one kind, and from your seat all of them are refused.

Where a kind is `ask` (`pay` by default; the owner chose `careful`, or that kind), `act` stops in front of any such
button (read off the live page, whatever selector you used; Ctrl/Cmd+Enter counts as send, and so
does a plain Enter in a composer with no form around it, which is how a chat box sends) and
exits **73** with the ask, a screenshot and an approval id.

- Relay the ask to the owner **with the screenshot**, in plain words: what will be bought,
  posted, sent or deleted, and where.
- Relay the ask's payload (who it goes to, what it says, the amount, the item), not the button's
  name. If the ask says the page showed none, say that and send the screenshot.
- Only on their explicit yes is it approved: `sudo 5dive browser approve <id>`, which the owner
  or their dashboard runs, or their Approve button on Telegram — not you (from a seat it is
  refused). Then re-run the SAME act with `--approved=<id>`. A yes covers exactly those steps,
  once, for 30 minutes.
- **Do not rephrase the steps to get around the stop.** A button renamed is still an order
  placed. The check reads literal button labels, so a purchase behind a button labelled
  "Continue" is not caught by it — under `careful` that one is on you: if a step will pay, post,
  send or delete, ask first even when `act` does not stop you.

## From real jobs

Lessons from browser jobs that went wrong on real sites (DIVE-5390). One line each:

- **Read the saved record back.** After a submit, open what the site saved and check every field landed where you meant: a URL can land in the name field while the page says "success".
- **Look at the screenshot after a submit**, not only the exit code. A success line can show for a moment and the form reset under it.
- **One run per form, every field in it.** Tabs keep page state between runs, so clear a pre-filled form and re-check it before you type.
- **Never press a site's autofill or AI-assist button.** It writes text you were not given.
- **A bot check can appear mid-flow**, after a sign-in or submit button, not only on page load. It is still a hard stop.
- **Only a real browser render proves a page has no bot check.** A plain fetch of the page cannot see one.
- **Patterns:** `--expect` already ignores case, so never put `(?i)` in it (grep -E rejects it). `--wait-for='text=<words>'` matches plain words in any case. A regex works only in a `wait_for` step's selector, as `text=/word/i`.
- **A form that only says "thanks"** (Tally and the like) leaves no record to read back: report "submitted, not live".
- **Served is not signed in.** Before you hand off work that needs the login, check `5dive browser status <site>` reads authenticated.
- **Input mode: Chrome's "Restore pages?" bubble can cover the header's sign-in link or avatar.** Close it and take a new `shot` before you judge signed in or out, and check the URL bar in the screenshot shows the page you meant.