Scrape

---
name: scrape
description: Pull data from a web page. First call on a new intent prototypes the
  flow via $B primitives and returns JSON. Subsequent calls on a matching intent route
  to a codified browser-skill and return in ~200ms. Read-only — for mutating flows
  (form fills, clicks, submissions), use /automate. Use when asked to "scrape", "get
  data from", "pull", "extract from", or "what's on" a page. (gstack)
---

## Skill routing

When the user's request matches an available skill, invoke it via the Skill tool. When in doubt, invoke the skill.

Key routing rules:
- Product ideas/brainstorming → invoke /office-hours
- Strategy/scope → invoke /plan-ceo-review
- Architecture → invoke /plan-eng-review
- Design system/plan review → invoke /design-consultation or /plan-design-review
- Full review pipeline → invoke /autoplan
- Bugs/errors → invoke /investigate
- QA/testing site behavior → invoke /qa or /qa-only
- Code review/diff check → invoke /review
- Visual polish → invoke /design-review
- Ship/deploy/PR → invoke /ship or /land-and-deploy
- Save progress → invoke /context-save
- Resume context → invoke /context-restore
```

Then commit the change: `git add CLAUDE.md && git commit -m "chore: add gstack skill routing rules to CLAUDE.md"`

If B: run `~/.claude/skills/gstack/bin/gstack-config set routing_declined true` and say they can re-enable with `gstack-config set routing_declined false`.

This only happens once per project. Skip if `HAS_ROUTING` is `yes` or `ROUTING_DECLINED` is `true`.

If `VENDORED_GSTACK` is `yes`, warn once via AskUserQuestion unless `~/.gstack/.vendoring-warned-$SLUG` exists:

> This project has gstack vendored in `.claude/skills/gstack/`. Vendoring is deprecated.
> Migrate to team mode?

Options:
- A) Yes, migrate to team mode now
- B) No, I'll handle it myself

If A:
1. Run `git rm -r .claude/skills/gstack/`
2. Run `echo '.claude/skills/gstack/' >> .gitignore`
3. Run `~/.claude/skills/gstack/bin/gstack-team-init required` (or `optional`)
4. Run `git add .claude/ .gitignore CLAUDE.md && git commit -m "chore: migrate gstack from vendored to team mode"`
5. Tell the user: "Done. Each developer now runs: `cd ~/.claude/skills/gstack && ./setup --team`"

If B: say "OK, you're on your own to keep the vendored copy up to date."

Always run (regardless of choice):
```bash
eval "$(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)" 2>/dev/null || true
touch ~/.gstack/.vendoring-warned-${SLUG:-unknown}
```

If marker exists, skip.

If `SPAWNED_SESSION` is `"true"`, you are running inside a session spawned by an
AI orchestrator (e.g., OpenClaw). In spawned sessions:
- Do NOT use AskUserQuestion for interactive prompts. Auto-choose the recommended option.
- Do NOT run upgrade checks, telemetry prompts, routing injection, or lake intro.
- Focus on completing the task and reporting results via prose output.
- End with a completion report: what shipped, decisions made, anything uncertain.

## Step 1 — Determine intent

The user's request after `/scrape` is the intent. If they did not include
one, ask once:

> "What do you want to scrape? Describe it in one line, e.g. 'top stories
> on Hacker News' or 'product names + prices on example.com/products'."

Do not ask multiple clarifying questions up front. Any further questions
go in the prototype path where they're cheaper.

## Step 2 — Refuse mutating intents

If the intent implies writes — verbs like *submit*, *post*, *send*, *log
in*, *click X*, *fill the form*, *delete*, *create*, *order*, *book* —
respond:

> "/scrape is read-only. For mutating flows, use /automate (browser-skills
> Phase 2 P0 in TODOS.md — not yet shipped). Until then, use $B click /
> $B fill / $B type directly."

Stop. Do not enter the match or prototype path.

## Step 3 — Match phase

List existing browser-skills:

```bash
$B skill list
```

For each skill, `$B skill show <name>` exposes the full SKILL.md including
`triggers:`, `description:`, and `host:`. Read these and judge whether the
user's intent semantically matches one of them.

A confident match means **all three** are true:

- The intent's domain matches the skill's `host` (or one of its hostnames)
- A `triggers:` phrase or the `description:` covers the same data the
  intent asks for
- The intent does not require args the skill does not declare in `args:`

If matched, parse any `--arg key=value` from the intent (or pass none for
zero-arg skills) and run:

```bash
$B skill run <name> [--arg key=value ...]
```

Emit the JSON the skill prints to stdout. Stop.

If matching is ambiguous (two skills could plausibly fit), pick the
narrower-tier one (project > global > bundled — `$B skill list` shows the
tier). If still ambiguous, fall through to the prototype path rather than
guess wrong.

## Step 4 — Prototype phase

No match. Drive the page using `$B` primitives:

1. `$B goto <url>` — navigate to the target. The user's intent usually
   names a host or a URL; use it directly.
2. `$B snapshot --text` (or `$B text`) — get a clean text view of the
   page to find selectors.
3. `$B html` — pull the raw HTML when you need to parse structured data
   (lists, tables, repeated rows).
4. `$B links` — when the intent is to gather URLs.
5. Iterate: try a selector, check the output, refine.

Emit the result as JSON on stdout (one document, not pretty-printed).
Use a stable shape — typically `{ "items": [...], "count": N }` or
similar — so downstream consumers can treat it as data.

## Step 5 — Skillify nudge

After a successful prototype, append exactly one line:

> "Say /skillify to make this a permanent skill (200ms on next call)."

That is the entire nudge. Do not nag, do not list pros, do not push.
Proactive surfacing is a Phase 3 knob (`gstack-config browser_skillify_prompts`),
not this skill's job.

## When the prototype fails

If the page loads but data extraction does not yield a sensible JSON shape
after 3-4 selector attempts:

- Report what you tried, what came back, and what's blocking (lazy-loaded,
  JS-rendered, paywalled, etc.).
- Do NOT write a partial result and call it done.
- Do NOT suggest /skillify on a broken prototype.
- Ask the user whether they want to (a) try a different selector, (b)
  switch to a different page, or (c) stop.

## What this skill does NOT do

- Mutating actions (use /automate when shipped, or $B primitives directly)
- Auth flows / cookie import (use /setup-browser-cookies first)
- Multi-page crawls (this is one-shot per call)
- Anything that requires the daemon to not be running

## Output discipline

The match path returns whatever JSON the matched skill emits. The
prototype path returns whatever JSON you construct. In both cases:

- One JSON document, on stdout.
- Stderr (or chat) is for logs and the skillify nudge.
- Do not embed prose around the JSON in the chat reply unless the user
  asked for an explanation — many `/scrape` callers pipe the output to
  `jq`.

## Capture Learnings

If you discovered a non-obvious pattern, pitfall, or architectural insight during
this session, log it for future sessions:

```bash
~/.claude/skills/gstack/bin/gstack-learnings-log '{"skill":"scrape","type":"TYPE","key":"SHORT_KEY","insight":"DESCRIPTION","confidence":N,"source":"SOURCE","files":["path/to/relevant/file"]}'
```

**Types:** `pattern` (reusable approach), `pitfall` (what NOT to do), `preference`
(user stated), `architecture` (structural decision), `tool` (library/framework insight),
`operational` (project environment/CLI/workflow knowledge).

**Sources:** `observed` (you found this in the code), `user-stated` (user told you),
`inferred` (AI deduction), `cross-model` (both Claude and Codex agree).

**Confidence:** 1-10. Be honest. An observed pattern you verified in the code is 8-9.
An inference you're not sure about is 4-5. A user preference they explicitly stated is 10.

**files:** Include the specific file paths this learning references. This enables
staleness detection: if those files are later deleted, the learning can be flagged.

**Only log genuine discoveries.** Don't log obvious things. Don't log things the user
already knows. A good test: would this insight save time in a future session? If yes, log it.