Skip to content
Back to skills

Argent Device Interact

ASecurity

Interact with an iOS simulator or iPhone, Android emulator or phone, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.

  • 3,211 stars
  • 0 votes
  • 0 copies
  • 0 views
  • Added November 18, 2026
developmentgoreacttesting

Works with

  • mcp

Security analysis

A100/100

Pro scans all 4 files and shows the line behind each finding

Scanned November 18, 2026

npx -y skills add software-mansion/argent --skill argent-device-interact --agent claude-code

Installs into .claude/skills of the current project.

Are you the author of Argent Device Interact?

Add the live security badge to your README. It updates with every re-scan.

Security grade badge for Argent Device Interact
[![Security: A — Skills Directory](https://www.skillsdirectory.com/api/skills/software-mansion-argent-device-interact/badge)](https://www.skillsdirectory.com/skills/software-mansion-argent-device-interact)

More formats (shields.io, HTML) on the badges page. Keep it an A: scan every change in CI with Pro.

Download with Pro
SKILL.md
---
name: argent-device-interact
description: Interact with an iOS simulator or iPhone, Android emulator or phone, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.
---

Each tool's own description carries its parameters, defaults, return shape and examples. This skill covers choosing between tools and the gotchas those descriptions don't state.

## 1. Before You Start

- Get a target id from `list-devices`; if none is ready, `boot-device` one. Full setup flow: `argent-ios-simulator-setup` / `argent-ios-device-setup` / `argent-android-emulator-setup`. The same tool names drive iOS, Android and Chromium; each takes the `udid` from `list-devices`.
- Physical iPhone (`kind: "device"`): read `argent-ios-device-interact` first. Only the tools it lists exist there, and `describe` sees only the app `launch-app` registered.
- Mac app (`platform: "macos"`): read `argent-macos-app-interact` first.
- Read the matching reference before driving these targets: `references/chromium.md` for a `chromium` target, `references/harmonyos.md` for a `harmony` target, `references/foldables.md` for a device `list-devices` marks `foldable: true` (or before calling `fold`).
- Load a deferred tool's schema with ToolSearch before its first call (e.g. `gesture-pinch`, `gesture-rotate`, `gesture-custom`); without it, numbers go out as strings and fail validation.
- If you delegate device tasks to sub-agents, make sure they have MCP permissions.

## 2. Choosing the Right Tool

| Action                         | Tool                | Notes                                                          |
| ------------------------------ | ------------------- | -------------------------------------------------------------- |
| Several known steps            | `run-sequence`      | One call, one screenshot at the end (§5)                       |
| Open an app                    | `launch-app`        | **Never tap home-screen icons**                                |
| Restart an app                 | `restart-app`       | Clean in-memory state                                          |
| Open URL / deep link           | `open-url`          | Web pages, deep links, URL schemes                             |
| Tap                            | `gesture-tap`       | Buttons, links, checkboxes                                     |
| Scroll / swipe                 | `gesture-swipe`     | Touch targets; not Chromium                                    |
| Scroll (Chromium)              | `gesture-scroll`    | Mouse wheel                                                    |
| Drag (Chromium)                | `gesture-drag`      | Sliders, drag-and-drop, text selection                         |
| Long press, drag & drop        | `gesture-custom`    | Also any other non-linear touch sequence                       |
| Pinch / zoom                   | `gesture-pinch`     |                                                                |
| Two-finger rotation            | `gesture-rotate`    |                                                                |
| Hardware key                   | `button`            | Home, back, power, volume, appSwitch, actionButton             |
| Type text or press a named key | `keyboard`          | Every platform; text or one key per call (§6)                  |
| Paste text                     | `paste`             | Only where a user would paste (§6). Sim/emu, Mac app           |
| Rotate device                  | `rotate`            | Orientation changes                                            |
| Fold device                    | `fold`              | Foldables only: `references/foldables.md`                      |
| Shake device                   | `shake`             | Shake handlers (sim/emu only), Undo-typing prompt, RN dev menu |
| Wait for UI                    | `await-ui-element`  | Block until an element is visible/hidden/exists/contains text  |
| Wait for idle                  | `await-screen-idle` | Block until a non-empty screen tree stops changing (§4)        |

## 3. Finding Tap Targets

After an action moves you to a different screen, or whenever you don't know an element's coordinates, discover before tapping. Interaction results already carry the element tree (`--- Elements after action (describe) ---`): use it, and call a discovery tool only when no fresh tree covers the current screen. Each tree has a `Tree id`. When the tree is identical to an earlier one, the result names that id in place of the tree: use that tree, and call `describe` if it is not in your context. An identical tree does not mean the action did nothing.

| Need                                           | Tool                                                                                                                           |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| Any screen: app, system dialog, Home screen    | `describe`                                                                                                                     |
| UIKit properties of a view (iOS simulator)     | `native-find-views` (needs `bundleId` and a class, id, label, tag or `nativeID` to match)                                      |
| What sits at a known raw point (iOS simulator) | `native-view-at-point` (the visually deepest view; `hitTest: true` for the view that receives touch)                           |
| Final visual fallback                          | `screenshot`, only when the tools above cannot expose the control. Do not derive routine in-app navigation targets from pixels |

If `describe` returns an empty tree, follow its `hint`; with no hint, the screen is blank, loading or unlabeled: check the attached screenshot and retry once content has loaded.

For a React Native testID or component name that `describe` does not show, use `debugger-component-tree`; take tap points from `describe`.

**React Native:** LogBox toasts sit over the bottom of the screen and catch taps meant for what is under them. Argent hides them when a `debugger-*` tool connects and after `debugger-reload-metro`; after `restart-app`, call a `debugger-*` tool again. After any other reload they come back: close each by tapping the right end of its `describe` frame (the X), not its middle.

**System and permission dialogs:** on a simulator or emulator, `describe` returns a visible dialog's buttons with tap coordinates; keep using it for in-app modals too. Fall back to `screenshot` only if it does not expose the controls: then tap one obvious button (`Allow`, `OK`, `Don't Allow`, `Not Now`, `Continue`) at a time, check the returned screenshot, and go back to `describe` once the dialog is gone.

> **Prefer the dialog over the Settings tool.** When the app triggers its own permission prompt, answering it is the real user path. Use `settings-permissions` only when you can't make the change through the app: set a permission _before_ the app asks, re-enable one the user denied (iOS won't re-prompt), or reset it so the prompt reappears. See `argent-settings-permissions`.

## 4. Waiting: `await-ui-element` and `await-screen-idle`

**Never poll `screenshot`/`describe` in a loop to wait for something.** `await-ui-element` blocks server-side on the same tree `describe` reads. It has no bare-timer mode; for a plain pause, use your own harness sleep. Its description covers conditions, selector matching and defaults. What it does not tell you:

- A `hidden` check that succeeds **immediately** may be a false pass: its `note` then says the selector never matched anything. Treat that as a failed check and fix the selector; it does not mean "the element went away".
- The synthetic `ROOT` container `describe` prints is never matched, so a `role` like `AXGroup`/`html` won't trivially "match the screen".
- To disambiguate a loose selector, pin `role` to a text role like `StaticText`; that skips a same-named button.

`await-screen-idle` is for live diagnosis after a launch or navigation, before a raw tap on an element that may still be moving. Continue only when `settled: true`, and pair it with an `await-ui-element` for the destination: stillness does not identify a screen, and the tool can return during a presentation-layer animation. Do not record it in an e2e test; tests use `await: { idle: true }`, which also compares pixels.

## 5. Action Sequencing with `run-sequence`

Use `run-sequence` when every step is known in advance; if a step depends on what the previous one shows, use separate calls. Its description lists the allowed tools and their arguments. Typical batches:

- "scroll to bottom", "scroll until X": 3-5 swipes
- "clear and retype a field": triple-tap to select all, then type the new value
- typing then submitting: a `keyboard` text step, then a `keyboard` `enter` step (one call cannot carry both)
- "submit form": fill every field, then tap submit
- "go back to X": a known tap path

Gate a step that depends on a screen transition with an `await-ui-element` step rather than a fixed `delayMs`. If its condition is not met before the timeout, the sequence **stops there** and the later steps do not run, so a tap can't fire against a screen that never appeared. Any failing step also stops the sequence, with partial results.

## 6. Text Entry and Secrets

- Tap the field first. Typing or pasting with no focused field is a silent no-op.
- `paste` is **not** a faster `keyboard`. `keyboard` types the way a user types and stays the default for every text entry. Use `paste` only where a real user would paste: a 2FA / OTP code from another app, a long link or token, or testing how the app handles pasted input. That it can carry text `keyboard` can't type on a platform (multi-line text, non-ASCII on Android) is not a reason on its own.
- **Credentials:** type them as a `{{secret:<NAME>}}` placeholder in `text` (`keyboard`, `paste`, `run-sequence` keyboard steps, e2e `type` steps), never as plaintext, and never ask the user for the value. Do **not** `describe` or `screenshot` a non-secure field you just filled with a secret: submit or navigate away first, then verify the resulting screen.

## 7. Screenshots

Interaction tools already attach a screenshot. Call `screenshot` for the initial state, for a baseline before you change visible UI, to check state after a delay, or when the attached capture shows a transitional or loading frame.

Keep the default downscaled capture for UI context and state checks; do not use `scale: 1.0` as a readability or tapping aid. Full resolution is only for saving baseline/current PNGs for comparison (`scale: 1.0` with `includeImageInContext: false`); for visual regression checks use the `argent-screenshot-diff` skill.

If `screenshot` times out, restart the device transport with `stop-simulator-server`.

## 8. Android notes

- **First-launch permission prompts**: `reinstall-app` on Android installs with `-g`, so runtime permissions are pre-granted on first launch.

Files in this skill

  • SKILL.md11.1 KB
  • references/chromium.md1004 B
  • references/foldables.md811 B
  • references/harmonyos.md8.5 KB

Attribution

Is this your skill, or is something wrong with this listing? Request removal or report an issue. Author removals are honored within 72 hours.

View all in development →

Comments

Loading comments…

Newsletter

Get the weekly list of safe new skills, and the ones we pulled.