agent-browser skill by andreadellacorte

Browser Automation with agent-browser

Core Workflow

Every browser automation follows this pattern:

Navigate: agent-browser open <url>
Snapshot: agent-browser snapshot -i (get element refs like @e1, @e2)
Interact: Use refs to click, fill, select
Re-snapshot: After navigation or DOM changes, get fresh refs

agent-browser open https://example.com/form
agent-browser snapshot -i
# Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"
 
agent-browser fill @e1 "user@example.com"

Command Chaining

Commands can be chained with && in a single shell invocation. The browser persists between commands via a background daemon, so chaining is safe and more efficient than separate calls.

When to chain: Use && when you don't need to read the output of an intermediate command before proceeding (e.g., open + wait + screenshot). Run commands separately when you need to parse the output first (e.g., snapshot to discover refs, then interact using those refs).

Essential Commands

Common Patterns

Form Submission

Authentication with Auth Vault (Recommended)

Authentication with State Persistence

Session Persistence

Data Extraction

Parallel Sessions

Connect to Existing Chrome

Color Scheme (Dark Mode)

Visual Browser (Debugging)

Local Files (PDFs, HTML)

iOS Simulator (Mobile Safari)

Requirements: macOS with Xcode, Appium (npm install -g appium && appium driver install xcuitest)

Real devices: Works with physical iOS devices if pre-configured. Use --device "<UDID>" where UDID is from xcrun xctrace list devices.

Security

All security features are opt-in. By default, agent-browser imposes no restrictions on navigation, actions, or output.

Content Boundaries (Recommended for AI Agents)

Enable --content-boundaries to wrap page-sourced output in markers that help LLMs distinguish tool output from untrusted page content:

Domain Allowlist

Restrict navigation to trusted domains. Wildcards like *.example.com also match the bare domain example.com. Sub-resource requests, WebSocket, and EventSource connections to non-allowed domains are also blocked. Include CDN domains your target pages depend on:

Action Policy

Use a policy file to gate destructive actions:

Example policy.json:

Auth vault operations (auth login, etc.) bypass action policy but domain allowlist still applies.

Output Limits

Prevent context flooding from large pages:

Diffing (Verifying Changes)

Use diff snapshot after performing an action to verify it had the intended effect. This compares the current accessibility tree against the last snapshot taken in the session.

For visual regression testing or monitoring:

diff snapshot output uses + for additions and - for removals, similar to git diff. diff screenshot produces a diff image with changed pixels highlighted in red, plus a mismatch percentage.

Timeouts and Slow Pages

The default Playwright timeout is 25 seconds for local browsers. This can be overridden with the AGENT_BROWSER_DEFAULT_TIMEOUT environment variable (value in milliseconds). For slow websites or large pages, use explicit waits instead of relying on the default timeout:

When dealing with consistently slow websites, use wait --load networkidle after open to ensure the page is fully loaded before taking a snapshot. If a specific element is slow to render, wait for it directly with wait <selector> or wait @ref.

Session Management and Cleanup

When running multiple agents or automations concurrently, always use named sessions to avoid conflicts:

Always close your browser session when done to avoid leaked processes:

If a previous session was not closed properly, the daemon may still be running. Use agent-browser close to clean it up before starting new work.

Ref Lifecycle (Important)

Refs (@e1, @e2, etc.) are invalidated when the page changes. Always re-snapshot after:

Clicking links or buttons that navigate
Form submissions
Dynamic content loading (dropdowns, modals)

Annotated Screenshots (Vision Mode)

Use --annotate to take a screenshot with numbered labels overlaid on interactive elements. Each label [N] maps to ref @eN. This also caches refs, so you can interact with elements immediately without a separate snapshot.

Use annotated screenshots when:

The page has unlabeled icon buttons or visual-only elements
You need to verify visual layout or styling
Canvas or chart elements are present (invisible to text snapshots)
You need spatial reasoning about element positions

Semantic Locators (Alternative to Refs)

When refs are unavailable or unreliable, use semantic locators:

JavaScript Evaluation (eval)

Use eval to run JavaScript in the browser context. Shell quoting can corrupt complex expressions -- use --stdin or -b to avoid issues.

Why this matters: When the shell processes your command, inner double quotes, ! characters (history expansion), backticks, and $() can all corrupt the JavaScript before it reaches agent-browser. The --stdin and -b flags bypass shell interpretation entirely.

Rules of thumb:

Single-line, no nested quotes -> regular eval 'expression' with single quotes is fine
Nested quotes, arrow functions, template literals, or multiline -> use eval --stdin <<'EVALEOF'
Programmatic/generated scripts -> use eval -b with base64

Configuration File

Create agent-browser.json in the project root for persistent settings:

Priority (lowest to highest): ~/.agent-browser/config.json < ./agent-browser.json < env vars < CLI flags. Use --config <path> or AGENT_BROWSER_CONFIG env var for a custom config file (exits with error if missing/invalid). All CLI options map to camelCase keys (e.g., --executable-path -> "executablePath"). Boolean flags accept true/false values (e.g., --headed false overrides config). Extensions from user and project configs are merged, not replaced.

Deep-Dive Documentation

Reference	When to Use
references/commands.md	Full command reference with all options
references/snapshot-refs.md	Ref lifecycle, invalidation rules, troubleshooting
references/session-management.md	Parallel sessions, state persistence, concurrent scraping
references/authentication.md	Login flows, OAuth, 2FA handling, state reuse
references/video-recording.md	Recording workflows for debugging and documentation
references/profiling.md	Chrome DevTools profiling for performance analysis
references/proxy-support.md	Proxy configuration, geo-testing, rotating proxies

Ready-to-Use Templates

Template	Description
templates/form-automation.sh	Form filling with validation
templates/authenticated-session.sh	Login once, reuse state
templates/capture-workflow.sh	Content extraction with screenshots