CLI
WebSkrap installs the webskrap command.
Install browsers
webskrap install
webskrap install --format jsonwebskrap install downloads Chrome for Testing into the user cache folder.
--force downloads it again. Use JSON output in scripts.
Check environment
webskrap doctor
webskrap doctor --format jsonThe CLI fetch command runs headless Chrome, so doctor checks that headless
Chrome can launch. It also reports the host
timezone and warns when it is UTC, which pages read as a server.
List profiles
webskrap profiles
webskrap profiles --format jsonFetch a page
webskrap fetch https://example.comThe default human output prints status, final URL, title, and artifact paths.
For LLMs and shell automation, use JSON:
webskrap fetch https://example.com --format json --max-chars 12000JSON output includes url, final_url, status, ok, title, headers,
text, text_length, text_truncated, and elapsed_ms.
Wait for deferred content with a visible CSS selector or snapshot ref:
webskrap fetch https://example.com --ready-selector '#content' --format json--timeout-ms applies separately to navigation and this readiness wait, which
starts after consent dismissal. A missing selector exits 3 (timeout). Known
challenge and block pages (DataDome, Cloudflare, Akamai, AWS WAF) exit 11
(blocked) without retrying, even when served with HTTP 200. Ordinary HTTP errors still return ok: false.
Search the web
webskrap search "example domain"
webskrap search "example domain" --engine bing --max-results 5 --format jsonsearch loads Bing's results page (--engine bing, the default)
or DuckDuckGo's HTML page (--engine ddg) in the same stealth browser and
prints each hit's title, URL and snippet. The engine's click-tracking redirect is unwrapped, so the URL is the
destination. Google is not offered. JSON output carries query, engine,
hits, hits_total and hits_truncated.
A blocked failure (exit 11) means the engine served a bot challenge instead
of results. Switch engine, reuse --user-data-dir, or change the exit IP.
Nothing is retried and no CAPTCHA is solved.
Text and stdout
Print raw fetched content to stdout:
webskrap fetch https://example.com --stdoutReturn readable body text instead of HTML:
webskrap fetch https://example.com --stdout --text-only
webskrap fetch https://example.com --format json --text-onlySuppress the human summary when writing artifacts:
webskrap fetch https://example.com --quiet --output example.htmlScreenshot and output
webskrap fetch https://example.com --screenshot example.png
webskrap fetch https://example.com --output example.html
webskrap fetch https://example.com -o example.html --screenshot example.pngParent directories are created automatically for --output and --screenshot.
Wait and timeout
webskrap fetch https://example.com --wait-until load --timeout-ms 60000Supported wait states are commit, domcontentloaded, load, and
networkidle. Prefer domcontentloaded for fast HTML collection, load for
regular page assets, and networkidle only when a page actually needs a quiet
network.
Resource policy
webskrap fetch https://example.com --resource-policy lite
webskrap fetch https://example.com --resource-policy documents -o page.htmlThe fetch default is auto. With --text-only, it skips images, fonts and
media unless a screenshot is requested. HTML and screenshots load all resources.
Explicit policies take priority. lite blocks images, fonts and media.
documents also blocks stylesheets; use it only when removing CSS preserves
the required result.
Cookie banners
webskrap fetch clicks the reject control of a cookie consent notice before
reading the page, so banners do not bury the text and no optional cookies are
accepted. The human output prints Cookie notice: declined (...) when it fired,
and the JSON output carries cookie_notice_declined.
webskrap fetch https://example.com --no-decline-cookies
webskrap fetch https://example.com --decline-cookies-timeout-ms 4000The timeout is how long to wait for a late-injected notice, and the per-fetch
cost on pages that have none. Use 0 for a single immediate check.
Browser channels
--channel defaults to chrome, which does not exist on every platform (Linux
ARM64 has no Chrome build). When the channel cannot launch, fetch and
search print a notice on stderr and try installed Edge, then Chrome for
Testing (the full browser in new headless mode), so piped stdout stays clean. Sandbox failures stop without trying another channel.
If nothing launches you get a one-line error and Run: webskrap install, not a
traceback.
webskrap doctor follows the same Chrome, Edge, Chromium order. JSON output
names the working channel and browser_identity, the host
cpu_architecture, and font_count, the number of distinct Fontconfig
families. font_count is null where fc-list is unavailable. Human output
shows these values too.
Headless stealth controls
webskrap fetch always runs headless. Use real Chrome and a stable
profile directory when browser continuity matters:
webskrap fetch https://example.com \
--channel chrome \
--user-data-dir .webskrap/headless-profile \
--virtual-display--virtual-display runs Chrome headed on a private Xvfb display (Linux, Xvfb
installed), so no HeadlessChrome token or empty client hints reach the site.
--mask-headless-user-agent rewrites the token under headless mode instead,
but Chromium then empties the high-entropy client hints. --gpu mesa renders
WebGL with Mesa's lavapipe instead of SwiftShader (Linux,
mesa-vulkan-drivers); see the Stealth guide's WebGL renderer section.
For fingerprint-statistics or WebRTC leak-test pages, apply profile locale/timezone/media metadata and block non-proxied WebRTC UDP candidates without viewport, user-agent, or JavaScript patches:
webskrap fetch https://amiunique.org/fr/fingerprint \
--channel chrome \
--virtual-display \
--profile-mode locale \
--reduce-fingerprint-surface \
--webrtc-ip-handling-policy disable_non_proxied_udpPass additional browser flags with repeated --launch-arg=... options. Use the
equals form when the browser flag itself starts with --.
webskrap fetch https://example.com \
--launch-arg=--no-first-run \
--launch-arg=--no-default-browser-checkInteractive browser sessions
webskrap browser drives a persistent browser with commands modeled on the
official Playwright CLI,
implemented natively on WebSkrap's own CDP client. open launches a detached
browser that keeps running between commands; every other command reconnects
to it over CDP, acts on the current page, and exits. It uses installed Chrome,
then Edge, then Chrome for Testing. Installed Chrome and Edge include the
platform's sandbox support, such as the AppArmor profile Ubuntu 23.10+ needs.
The session uses the same stealth path as webskrap fetch: commands never
enable the CDP Runtime domain, so no Runtime.enable leak reaches the page or
its workers, and the browser launches with
--disable-blink-features=AutomationControlled (navigator.webdriver stays
false) plus the 1920x1080 virtual screen used by headless fetches. Headless
sessions still report HeadlessChrome in the user agent; pass --headed when
a site checks it. browser eval runs in the page's own JavaScript world, so
page globals are visible.
webskrap browser open https://example.com
webskrap browser snapshot
webskrap browser click e15
webskrap browser fill "input[name=q]" "chromium"
webskrap browser press Enter
webskrap browser screenshot page.png --full-page
webskrap browser screenshot button.png --target e15 # one element
webskrap browser closesnapshot prints an aria snapshot of the page where each element carries a
ref like [ref=e15]. Interaction commands (click, dblclick, hover,
fill, type, select, check, uncheck) accept either a ref or a CSS
selector. Refs describe the current page, so take a fresh
snapshot after the page changes. Iframe content is listed under its iframe line. Refs in a frame look
like f12345e15. CSS selectors match in the main frame only.
click and dblclick scroll with the mouse wheel, move the cursor along a
curved path and hold the button for a human 60-140 ms; type clicks into the
field and types with human keystroke timing. Both refuse a target covered by
another element. They are slower than mechanical input on purpose (see
Humanizer); fill still sets a value at once.
Navigation uses goto, back, forward, and reload; press sends a key
to the page, and eval runs JavaScript and prints the JSON result. All
commands support --format json for scripting and agents, and fail with a
one-line error instead of a traceback. The exit status tells the kind of
failure; JSON output carries the same name in code:
| Exit | Code |
|---|---|
| 1 | internal |
| 2 | usage |
| 3 | timeout |
| 4 | navigation |
| 5 | browser_launch |
| 6 | sandbox |
| 7 | no_session |
| 8 | session_unreachable |
| 9 | stale_ref |
| 10 | path_rejected |
| 11 | blocked |
open URL and goto take --decline-cookies to click a consent notice's
reject control. They also run the fetch tools' challenge and block check, but
report a match instead of failing: the human output prints a blocked: line,
and JSON output carries it in blocked. The page stays open.
Sessions are named with -s/--session (default default) and listed with
webskrap browser list. Each session stores its browser profile under
~/.webskrap/browser/<name>/ (override the root with WEBSKRAP_BROWSER_DIR),
so cookies and logins persist across open/close. Those directories are
created 0700 on POSIX, because they hold cookies and logged-in state; they
are not encrypted, so use webskrap browser close --delete-data to remove a
session's profile, or close --all to stop every session. A root you set up
yourself via WEBSKRAP_BROWSER_DIR keeps whatever permissions you gave it.
Open and close operations for the same session are serialized across processes. Concurrent commands cannot launch two browsers against one profile or leave a stale state file behind.
Vision and coordinate input
webskrap browser open https://example.com -s visual
webskrap browser view viewport.png -s visual --format json
# Inspect the PNG with your image-viewing tool before choosing coordinates.
webskrap browser mouse click 120 80 -s visual
webskrap browser insert-text "hello 世界" -s visual
webskrap browser press Enter -s visual
webskrap browser mouse scroll 400 300 --delta-y 500 -s visual
webskrap browser view viewport.png -s visual --format json
# Inspect the new view before choosing the drag points.
webskrap browser mouse drag 100 150 --end-x 250 --end-y 180 -s visual
webskrap browser view viewport.png -s visual --format json
webskrap browser close -s visualview writes a viewport PNG and returns path (absolute), url, title,
width, height, scroll_x, scroll_y, mime_type, and coordinate_system
(viewport-css-pixels). Omitting the path generates a unique PNG in the current
directory. JSON contains metadata, not base64 image data: the agent must open
the file with an image-viewing tool.
One image pixel equals one CSS pixel, even on high-DPI pages. (0, 0) is the
viewport's top-left. Map resized previews back to the original dimensions;
do not add document scroll offsets. screenshot --full-page remains available
for export, but its coordinates cannot be used directly with mouse.
mouse accepts click, dblclick, move, scroll, or drag, with required
x y. --button is left (default), middle, or right for click/double-click/
drag. Drag requires both --end-x and --end-y. Scroll uses --delta-x and
--delta-y over the element under x y; positive vertical deltas scroll down,
including in nested scroll areas. Non-finite, out-of-viewport, and unused
arguments fail with usage. Coordinate input uses direct browser events;
selector-based click and type retain their human timing and covered-target
checks.
insert-text inserts literal Unicode into the focused control with an input
event. It does not interpret Enter as a key or emit per-character keydown/
keyup events. Focus with a click or Tab; use press for shortcuts and keys,
or selector-based type for controls that require individual keystrokes.
Take a fresh view after navigation, scrolling, or layout changes. Wheel input may return before scrolling finishes; wait for the expected page condition. Prefer refs/selectors for known controls, and run dependent session actions sequentially. This shows the page viewport, not browser chrome or OS dialogs.
Chromium sandbox
open keeps Chromium's OS sandbox, which is what contains a renderer
compromised by a hostile page. Some environments cannot start it, such as
containers without unprivileged user namespaces or images that disable them.
There the browser exits during startup with a message saying so.
webskrap browser open --no-sandbox # this session only
WEBSKRAP_CHROMIUM_SANDBOX=0 webskrap browser open # every session on this host--no-sandbox wins over the environment variable, and a session started
without the sandbox prints a warning. webskrap browser list shows a
Sandbox column for running sessions. Prefer fixing the host: sandboxing is
the difference between a compromised renderer and a compromised machine.
WebSkrap never drops the sandbox on its own, and never retries a failed launch
without it.
To route a session through a proxy, pass it when the browser starts:
webskrap browser open --proxy socks5://proxy.example:1080
webskrap browser open --proxy http://proxy.example:8080 \
--webrtc-ip-handling-policy default_public_interface_onlyWith --proxy, WebRTC defaults to disable_non_proxied_udp, so pages see
neither the host's LAN address nor its direct public address in ICE candidates;
--webrtc-ip-handling-policy overrides it, with or without a proxy. Both
settings are fixed at launch.
For an http or https proxy that asks for a password, put the credentials in the environment, not in the URL:
export WEBSKRAP_PROXY_USERNAME=user
read -rs WEBSKRAP_PROXY_PASSWORD && export WEBSKRAP_PROXY_PASSWORD
webskrap browser open --proxy http://proxy.example:8080WebSkrap starts a helper process that keeps the credentials in memory. It
answers the proxy for every page, frame and worker, and stops when the browser
closes. The credentials are not written to disk, not put on a command line,
and not passed on to the browser's environment. SOCKS proxies cannot use
credentials, because Chromium does not answer their challenge. If the helper
stops, later commands fail with session_unreachable: close the session and
open it again.
Reopening a running session with a different proxy or policy fails with a
usage error instead of reusing a browser that bypasses the proxy; reopening
it without --proxy reuses it as it is. webskrap browser list shows each
running session's proxy, with (auth) when the helper answers it.
Commands act on the session's current tab. Manage tabs with these commands:
webskrap browser tab-list # * marks the current tab
webskrap browser tab-new https://example.com
webskrap browser tab-select 0 # index from tab-list, in opening order
webskrap browser tab-close # the current tab, or pass an indexA popup or a new tab becomes current. When it closes, the tab selected before it is current again. When the current tab closes, the newest tab left is current.
Deliberate limitations versus the official Playwright CLI: Chromium-based
browsers only, and no commands for network mocking, tracing, or video. back and forward always reload the page,
because the back/forward cache is disabled to keep history navigation
deterministic.
The MCP server exposes the
same sessions to agents as browser_* tools.