WebSkrap

Scrape JavaScript Pages with Playwright and Python

Many web pages do not expose their main content in the initial HTML. WebSkrap uses Playwright and Chromium so Python scripts can scrape JavaScript-rendered pages after the browser has loaded them.

Browser-backed fetch

import asyncio
 
from webskrap import WebSkrapClient
 
 
async def main() -> None:
    async with WebSkrapClient() as client:
        result = await client.fetch(
            "https://example.com",
            wait_until="domcontentloaded",
            timeout_ms=30_000,
            screenshot="debug.png",
        )
 
    print(result.ok)
    print(result.title)
    print(result.screenshot_path)
 
 
asyncio.run(main())

Faster scraping with resource routing

Images, fonts, media, and trackers often slow scraping down. WebSkrap supports resource policies so you can load only what the page needs.

from webskrap import ResourcePolicy, SessionConfig
 
config = SessionConfig(resource_policy=ResourcePolicy.LITE)

Use DOCUMENTS when you only need top-level document HTML. Use LITE when pages need scripts or styles but not heavy media.

Debug JavaScript pages

For tricky pages, keep a headed browser open:

from pathlib import Path
from webskrap import SessionConfig
 
config = SessionConfig(
    headless=False,
    user_data_dir=Path(".webskrap/debug-profile"),
)

Then inspect what the browser actually sees before changing the scraper.