Website Screenshot API and Image Scraper in Node.js and Python

Text and metadata tell you what a page says. A screenshot tells you what it looks like, which is the only thing that catches a broken stylesheet, a cookie wall covering the content, a hero image that failed to load, or a competitor's redesign. And an image inventory tells you what the page loads, which matters for performance work, asset audits and checking that nobody is hotlinking your CDN.

Both APIs in this lesson return URLs rather than file contents. That one design decision drives most of the code you write around them.

Capturing a page

Website Screenshot renders a URL in a browser and stores the image. It costs 40 credits, the most expensive call in this course, so it pays to know the parameters before you start looping.

Parameter	Default	Notes
`url`	required	The page to capture
`type`	`png`	Image format; see the note below
`width`	`1920`	Viewport width, 100 to 3840 pixels
`height`	`1080`	Viewport height, 100 to 3840 pixels
`fullpage`	`false`	Premium. Capture the whole scrollable page instead of the viewport

Try it liveWebsite Screenshot40 credits

POSThttps://api.apiverve.com/v1/webscreenshots

curl -X POST "https://api.apiverve.com/v1/webscreenshots" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","width":1280,"height":800,"type":"jpeg"}'

urlwidthheighttype

imageName ends in .jpg. downloadURL is a signed link and expires is in milliseconds.

The response:

{
  "width": 1280,
  "height": 800,
  "scaleFactor": 1,
  "imageName": "1423a6ce-....jpg",
  "expires": 1792867371256,
  "url": "https://example.com",
  "downloadURL": "https://storage.googleapis.com/apiverve/APIData/webscreenshots/1423a6ce-....jpg?GoogleAccessId=...&Expires=1792867371&Signature=..."
}

width, height and scaleFactor (the device pixel ratio used) are premium fields that report what was actually captured. imageName and downloadURL are what you use.

On format: png and jpeg behaved as expected in testing. A request for webp came back with a .png image name, so check the extension of imageName rather than trusting the format you asked for, and convert locally if you need WebP. JPEG is the better default for photographs and busy pages, typically a fraction of the PNG size. PNG is better for text-heavy pages you plan to compare pixel by pixel, because JPEG compression noise shows up as differences that are not there.

The viewport is the layout

width is not just the image size. It is the browser window the page is laid out in, so it decides which responsive breakpoint you capture. A 390-pixel width gets the mobile layout with the hamburger menu; 1280 gets the desktop layout. If you are checking that a page works on phones, capture at a phone width. Do not capture at 1920 and shrink the image afterwards, which gives you a tiny desktop page, not a mobile one.

height is the visible window, and without fullpage the capture stops there. For "what does a visitor see before scrolling", that is exactly what you want.

Check yourself

You need to check that a landing page's mobile layout is not broken. Which capture is right?

The link expires: copy the file

downloadURL is a signed storage URL. The signature is in the query string, and it stops working at the time in expires. Two details matter:

expires is in milliseconds. The value above, 1792867371256, is a Unix timestamp in milliseconds. Compare it with Date.now() in JavaScript or time.time() * 1000 in Python. Treating it as seconds puts the expiry tens of thousands of years in the future, and your "is this still valid" check never fails until the link does.
The window is short. In testing the link was valid for about 48 hours. Read expires from each response rather than hard-coding a duration.

So never store downloadURL in a database as if it were the image. It is a claim ticket. Download the file as soon as the call returns and put it in storage you control.

// Node 18+, ESM
import { writeFile, mkdir } from 'node:fs/promises';
import { extname } from 'node:path';
import { apiverve } from './apiverve.mjs';

export async function capture(url, { width = 1280, height = 800, type = 'jpeg' } = {}) {
  const shot = await apiverve('webscreenshots', { url, width, height, type });
  if (shot.expires <= Date.now()) throw new Error('screenshot link already expired');

  const res = await fetch(shot.downloadURL, { signal: AbortSignal.timeout(30_000) });
  if (!res.ok) throw new Error(`download failed: ${res.status}`);
  const bytes = Buffer.from(await res.arrayBuffer());

  await mkdir('shots', { recursive: true });
  const file = `shots/${new URL(url).hostname}-${Date.now()}${extname(shot.imageName)}`;
  await writeFile(file, bytes);
  return { file, bytes: bytes.length, capturedAt: new Date().toISOString() };
}

console.log(await capture('https://example.com'));

import os
import time
from urllib.parse import urlparse

import requests

from apiverve import apiverve

def capture(url: str, width: int = 1280, height: int = 800, type_: str = "jpeg") -> dict:
    shot = apiverve("webscreenshots", {"url": url, "width": width, "height": height, "type": type_})
    if shot["expires"] <= time.time() * 1000:
        raise RuntimeError("screenshot link already expired")

    res = requests.get(shot["downloadURL"], timeout=30)
    res.raise_for_status()

    os.makedirs("shots", exist_ok=True)
    ext = os.path.splitext(shot["imageName"])[1]
    path = f"shots/{urlparse(url).hostname}-{int(time.time())}{ext}"
    with open(path, "wb") as f:
        f.write(res.content)
    return {"file": path, "bytes": len(res.content)}

if __name__ == "__main__":
    print(capture("https://example.com"))

In production, swap the local folder for object storage and record the capture time beside the file. The download is a plain HTTPS GET with no API key, so it does not need to happen on the same machine that made the API call; a queue worker can do it. It does need to happen well inside the expiry window.

Thumbnails and visual checks

Thumbnails

For a directory of sites, a bookmarking tool or a "recently viewed" list, capture once at a desktop size such as 1280 by 800 and generate smaller versions yourself. Requesting a 320-pixel-wide capture does not give you a thumbnail of the desktop page; it gives you the page laid out for a 320-pixel screen. Resize locally with whatever image library you already use, and you get several sizes from one 40-credit capture.

Refresh on a schedule that matches how often the page changes. A weekly recapture of a few thousand homepages is a sensible budget; recapturing on every view is not.

Visual regression checks

Screenshots are the cheapest way to notice that a page broke in a way your uptime check cannot see: the server returned 200, and the page is a white screen because a script failed. A simple monitor captures the same URL at the same viewport on a schedule and compares each capture with the last good one.

The comparison is where people go wrong. Hashing the file and alerting when the hash changes produces an alert on every run, because a rotating banner, a timestamp, a carousel position or an ad changes a few pixels. Compare at the pixel level and alert on the proportion of the image that changed, with a threshold you tune per page. Better still, mask regions you know are dynamic before comparing. Keep the format and viewport fixed for every capture of a page; changing either makes every comparison meaningless.

Pages that show cookie consent banners or region-specific content will capture whatever a fresh visitor from the capture location sees. That is useful when you want to check the first impression and annoying when you want the content behind the banner. Decide which you are monitoring and accept that the other is out of scope.

Check yourself

A nightly monitor alerts whenever the SHA-256 of a page's screenshot changes. It fires every night although nothing is broken. What is the fix?

Answer the check above to continue.

Every image a page loads

Website Images Scraper lists the images on a page: images[] with src and external, plus imageCount, internalImageCount, externalImageCount, uniqueDomains and maxLinksReached. It costs 25 credits.

Try it liveWebsite Images Scraper25 credits

POSThttps://api.apiverve.com/v1/webimagescraper

curl -X POST "https://api.apiverve.com/v1/webimagescraper" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://www.wikipedia.org"}'

url

external is true for images on a different host. uniqueDomains lists the external hosts.

On wikipedia.org, the logo is served from www.wikipedia.org itself and the donation badges come from upload.wikimedia.org, so the page reports one internal image, the rest external, and uniqueDomains: ["upload.wikimedia.org"]. "External" means a different hostname from the page, not a different company: a site that serves its own images from cdn.example.com will show all of them as external. Group by uniqueDomains and decide for yourself which hosts are yours.

The default limit is 25 images, raising it with maxlinks is premium, and maxLinksReached tells you whether you hit the cap. As with links in lesson two, a capped result cannot prove an image is absent.

Query strings on image URLs

includequery defaults to false, which trims query strings from each src. For deduplication that is helpful. For image CDNs it changes the meaning of the URL: photo.jpg?w=1600&q=80 and photo.jpg?w=200 are different files, and the trimmed photo.jpg may be the full-size original or may not resolve at all. If you are auditing image weight, or you intend to fetch the images, set includequery to true so you get the URLs the browser actually requested.

What an inventory is good for

Asset audits. Find images still served from an old domain after a migration, or third-party hosts you did not know the page depended on.
Broken image checks. Send a HEAD request to each src and report anything that is not 200. Throttle these, since you are now making requests to someone's server.
Choosing a preview image. When a page has no og:image, the first sizeable internal image is often a better fallback than nothing.

image_audit.py

from collections import Counter
from urllib.parse import urlparse

import requests

from apiverve import apiverve

def audit(url: str) -> None:
    data = apiverve("webimagescraper", {"url": url, "includequery": True})
    hosts = Counter(urlparse(img["src"]).hostname for img in data["images"])
    print(f"{data['imageCount']} images, {data['externalImageCount']} external")
    for host, n in hosts.most_common():
        print(f"  {n:3}  {host}")
    if data["maxLinksReached"]:
        print("  (list truncated at the limit)")

    for img in data["images"]:
        try:
            r = requests.head(img["src"], timeout=10, allow_redirects=True)
            if r.status_code != 200:
                print(f"  BROKEN {r.status_code} {img['src']}")
        except requests.RequestException as e:
            print(f"  ERROR {img['src']}: {e.__class__.__name__}")

if __name__ == "__main__":
    audit("https://www.wikipedia.org")

Rights and manners

A list of image URLs is not a licence to use the images. Photos, illustrations and logos belong to whoever made them, and finding them on a public page does not change that. Inventories for audits, monitoring and your own sites are routine. Downloading competitors' product photos to use on your store is not.

Screenshots sit in a similar place. Capturing a page to see whether it is up, to archive your own marketing site, or to show a thumbnail next to a link that points back to the source is normal practice. Publishing screenshots of other people's content as if it were yours is not. If a page shows personal data, so does its screenshot, and it sits in your storage with whatever retention obligations apply to you.

Finally, both calls make a real browser or crawler request to the target site. Spread captures of the same domain out rather than firing a hundred at once.

Next, the polite alternative to scraping altogether: feeds that publishers maintain for exactly this purpose, and sitemaps, in RSS feeds and sitemaps.

Finished the lesson?Mark it complete to track your progress toward the certification.

Next: RSS feeds and sitemaps →

Screenshots and page images

Capturing a page

The viewport is the layout

The link expires: copy the file

Thumbnails and visual checks

Thumbnails

Visual regression checks

Every image a page loads

Query strings on image URLs

What an inventory is good for

Rights and manners

Run every example on your own key. Free plan, no card.

Scaling up?