Browser API Response Shape Drift Monitor | Playwright

Observe the JSON responses a public page actually uses, then compare their sampled structure and primitive types against a persistent baseline. Detect schema drift without exporting payload values, JSON field names, headers, cookies, or request bodies.

Quick start

{
  "urls": ["https://googlechrome.github.io/samples/fetch-api/fetch-json.html"],
  "monitorKey": "public-api-daily",
  "captureMs": 2000,
  "navigationTimeoutSeconds": 25,
  "maxResponses": 20,
  "maxChanges": 50,
  "resetBaseline": false
}

First run: BASELINE_CREATED. Later runs with the same monitorKey and exact page/endpoint URL: UNCHANGED or CHANGED. Values changing while types stay stable do not create drift. Set resetBaseline true only when intentionally accepting a new structure.

Use cases

• Catch JSON type and structural regressions after a frontend/API deployment.
• Monitor anonymous browser data feeds before a scraper integration breaks.
• Add scheduled response-shape checks alongside HAR traffic diagnostics.
• Identify which browser-observed endpoint shape changed without retaining personal data.

What it captures

GET responses with JSON content types from document, XHR, or fetch traffic. Only successful responses up to 256 KiB are accepted. Repeated identical endpoint URLs are grouped and their type observations merged. Credential-bearing and common account/authentication endpoints are skipped. POST/PUT/DELETE and other mutations are blocked, so GraphQL POST traffic is unsupported.

Input and limits

Field	Default	Limit / behavior
urls	Official GoogleChrome fetch sample	1-5 public HTTP(S) page URLs
monitorKey	public-api-daily	3-64 safe letters, digits, hyphens, underscores
captureMs	2000	0-10,000 ms after DOM readiness
navigationTimeoutSeconds	25	5-45 seconds
maxResponses	20	1-40 JSON samples per page
maxChanges	50	1-200 change examples per endpoint
resetBaseline	false	Accept this run as a new baseline

Every page has a 16 MiB network transfer limit, 200 connections, and 10 seconds for body processing. Shapes sample up to 100 array items, 500 distinct paths, 5,000 nodes, and depth 12 per response. Truncated or capped captures are marked. No credentials, cookies, proxy configuration, custom scripts, or headers are accepted.

Output

One dataset row per page. CHECKED contains endpoints with BASELINE_CREATED, UNCHANGED, or CHANGED. NO_JSON_RESPONSES is a free diagnostic result. ERROR indicates capture/storage failure and is free. If all pages fail, the run fails after storing error rows.

Endpoint records contain endpointId, origin, current/previous shape hash, path count, response count, baseline key, truncated flag, change count, and capped added/removed/type-changed paths. Page paths and JSON keys are hashed. Query strings and body values never persist. The OUTPUT record summarizes succeeded, failed, and charged pages.

Baselines are kept in the account's named key-value store browser-api-response-shape-baselines. Preserve it for scheduled checks. Keys combine monitorKey, exact page URL, and endpoint URL identity. Avoid concurrent runs for the same monitorKey; storage follows last-writer semantics.

Privacy-safe paths

A structural path looks like $/k:HASH/[]/k:HASH. Property tokens are the first 16 hexadecimal characters of SHA-256 over the JSON property name; endpointId hashes GET plus the exact URL. Developers who know their API keys can compute the same tokens locally. This deliberate redaction is less readable than raw schema names but protects arbitrary dynamic keys and identifiers.

Pricing proposal

Standard tier: USD 0.015 per api-shape-page-checked event. Charged only when a page yields at least one accepted JSON shape, the dataset report is stored, and every associated baseline is stored. No charge for failed or no-JSON pages. First baseline creation is one useful check. Platform compute/storage costs depend on the user's Apify plan. The cloud listing uses PAY_PER_EVENT pricing at USD 0.015 per successfully stored page check.

Sample inference limitations

This reports observed shape drift, not a full JSON Schema or OpenAPI validity/breaking-change verdict. Empty arrays, nulls, optional fields and changing samples can cause drift. Missing traffic does not prove an endpoint was removed. Fresh browser contexts can miss authenticated, interactive, lazy or service-worker requests. Endpoint queries changing produce distinct identities. Capture does not click, scroll or replay APIs. It does not certify privacy compliance or guarantee scraping reliability.

Suggested tasks

• Daily browser API shape regression: stable key, reset false.
• Release baseline approval: reset true once, then false.
• Focused anonymous integration watch: one page and a short capture window.

Local execution

Install Node 22+ and Chromium. Set APIFY_CHROME_EXECUTABLE_PATH to the browser executable locally, install locked dependencies, then npm test. Store input in storage/key_value_stores/default/INPUT.json and run npm start. Docker pins the Playwright Chrome base image; local smoke uses the installed Chrome.