# Universal Web Snapshot - HTML, Text, Markdown Capture (`fetchcraft/universal-web-snapshot`) Actor
Capture a clean HTML, plain-text, or markdown snapshot of any URL. Built for archival, change detection, and downstream LLM input. Stores rendered title, final URL, status, fetched-at timestamp. $0.003 per snapshot. Free preview run.
- **URL**: https://apify.com/fetchcraft/universal-web-snapshot.md
- **Developed by:** [Emily Ward](https://apify.com/fetchcraft) (community)
- **Categories:** Developer tools, Automation, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet
## Pricing
$50.00 / 1,000 snapshot takens
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Universal Web Snapshot
**Snapshot any URL via a connector chain: static HTML first, Playwright browser if static returns thin content, Wayback Machine if both fail.**
Pure scraper. No API keys. No LLM dependency. Designed to be robust against the three most common scraping failure modes:
| Failure mode | Tier that handles it |
| --- | --- |
| Site returns HTML but it's a JS-only SPA (Notion, Linear, Figma) | Playwright browser |
| Site blocks datacenter IPs | Apify proxy routing |
| Site is down right now but has Wayback snapshots | Wayback Machine |
### What you get back per URL
| Field | Description |
| --- | --- |
| `input_url`, `final_url`, `status` | Identity + HTTP response |
| `tier_used` | "static" / "browser" / "wayback" - tells you which connector won |
| `archived_at` | If Wayback was used, the snapshot timestamp |
| `signals.title`, `og_title`, `meta_description`, `og_description`, `canonical_url` | Standard meta fields |
| `signals.h1[]`, `h2[]` | First N headings (typically your hero copy) |
| `signals.json_ld[]` | Any JSON-LD structured data the page exposes |
| `text_excerpt` (first 2000 chars), `text_full` (up to 20k), `text_length` | Cleaned text content |
| `screenshot_url` | Optional PNG (Playwright tier only) saved to actor KV store |
| `tries[]` | Per-tier result so you can see what was attempted |
| `elapsed_ms` | Total processing time |
### Pricing
**$0.05 per successful snapshot.** Failed URLs (all three tiers exhausted) are not charged.
| Use case | URLs | Cost |
| --- | --- | --- |
| Quick competitor sweep | 20 | $1.00 |
| Daily monitoring (100 URLs/day) | 3,000/mo | $150/mo |
| One-off content audit | 500 | $25.00 |
### How the connector chain works
````
1. Static fetch (cheap, fast) ─┐
├─ HTTP 200 + > 300 chars │ Done. tier\_used="static".
└─ Thin/blocked/missing content ─┤
│
2. Playwright browser (heavier) ─┤
├─ Renders JS, waits for network │ Done. tier\_used="browser".
└─ Browser failure / still thin ─┤
│
3. Wayback Machine (last resort) ─┤
├─ Finds archived snapshot │ Done. tier\_used="wayback".
└─ No archive available ─┘ Error returned.
````
The buyer is only charged for successful snapshots, regardless of how many tiers were tried.
### Use cases
- **Pricing intel**: monitor competitor pricing pages even when they are JS-rendered.
- **Content audits**: snapshot 500 URLs across a domain, get clean text for analysis.
- **AI training data**: prepare structured input for LLM pipelines from any web source.
- **Compliance / legal**: archive a copy of pages with a Wayback timestamp for evidence.
- **SEO research**: extract title + meta + h1/h2 across competitor sites.
- **Investor due diligence**: snapshot a company's site at a point in time.
### Why this is a connector / plugin architecture
The actor's `src/lib/scraping.js` exposes:
- `fetchStatic(url, opts)` - tier 1 implementation
- `fetchBrowser(url, opts)` - tier 2 implementation
- `fetchWayback(url, opts)` - tier 3 implementation
- `smartFetch(url, opts)` - the orchestrator
- `cleanHtml(html)`, `extractSignals(html)`, `normalizeUrl(input)` - shared utilities
Any future scraper actor can import the same lib. This means a Pricing Watcher v2 (or any other actor that needs robust scraping) gets the same multi-tier fetch for free. The pattern lives once.
### What this actor does NOT do
- It does not log into authenticated sites.
- It does not download non-HTML assets (PDFs, videos, etc).
- It does not paginate within a single URL (use a separate crawler for that).
### Tags
`scraping` `web-snapshot` `wayback` `playwright` `connector` `static-fetch` `headless-chrome` `content-extraction` `seo`
---
Made by Emily Ward, Cancel Costs.
### Integrations
This actor works out of the box with every Apify-supported integration:
- **API**: call via Apify API or any official SDK (Python, JavaScript, PHP, .NET). Returns a clean dataset URL.
- **Schedule**: set a daily, weekly, or custom cron cadence in Apify Console. Combine with `notification` for fresh feeds.
- **Webhooks**: wire `ACTOR.RUN.SUCCEEDED` to Slack, Discord, Zapier, Make, n8n, Pipedream, or any HTTPS endpoint.
- **MCP**: this actor is discoverable through Apify's hosted MCP server at `mcp.apify.com` for Claude, Cursor, Cline, Windsurf, and other MCP clients.
- **n8n / Make / Zapier**: native HTTP-Request integration. Trigger the actor on schedule, pipe results to Google Sheets, Airtable, your CRM, or any database.
### Try it free
Every Apify user gets **$5/month in free platform credits** (around 250 events at this actor's per-event price). Run preview mode first to confirm output shape before scaling.
New to Apify? [Sign up here](https://apify.com/sign-up?fpr=fetchcraft) to get free credits on signup.
### What's New
- 2026-06-03: Metadata, categories, and SEO refreshed. Latest version live on Apify Store.
### Last Updated
2026-06-03
# Actor input Schema
## `urls` (type: `array`):
List of URLs to fetch. Each URL is tried via the static-fetch tier first, then the Playwright tier if static returns thin content, then the Wayback Machine if both fail.
## `enable_browser` (type: `boolean`):
If on, fall back to a real Chromium browser when static HTML is too thin (typical for SPAs like Notion, Linear, Figma). Adds ~3-8 seconds per URL to runs that need it.
## `enable_wayback` (type: `boolean`):
If on and both static + browser fail, fetch the latest Wayback Machine snapshot for the URL.
## `want_screenshot` (type: `boolean`):
If on (and the Playwright tier was used), save a PNG to the actor's key-value store and include a URL in the output. Adds ~1 second + extra storage.
## `content_must_match` (type: `string`):
If set, the fetch is considered 'thin' (and the next tier is tried) unless the returned HTML matches this case-insensitive regex. Useful for SPAs where static HTML loads but the actual content is JS-rendered.
## `proxy_groups` (type: `array`):
Apify proxy groups to route the static-fetch tier through. Leave default to use shared datacenter proxies. Empty array = no proxy.
## `max_concurrency` (type: `integer`):
How many URLs to snapshot in parallel. Lower if you have many heavy browser-tier URLs.
## Actor input object example
```json
{
"urls": [
"https://linear.app/pricing",
"https://www.notion.so/pricing"
],
"enable_browser": true,
"enable_wayback": true,
"want_screenshot": false,
"proxy_groups": [
"BUYPROXIES94952"
],
"max_concurrency": 3
}
````
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"urls": [
"https://linear.app/pricing",
"https://www.notion.so/pricing"
]
};
// Run the Actor and wait for it to finish
const run = await client.actor("fetchcraft/universal-web-snapshot").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = { "urls": [
"https://linear.app/pricing",
"https://www.notion.so/pricing",
] }
# Run the Actor and wait for it to finish
run = client.actor("fetchcraft/universal-web-snapshot").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"urls": [
"https://linear.app/pricing",
"https://www.notion.so/pricing"
]
}' |
apify call fetchcraft/universal-web-snapshot --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=fetchcraft/universal-web-snapshot",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Universal Web Snapshot - HTML, Text, Markdown Capture",
"description": "Capture a clean HTML, plain-text, or markdown snapshot of any URL. Built for archival, change detection, and downstream LLM input. Stores rendered title, final URL, status, fetched-at timestamp. $0.003 per snapshot. Free preview run.",
"version": "0.1",
"x-build-id": "WL2FUKwD5oiQc1iTU"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/fetchcraft~universal-web-snapshot/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-fetchcraft-universal-web-snapshot",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/fetchcraft~universal-web-snapshot/runs": {
"post": {
"operationId": "runs-sync-fetchcraft-universal-web-snapshot",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/fetchcraft~universal-web-snapshot/run-sync": {
"post": {
"operationId": "run-sync-fetchcraft-universal-web-snapshot",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"urls"
],
"properties": {
"urls": {
"title": "URLs to snapshot",
"type": "array",
"description": "List of URLs to fetch. Each URL is tried via the static-fetch tier first, then the Playwright tier if static returns thin content, then the Wayback Machine if both fail.",
"items": {
"type": "string"
}
},
"enable_browser": {
"title": "Enable Playwright tier",
"type": "boolean",
"description": "If on, fall back to a real Chromium browser when static HTML is too thin (typical for SPAs like Notion, Linear, Figma). Adds ~3-8 seconds per URL to runs that need it.",
"default": true
},
"enable_wayback": {
"title": "Enable Wayback fallback",
"type": "boolean",
"description": "If on and both static + browser fail, fetch the latest Wayback Machine snapshot for the URL.",
"default": true
},
"want_screenshot": {
"title": "Capture a screenshot",
"type": "boolean",
"description": "If on (and the Playwright tier was used), save a PNG to the actor's key-value store and include a URL in the output. Adds ~1 second + extra storage.",
"default": false
},
"content_must_match": {
"title": "Content gate (regex, optional)",
"type": "string",
"description": "If set, the fetch is considered 'thin' (and the next tier is tried) unless the returned HTML matches this case-insensitive regex. Useful for SPAs where static HTML loads but the actual content is JS-rendered."
},
"proxy_groups": {
"title": "Apify proxy groups",
"type": "array",
"description": "Apify proxy groups to route the static-fetch tier through. Leave default to use shared datacenter proxies. Empty array = no proxy.",
"default": [
"BUYPROXIES94952"
],
"items": {
"type": "string"
}
},
"max_concurrency": {
"title": "Max parallel URLs",
"minimum": 1,
"maximum": 10,
"type": "integer",
"description": "How many URLs to snapshot in parallel. Lower if you have many heavy browser-tier URLs.",
"default": 3
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```