# Website History Checker (Wayback Machine) (`scrapemint/website-history-checker`) Actor Check any domain's Wayback Machine history in bulk: first and last snapshot, captures per year, gaps and dark periods, plus archive links to every era. Vet expired domains before buying. No login, no API key. Pay per row. - **URL**: https://apify.com/scrapemint/website-history-checker.md - **Developed by:** [Ken M](https://apify.com/scrapemint) (community) - **Categories:** SEO tools, Business, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Website History Checker (Wayback Machine) Check the full archive history of any domain, in bulk. One summary row per site tells you when it first appeared on the web, when it was last captured, how many snapshots exist, which years are missing and the longest dark period. Optional snapshot rows give you a direct archive link to every era of the site. The data source is the Internet Archive's Wayback Machine, the same record SEO tools charge $99 a month to query. No login, no API key, no browser. ### What you get **One summary row per site:** - `archived` (false means the domain has never been captured, itself a signal) - `firstSnapshot`, `lastSnapshot`, `totalSnapshots` - `firstYear`, `lastYear`, `yearsWithCaptures`, `missingYears`, `longestGapYears` - `snapshotsPerYear` (capture count for every year) - `waybackFirstUrl`, `waybackLastUrl`, `calendarUrl` (jump straight to the archive) **Optional snapshot rows** (yearly, monthly, or every capture): - `timestamp`, `year`, `statusCode`, `mimetype`, `lengthBytes` - `originalUrl`, `waybackUrl` (direct link to that capture) ### Input - `websites`: domains (`shopify.com`) or full URLs (a specific page's history works too) - `includeSnapshots`: summary only, or summary plus snapshot rows - `granularity`: `yearly`, `monthly`, or `all` - `fromYear`, `toYear`: bound the snapshot range - `maxSnapshotsPerSite`, `maxRows` ### Example input ```json { "websites": ["shopify.com", "stripe.com"], "granularity": "yearly" } ```` Vetting a list of expired domains before buying: ```json { "websites": ["expired-domain-1.com", "expired-domain-2.net", "expired-domain-3.org"], "includeSnapshots": false, "maxRows": 5000 } ``` ### Uses - Expired and aged domain vetting: a domain with 15 years of continuous captures and no gaps is a different purchase than one that went dark for 6 years or hosted something else entirely. Open the `waybackUrl` rows and see exactly what lived there in each era. - Domain portfolio due diligence in bulk: summaries only, thousands of domains per run. - Research and legal: what a site claimed on a given date, with a citable archive link. - Brand history: when competitors launched, redesigned or went dark. - Pairs with the [Domain WHOIS & Age Checker](https://apify.com/scrapemint/domain-whois-checker) (registration data) and the [Website Tech Stack Detector](https://apify.com/scrapemint/website-tech-stack-detector) (what runs there today): register date, history, and current stack in three runs. ### Pricing Pay per row: summaries $0.005, snapshot rows $0.001. The first 2 rows of every run are free. Vetting 100 domains summary-only costs 50 cents; a full yearly snapshot trail for one domain costs about 3 cents. ### Notes - Data comes from the Internet Archive's public Wayback Machine index, the canonical record of the web since 1996. - A site that was never archived returns a summary row with `archived: false`, so your input list reconciles one to one. - Requests are paced and the Actor stops early with partial results if archive.org throttles, so a blocked run never spins compute. - This Actor reads only public data and never logs in. # Actor input Schema ## `websites` (type: `array`): Domains or URLs to check, e.g. shopify.com or https://example.com/pricing. One entry per site; a bare domain checks the homepage's capture history. ## `includeSnapshots` (type: `boolean`): Besides the one summary row per site, also emit one row per archived snapshot (at the chosen granularity) with status code, mimetype, size and a direct archive link. ## `granularity` (type: `string`): How densely to sample the archive. Yearly returns roughly one snapshot per year (best for vetting), monthly one per month, all returns every capture up to your caps. ## `fromYear` (type: `integer`): Only include snapshots from this year on. Leave 0 for no lower bound. ## `toYear` (type: `integer`): Only include snapshots up to this year. Leave 0 for no upper bound. ## `maxSnapshotsPerSite` (type: `integer`): Cap on snapshot rows per site (the summary row does not count against this). ## `maxRows` (type: `integer`): Maximum rows across all sites, summaries plus snapshots. First 2 rows per run are free. ## `proxyConfiguration` (type: `object`): Optional. archive.org is scraper friendly and requests are paced, so proxy is off by default. ## Actor input object example ```json { "websites": [ "shopify.com" ], "includeSnapshots": true, "granularity": "yearly", "fromYear": 0, "toYear": 0, "maxSnapshotsPerSite": 100, "maxRows": 500 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "websites": [ "shopify.com" ] }; // Run the Actor and wait for it to finish const run = await client.actor("scrapemint/website-history-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "websites": ["shopify.com"] } # Run the Actor and wait for it to finish run = client.actor("scrapemint/website-history-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "websites": [ "shopify.com" ] }' | apify call scrapemint/website-history-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=scrapemint/website-history-checker", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Website History Checker (Wayback Machine)", "description": "Check any domain's Wayback Machine history in bulk: first and last snapshot, captures per year, gaps and dark periods, plus archive links to every era. Vet expired domains before buying. No login, no API key. Pay per row.", "version": "0.1", "x-build-id": "hDRB3WuGfmsu4EiNV" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/scrapemint~website-history-checker/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-scrapemint-website-history-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/scrapemint~website-history-checker/runs": { "post": { "operationId": "runs-sync-scrapemint-website-history-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/scrapemint~website-history-checker/run-sync": { "post": { "operationId": "run-sync-scrapemint-website-history-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "websites" ], "properties": { "websites": { "title": "Websites", "type": "array", "description": "Domains or URLs to check, e.g. shopify.com or https://example.com/pricing. One entry per site; a bare domain checks the homepage's capture history.", "items": { "type": "string" } }, "includeSnapshots": { "title": "Include snapshot rows", "type": "boolean", "description": "Besides the one summary row per site, also emit one row per archived snapshot (at the chosen granularity) with status code, mimetype, size and a direct archive link.", "default": true }, "granularity": { "title": "Snapshot granularity", "enum": [ "yearly", "monthly", "all" ], "type": "string", "description": "How densely to sample the archive. Yearly returns roughly one snapshot per year (best for vetting), monthly one per month, all returns every capture up to your caps.", "default": "yearly" }, "fromYear": { "title": "From year", "type": "integer", "description": "Only include snapshots from this year on. Leave 0 for no lower bound.", "default": 0 }, "toYear": { "title": "To year", "type": "integer", "description": "Only include snapshots up to this year. Leave 0 for no upper bound.", "default": 0 }, "maxSnapshotsPerSite": { "title": "Max snapshot rows per site", "type": "integer", "description": "Cap on snapshot rows per site (the summary row does not count against this).", "default": 100 }, "maxRows": { "title": "Max rows total", "type": "integer", "description": "Maximum rows across all sites, summaries plus snapshots. First 2 rows per run are free.", "default": 500 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Optional. archive.org is scraper friendly and requests are paced, so proxy is off by default." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```