# Website Change Detector (`ichigowa/website-change-detector`) Actor Monitor websites for content changes on a schedule. Diffs main content between runs, ignores noise via regex patterns, optionally summarizes changes with an LLM, and alerts via webhook (n8n, Make, Slack). - **URL**: https://apify.com/ichigowa/website-change-detector.md - **Developed by:** [kyle herman](https://apify.com/ichigowa) (community) - **Categories:** Automation, Developer tools - **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Website Change Detector — Website Change Monitoring API Monitor website changes on a schedule and get alerted the moment a page's content actually changes. This Actor is a **website change detection API** you can call from Apify Schedules, n8n, Make, Zapier, or plain cron: it fetches your URLs, extracts the main content, diffs it against the snapshot from the previous run, filters out noise (timestamps, view counters), optionally writes an AI summary of each change, and POSTs changed pages to your webhook. No API keys required for the core functionality — bring your own OpenAI-compatible key only if you want AI summaries. ### How it works 1. **Fetch** — every URL in `startUrls` is downloaded concurrently (browser User-Agent, redirects followed, 30 s timeout, 2 retries). 2. **Extract** — in `text` mode the main readable content is extracted (via [trafilatura](https://trafilatura.readthedocs.io/), with a BeautifulSoup fallback); in `html` mode the raw markup is normalized (scripts, styles, comments, and whitespace stripped). An optional `cssSelector` narrows monitoring to one element (e.g. a price tag). 3. **Clean** — every regex in `ignorePatterns` is removed from **both** the old and new content, so volatile fragments like clocks and counters never trigger false positives. 4. **Diff** — the Actor computes a unified diff and a change percentage (`1 − SequenceMatcher.ratio()`). A page counts as `changed` only when the change percentage is at least `minChangePercent`. 5. **Remember** — the latest snapshot of each URL is stored in a named key-value store (`website-change-detector-state`), so state survives between scheduled runs. 6. **Alert** — changed rows are optionally summarized by an LLM and POSTed as a JSON array to your `webhookUrl`. The **first run** of a URL stores a baseline and is *not* reported as a change (set `notifyOnFirstRun: true` to change that). ### Output schema One dataset row per monitored URL per run — a full audit trail, not just the changes: | Field | Type | Description | |---|---|---| | `url` | string | The monitored URL. | | `status` | string | `baseline` (first snapshot), `changed`, `unchanged`, or `error`. | | `change_percent` | number \| null | Percentage of content that changed (0–100). `null` on fetch errors. | | `diff` | string \| null | Unified diff of the change (truncated to 5 000 characters). Only set for `changed` rows. | | `summary` | string \| null | 2–3 sentence AI summary of the change. Only set when an LLM key is provided. | | `checked_at` | string | ISO 8601 UTC timestamp of the check. | | `error` | string \| null | Error message when `status` is `error` (the run still succeeds). | Example `changed` row: ```json { "url": "https://example.com/pricing", "status": "changed", "change_percent": 4.2, "diff": "--- previous\n+++ current\n@@ -3,1 +3,1 @@\n-Pro plan: $29/mo\n+Pro plan: $39/mo", "summary": "The Pro plan price increased from $29 to $39 per month.", "checked_at": "2026-07-21T09:00:12.345678+00:00", "error": null } ```` ### Scheduling guide — monitor website changes automatically #### Apify Schedules (simplest) 1. Open the Actor → **Schedules** → create a schedule (e.g. `@hourly` or `0 9 * * *` for 9:00 daily). 2. Save your input (URLs, ignore patterns, webhook) with the schedule. 3. Each scheduled run compares against the previous run's snapshots automatically — state lives in the named key-value store `website-change-detector-state`. #### Website change monitor for n8n Two integration options: - **Push (recommended):** set `webhookUrl` to an n8n *Webhook* node URL. Only changed pages are POSTed as a JSON array, so your workflow triggers exactly when something changes. Combine with an Apify Schedule. - **Pull:** use the *Apify* node (or an HTTP Request node against `https://api.apify.com/v2/acts//run-sync-get-dataset-items`) on an n8n Schedule Trigger, then filter rows where `status == "changed"`. #### Make (Integromat) Create a *Custom webhook* module, paste its URL into `webhookUrl`, and add a schedule for the Actor in Apify Console. Each changed page arrives as an item in the webhook payload array. #### Slack / Discord Point `webhookUrl` at a small n8n/Make flow that formats the `summary` + `url` fields into a chat message, or POST directly to a service that accepts arbitrary JSON. ### Ignore-patterns cookbook `ignorePatterns` are Python regexes stripped from both sides before diffing. Common recipes: | Noise | Pattern | |---|---| | Clock times (`14:03:59`) | `\d{2}:\d{2}(:\d{2})?` | | ISO dates (`2026-07-21`) | `\d{4}-\d{2}-\d{2}` | | Human dates (`July 21, 2026`) | `(January|February|March|April|May|June|July|August|September|October|November|December)\s+\d{1,2},?\s+\d{4}` | | View / like counters | `[\d,.]+\s*(views?|likes?|comments?|shares?)` | | "x minutes ago" | `\d+\s*(seconds?|minutes?|hours?|days?)\s+ago` | | Session / cache-buster IDs | `[?&](sid|session|v|t)=[\w-]+` | | Prices you *don't* care about | `\$[\d,.]+` | | Copyright year | `©\s*\d{4}` | Tip: also raise `minChangePercent` (e.g. to `2`) to ignore tiny cosmetic edits site-wide. ### AI change summaries (optional LLM setup) Provide `llmApiKey` and the Actor writes a 2–3 sentence summary of every detected change using an OpenAI-compatible chat-completions API: ```json { "llmApiKey": "sk-...", "llmModel": "gpt-4o-mini", "llmApiBase": "https://api.openai.com/v1" } ``` Works with any OpenAI-compatible provider — OpenAI, Groq, Together, OpenRouter, or a self-hosted server — just change `llmApiBase` and `llmModel`. If the key is absent the Actor silently skips summaries; if an LLM call fails, the change is still reported (with `summary: null`). Your key is only ever sent to the API base you configure and is stored as a secret input field. ### FAQ **Does the default run need any API keys?** No. Change detection, diffing, and webhooks work with zero external keys. Keys are only needed for optional AI summaries. **How is this different from a simple HTTP-hash checker?** Hash checkers fire on every rotating ad, timestamp, or CSRF token. This Actor extracts *main content*, lets you strip volatile fragments with regexes, and applies a minimum-change threshold — so alerts mean something actually changed. **Where is the previous snapshot stored?** In a named key-value store `website-change-detector-state` (key = SHA-256 of the URL). Delete a key (or the store) to reset the baseline for a URL. **Can I monitor JavaScript-rendered pages?** The Actor fetches raw HTML over HTTP. Pages that render all content client-side may need a pre-rendering proxy; most news sites, docs, pricing pages, blogs, and government sites work out of the box. **What happens when a site is down?** The row gets `status: "error"` with the error message, other URLs are unaffected, and the run still finishes successfully — no broken schedules, no false "changed" alerts. **How am I charged?** Pay-per-event: a small fee per URL checked and a fee per change detected. Unreachable URLs are not charged. **Can I track only part of a page?** Yes — set `cssSelector` (e.g. `#pricing-table`, `article.main`, `.stock-status`) and only that element's content is compared. # Actor input Schema ## `startUrls` (type: `array`): List of web pages to monitor for content changes. Each run compares the current content against the snapshot from the previous run. ## `cssSelector` (type: `string`): Only monitor the part of the page matched by this CSS selector (e.g. `#main-content`, `.price`, `article`). Leave empty to monitor the main content of the whole page. ## `checkMode` (type: `string`): `text` extracts readable article/main text (recommended, robust against markup noise). `html` compares normalized raw HTML (scripts, styles, and whitespace stripped) — use it to detect attribute or structure changes. ## `ignorePatterns` (type: `array`): Regular expressions removed from both old and new content before comparing. Use them to ignore volatile fragments like timestamps, view counters, or session IDs. One pattern per line, e.g. `\d{2}:\d{2}:\d{2}` or `\d+ views`. ## `minChangePercent` (type: `string`): Minimum percentage of content that must change (0–100) for a page to be reported as `changed`. Smaller edits are reported as `unchanged`. Default: 0.5. ## `webhookUrl` (type: `string`): If set, a JSON array of all changed rows is POSTed to this URL at the end of the run. Perfect for n8n, Make, Zapier, or Slack incoming webhooks. ## `llmApiKey` (type: `string`): OpenAI-compatible API key. When provided, each detected change gets a 2–3 sentence AI summary. Leave empty to skip summaries — the Actor works fully without any key. ## `llmModel` (type: `string`): Model name used for change summaries (only used when an LLM API key is provided). ## `llmApiBase` (type: `string`): Base URL of the OpenAI-compatible API. Change it to use providers like Groq, Together, OpenRouter, or a local server. ## `notifyOnFirstRun` (type: `boolean`): When enabled, the very first snapshot of a URL is reported as `changed` (and sent to the webhook). When disabled (default), the first run only stores a baseline. ## Actor input object example ```json { "startUrls": [ { "url": "https://example.com" }, { "url": "https://www.wikipedia.org" } ], "checkMode": "text", "ignorePatterns": [], "minChangePercent": "0.5", "llmModel": "gpt-4o-mini", "llmApiBase": "https://api.openai.com/v1", "notifyOnFirstRun": false } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://example.com" }, { "url": "https://www.wikipedia.org" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("ichigowa/website-change-detector").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [ { "url": "https://example.com" }, { "url": "https://www.wikipedia.org" }, ] } # Run the Actor and wait for it to finish run = client.actor("ichigowa/website-change-detector").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://example.com" }, { "url": "https://www.wikipedia.org" } ] }' | apify call ichigowa/website-change-detector --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=ichigowa/website-change-detector", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Website Change Detector", "description": "Monitor websites for content changes on a schedule. Diffs main content between runs, ignores noise via regex patterns, optionally summarizes changes with an LLM, and alerts via webhook (n8n, Make, Slack).", "version": "0.0", "x-build-id": "mxOOeWQ1Y545w5mao" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/ichigowa~website-change-detector/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-ichigowa-website-change-detector", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/ichigowa~website-change-detector/runs": { "post": { "operationId": "runs-sync-ichigowa-website-change-detector", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/ichigowa~website-change-detector/run-sync": { "post": { "operationId": "run-sync-ichigowa-website-change-detector", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrls": { "title": "URLs to monitor", "type": "array", "description": "List of web pages to monitor for content changes. Each run compares the current content against the snapshot from the previous run.", "default": [ { "url": "https://example.com" }, { "url": "https://www.wikipedia.org" } ], "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "cssSelector": { "title": "CSS selector (optional)", "type": "string", "description": "Only monitor the part of the page matched by this CSS selector (e.g. `#main-content`, `.price`, `article`). Leave empty to monitor the main content of the whole page." }, "checkMode": { "title": "Check mode", "enum": [ "text", "html" ], "type": "string", "description": "`text` extracts readable article/main text (recommended, robust against markup noise). `html` compares normalized raw HTML (scripts, styles, and whitespace stripped) — use it to detect attribute or structure changes.", "default": "text" }, "ignorePatterns": { "title": "Ignore patterns (regex)", "type": "array", "description": "Regular expressions removed from both old and new content before comparing. Use them to ignore volatile fragments like timestamps, view counters, or session IDs. One pattern per line, e.g. `\\d{2}:\\d{2}:\\d{2}` or `\\d+ views`.", "default": [], "items": { "type": "string" } }, "minChangePercent": { "title": "Minimum change percent", "type": "string", "description": "Minimum percentage of content that must change (0–100) for a page to be reported as `changed`. Smaller edits are reported as `unchanged`. Default: 0.5.", "default": "0.5" }, "webhookUrl": { "title": "Webhook URL (optional)", "type": "string", "description": "If set, a JSON array of all changed rows is POSTed to this URL at the end of the run. Perfect for n8n, Make, Zapier, or Slack incoming webhooks." }, "llmApiKey": { "title": "LLM API key (optional)", "type": "string", "description": "OpenAI-compatible API key. When provided, each detected change gets a 2–3 sentence AI summary. Leave empty to skip summaries — the Actor works fully without any key." }, "llmModel": { "title": "LLM model", "type": "string", "description": "Model name used for change summaries (only used when an LLM API key is provided).", "default": "gpt-4o-mini" }, "llmApiBase": { "title": "LLM API base URL", "type": "string", "description": "Base URL of the OpenAI-compatible API. Change it to use providers like Groq, Together, OpenRouter, or a local server.", "default": "https://api.openai.com/v1" }, "notifyOnFirstRun": { "title": "Treat first snapshot as a change", "type": "boolean", "description": "When enabled, the very first snapshot of a URL is reported as `changed` (and sent to the webhook). When disabled (default), the first run only stores a baseline.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```