Website Changes Detector

Pricing

Pay per usage

Try for free

Go to Apify Store

Website Changes Detector

Try for free

Developed by

Tri⟁angle

Maintained by Apify

Automatically track website changes (new, updated, or removed pages) using periodic re-crawling. Supports flexible crawling settings, a list of URLs or sections of websites, antiblocking, filtering by keywords, and more.

5.0 (1)

Pricing

Pay per usage

Last modified

3 months ago

Developer tools

Automation

Website Content Crawler Input

wccInputobjectOptional

All input properties for the Website Content Crawler should be here. See docs here: https://apify.com/apify/website-content-crawler#input

Memory

wccMemoryEnumOptional

Amount of memory (RAM) allocated for the Website Content Crawler run in megabytes.

Value options:

"32768": string"16384": string"8192": string"4096": string"2048": string"1024": string"512": string

Default value of this property is "4096"

Timeout

wccTimeoutintegerOptional

Timeout for the Website Content Crawler Actor run in seconds. Zero value means there is no timeout, and the Actor runs until completion, or maybe infinitely. Default is 360,000 seconds (100 hours).

Default value of this property is 360000

WCC Dataset Prefix

websiteContentDatasetNamePrefixstringRequired

Prefix for naming the Website Content Crawler datasets (e.g., 'projectX-prod'). If empty, default prefix ('wcc-changes-detector') will be used.

Include URLs From Previous Website Content Crawler Run

includePreviousRunUrlsbooleanOptional

If checked, all URLs successfully crawled in the most recent (previous) dataset (with the same prefix) will be added to the start URLs for the new WCC crawl, alongside any URLs provided in the 'Website Content Crawler Input'. This effectively re-crawls pages found last time.

Default value of this property is false

Filter by Keywords

filterKeywordsarrayOptional

Only include pages whose text content contains at least one of these keywords. Leave empty to disable keyword filtering.

Default value of this property is []

Skip Crawling

skipCrawlbooleanOptional

If checked, do not run a new WCC crawl. Compare the two most recent existing datasets with the specified prefix.

Default value of this property is false

Max WCC Datasets to Keep

websiteContentDatasetMaxCountintegerRequired

Maximum number of historical WCC datasets with this prefix to retain. Older ones will be deleted. Minimum is 2. Maximum is 1000.

Default value of this property is 5

Change Item Types to Return (NEW, UPDATED, REMOVED, SAME)

returnChangeTypesarrayOptional

Select the types of changes to return. If none are selected, 'NEW' and 'UPDATED' will be returned.

Default value of this property is ["NEW","UPDATED"]

Sitemap Change Orchestrator

tri_angle/sitemap-change-orchestrator

Monitor website sitemaps for new, updated, or removed URLs. Integration with the Website Content Crawler (WCC) allows feeding only relevant URLs. This ensures your web crawls are efficient, targeted, and resource-optimized, keeping your datasets fresh for any application.

Tri⟁angle

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

11K

4.9

Website Change Detector

eloquent_mountain/website-change-detector

Monitors websites for changes. Detects modifications to HTML structure and visual differences via screenshots. Provides detailed change reports including HTML diff. Track multiple URLs. Use tasks for recurring runs. Integrate as API

Paco

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

94K

4.0

Content Checker

jakubbalada/content-checker

Monitor a website or web page for content changes. Automatically saves before and after screenshots and sends an email notification when content changes are detected.

Jakub Balada

2.1K

4.9

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

9.9K

5.0

RAG Web Browser

apify/rag-web-browser

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs.

Apify

7.2K

4.9

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

84K

4.6

HTML Scraper pro

scrapingxpert/html-scraper-pro

The HTML Scraper Pro is a powerful tool designed to extract the HTML source code and metadata from websites. It uses advanced web scraping techniques to retrieve the full HTML content of web pages,page title and HTTP status code.This tool is ideal for data extraction, website analysis, and archiving