Pricing

$10.00/month + usage

Go to Store

Incremental Web Crawler

Try for free

Developed by

AIRabbit

The Incremental Crawler efficiently fetches URLs of recently added or updated web pages on a target site, optimizing resources by focusing only on new content. Ideal for keeping up with the latest updates, it integrates seamlessly into workflows for content monitoring and analysis.

0.0 (0)

Pricing

$10.00/month + usage

Monthly users

Runs succeeded

>99%

Last modified

4 months ago

Automation

Integrations

URL

urlstringRequired

The URL to check for new pages

Days Ago

daysAgointegerOptional

Number of days to look back for new URLs

Default value of this property is 1

Language

languagestringOptional

Language for search results

Default value of this property is "en"

Country

countrystringOptional

Country for search results

Default value of this property is "us"

Search Domain

searchDomainstringOptional

Domain to use for search

Default value of this property is "google.com"

Next Run ID

nextRunIdstringOptional

ID of the next task to run

Next Run Attribute

nextRunAttributestringOptional

Attribute to replace in the next run (default: startUrls)

Default value of this property is "startUrls"

Max Results

maxResultsintegerOptional

Maximum number of results to fetch

Default value of this property is 10

Pricing

Pricing model

Rental

To use this Actor, you have to pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage.

Free trial

1 day

Price

$10.00

Pro Web Content Crawler (With Images)

assertive_analogy/pro-web-content-crawler

Pro Web Content Crawler is a powerful tool that digs deep into web content and images. It handles complex sites, dynamic pages, and hidden content, making it perfect for extracting both data and images. Customizable and API-ready for your unique data needs.

Gideon Nesh

Web Crawler

rigelbytes/webcrawler

This web crawler is designed to provide users with complete flexibility by allowing them to use their **own proxies**. The scraper collects all pages from the website and returns extracts the **MetaData**, **Title**, and **Content** of the page in MarkDown.

Rigel Bytes

Bandcamp Crawler

service-paradis/bandcamp-crawler

The Bandcamp.com crawler is a web scraping tool that allows you to extract data from the Bandcamp music platform. With this crawler, you can get information about albums, tracks, and much more. The crawler is built on top of Apify SDK, and you can run it both on the Apify platform and locally.

Alexandre Paradis

YouTube Channel Latest Post Date Tracker

newbs/Latest-Post-Time-Youtube

YouTube Channel Latest Post Tracker is a tool for tracking the most recent posts on YouTube channels. It scrapes and provides the latest update dates for selected youtube channels, ideal for monitoring content updates or video uploads on youtube efficiently.

Newbs

Fast Website Content Crawler

6sigmag/fast-website-content-crawler

A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

David Deng

559

Example Website Screenshot Crawler

dz_omar/example-website-screenshot-crawler

Automated website screenshot crawler using Pyppeteer and Apify. This open-source actor captures screenshots from specified URLs, uploads them to the Apify Key-Value Store, and provides easy access to the results, making it ideal for monitoring website changes and archiving web content.

Abdlhakim hefaia

Datasets Compare

petr_cermak/datasets-compare

Act for comparing crawler execution results. By default the final result set will contain only new and updated records.

Petr Cermak

Blog / Dated Content Crawler

diarmuidr/blog-content-crawler

Crawl an entire blog / knowledge base or filter to just the new content. Supporting relevant AI queries by filtering pages by date

Diarmuid

Example Process Crawl Results

apify/example-process-crawl-results

Iterates through all results from a crawler run and count them. Needs to be called from the crawler's finish webhook by adding an URL to finish the webhook of your crawler. Use this actor as a starting point to develop custom post-processing of data from the crawler.

Apify

Deep Website Content Crawler

6sigmag/deep-website-content-crawler

Scrape Failed Killer! A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

David Deng

256