Web Crawler

Pricing

$15.00/month + usage

Web Crawler

This web crawler is designed to provide users with complete flexibility by allowing them to use their **own proxies**. The scraper collects all pages from the website and returns extracts the **MetaData**, **Title**, and **Content** of the page in MarkDown.

Pricing

$15.00/month + usage

Rating

0.0

(0)

Developer

Rigel Bytes

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

6 months ago

Last modified

URL

urlstringRequired

URL of the web page to scrape

Proxy configuration

proxyConfigurationobjectOptional

Select proxy groups or use your custom proxies

Default value of this property is {"useApifyProxy":true,"apifyProxyGroups":["RESIDENTIAL"]}

Incremental Web Crawler

flamboyant_leaf/IncrementalCrawler-v2

The Incremental Crawler efficiently fetches URLs of recently added or updated web pages on a target site, optimizing resources by focusing only on new content. Ideal for keeping up with the latest updates, it integrates seamlessly into workflows for content monitoring and analysis.

AIRabbit

Dynamic Web Scraper

josejet/dynamic-web-scraper

Dynamic Web Scraper is an Apify Actor that gathers information online by simulating user browsing behavior on the web. It reduces the time and amount of scraped web pages by using a model (ChatGPT) to make decisions regarding browser navigation and results evaluation.

Pepa J W̚͠h̾̔̎̿͊͛̄͊e̢̦̲̰̦̋̇͗̾̑oi̟͈̯̝̊̉́̇͑̕ğ̆͘͡e͗͛o͊̔̇̄

260

Pro Web Content Crawler (With Images)

assertive_analogy/pro-web-content-crawler

Pro Web Content Crawler is a powerful tool that digs deep into web content and images. It handles complex sites, dynamic pages, and hidden content, making it perfect for extracting both data and images. Customizable and API-ready for your unique data needs.

Gideon Nesh

191

5.0

Enhanced Deep Content Crawler

assertive_analogy/advanced-crawler

A fast, Python-powered web crawler with smart content extraction, JS support, metadata capture, and duplicate detection. Ideal for SEO, content migration, and e-commerce scraping. Reliable, scalable, and easy to customize.

Gideon Nesh

1.0

Web Scraper Experimental Debug

mtrunkat/web-scraper-experimental-dbgr

Experimental version of Apify Web Scraper with Chrome debugger integrated

Marek Trunkát

DataDome Web Scraper

ecomscrape/datadome-web-scraper

DataDome Web Scraper extracts data from DataDome-protected websites. You can customize parameters such as proxies, timeouts, and JavaScript execution, making it ideal for reports, spreadsheets, and applications.

ecomscrape

Web Scraper Task

undrtkr984/web-scraper-task

Matt

128

Instant web data scraper - Scrape any website

curious_coder/instant-web-scraper

Scrape any public and private website data by providing just URL and optionally cookies and proxy information. This scraper is similar to instant data scraper but runs on cloud and can be used as API too!

Curious Coder

1.8K

4.9

Web Search Scraper

akash9078/web-search-scraper

Advanced web search scraper and data extraction API that delivers real-time search results with comprehensive content snippets. Perfect for research, competitive analysis, content discovery, and automated information gathering.

Akash Kumar Naik

Web Scraper and AI processor

scraping_samurai/web-scraper-and-ai-processor

Adaptive AI controller classifies page quality from fast HTTP fetches and selectively triggers headless rendering, then converts raw text into structured JSON from natural-language extraction prompts. Optimizes cost vs. accuracy with AI-guided escalation, retry, and thin/blocked content heuristics.