No credit card required

Website Content Crawler

apify/website-content-crawler

No credit card required

Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.

115

107

Can we limit the number of pages inside a child?

Opened 3 days ago by sai_sampath, last comment 17 hours ago

Crawling works, but starts at a random location half way in on a a given site

Opened 9 days ago by mascaritas, last comment 9 days ago

Crawling does not work for custom sitemaps

Opened 17 days ago by sai_sampath, last comment 3 days ago

long loading of docker (about 10 sec)

Opened 20 days ago by sash2s, last comment 18 days ago

Text field has HTML content error

Opened 20 days ago by motivated_leaflet, last comment 18 days ago

Require a field to filter the results

Opened 21 days ago by sai_sampath, last comment 16 days ago

Failed Crawling for G2 web pages

Opened 21 days ago by motivated_leaflet, last comment 21 days ago

Same screenshot is being saved to all scrape results

Opened 21 days ago by motivated_leaflet, last comment 17 days ago

Unable to improve crawling speed with memory and concurrency

Opened 21 days ago by motivated_leaflet, last comment 15 days ago

Website Content Crawler missed most of website content

Opened 22 days ago by meditative_windcheater, last comment 18 days ago

A clear error message with the webhook for sites that have no parsed data

Opened 22 days ago by sai_sampath, last comment 18 days ago

How to handle such Errors: cookies[0].sameSite: expected one of (Strict|Lax|None)

Opened a month ago by livingmydesign, last comment 25 days ago

Crawler doesn't seem to ever start the crawling process

Opened a month ago by jamesryancooper, last comment 25 days ago

Timeout setting does not work

Opened a month ago by manuel3, last comment 18 days ago

Cannot retrieve info in lazy load part

Opened a month ago by chutnarin, last comment a month ago

KV Store isn't downloading file attachments correctly

Opened a month ago by embrace_ai, last comment 10 days ago

Failed to process HTML with 'readableText', falling back to 'none'

Opened a month ago by cirez_d, last comment a month ago

Always fails, I have not seen even single run fully crawling the webite

Opened a month ago by cm_naidu, last comment a month ago

Remove cookies not behaving as expected

Opened a month ago by embrace_ai, last comment a month ago

Crawler not respecting maxCrawlDepth

Opened a month ago by alexharper, last comment a month ago

Developer

Apify

Actor metrics

1.9k monthly users
99.7% runs succeeded
2.9 days response time
Created in Mar 2023
Modified 11 days ago

Categories

Developer tools

Business

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

62.2k

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

61.7k

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Apify

43.2k

GPT Scraper

drobnikj/gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

4.2k

AI Product Matcher

equidem/ai-product-matcher

Match products across multiple e-commerce websites. Use this AI product matching Actor whenever you need to find matching pairs of products from different online shops for dynamic pricing, competitor analysis or market research.

Matěj Sochor

296

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

4.1k

Facebook Ads Scraper

apify/facebook-ads-scraper

Extract advertising data from one or multiple Facebook Pages. Get page details, reach estimates, publisher platforms, report count, number of impressions, ad IDs, timestamps, and more. Download Facebook ads data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

3.9k

AI Web Agent

apify/ai-web-agent

Use natural language prompts to browse the web, click on elements, fill and submit forms, extract data, and take screenshots using the OpenAI API.

Apify

385

📩📍 Google Maps Email Extractor

lukaskrivka/google-maps-with-contact-details

Extract Google Maps contact details. Scrape websites of Google Maps places for contact details and get email addresses, website, location, address, zipcode, phone number, social media links. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.