No credit card required

Crawl4ai

janbuchar/crawl4ai

No credit card required

EXPERIMENTAL - Wraps the https://crawl4ai.com/ library for retrieving text content from websites

Crawl4AI Actor

This Actor wraps the powerful Crawl4AI library, giving you a feature-packed web crawler and scraper with extra functionality like link-following and automatic retries for failed requests.

What this Actor does

Crawl and scrape websites with precision, using CSS, XPath, or even LLM-based extraction.
Generate clean Markdown output, perfect for RAG pipelines or direct ingestion into large language models.
Follow links automatically to dig deeper into websites without manual intervention.
Retry failed requests to ensure maximum data collection with minimal fuss.

Key features

Structured extraction: Parse repeated patterns from any website using CSS, XPath, or advanced LLM-based tools.
High performance: Crawl in parallel with ease, based on the available resources.
Open source and free: No API keys, no paywalls—just full access to your data.
Automatic retries and link following: Go beyond the base Crawl4AI functionality with smarter crawling features.

How to use

Input your target URLs: Provide the list of websites you want to crawl.
Set your extraction method (optional): Choose CSS, XPath, or LLM-based extraction depending on your needs.
Configure advanced options (optional): Use proxies, enable stealth mode, or tweak session settings.
Run the Actor: Let it handle crawling, link-following, and retrying any failed requests automatically.
Get your data: Export clean, structured Markdown ready for use in your projects.

Pricing

This Actor is completely free to use.

Developer

Jan Buchar

Actor Metrics

11 monthly users
1 bookmark
97% runs succeeded
Created in Oct 2024
Modified 18 days ago

Categories

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

6.4k

121

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

76.2k

460

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

5.3k

Tiktok Shop Scraper

excavator/tiktok-shop-scraper

This is the Actor for crawling data from the TikTok shop product URLs. For example: https://shop.tiktok.com/view/product/XXXXXXXXXX These URLs are only available for TikTok Shop US. You can test it here: https://apify.com/excavator/tiktok-shop-product

Excavator

Playwright Scraper

apify/playwright-scraper

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

Email Scraper

ib4ngz/email-scraper

This actor scrapes email addresses from a list of provided URLs. It recursively crawls pages, extracts unique emails, and stores them in a dataset. The actor supports DNS validation to ensure domain authenticity and allows filtering based on custom crawling depth.

Iqbal R

Mass Shopee Product Detail Pages Crawler

lazpee.com/mass-shopee-product-crawler

You just need to enter Shopee Detail Page Links. If getting blocked, you should change proxy to the right country of Shopee Domain. Please note Shopee Detail Pages not Category pages. Designed for mass crawling with smallest resource and proxy.

Lazpee.com

250

📧 Email & Contact Extractor From Any Web

easyapi/email-contact-extractor-from-any-web

A powerful web scraper that automatically extracts emails, phone numbers, and social media profiles from any website. Features deep crawling, smart retry mechanism, and advanced pattern recognition. Perfect for lead generation and contact discovery.

EasyApi

Tiktok Shop Product

excavator/tiktok-shop-product

This is the Actor for crawling data from the TikTok shop product URLs. For example: https://shop.tiktok.com/view/product/XXXXXXXXXX OR https://www.tiktok.com/t/XXXXXXXXX/ These URLs are only available for TikTok Shop US

Excavator

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

778