No credit card required

Cheerio Scraper

apify/cheerio-scraper

No credit card required

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

All issues Create new issue

Various Bugs

Closed

ollie10 opened this issue

Good Morning I found various bugs in the latest cheerio scraper, I spoke in chat with Tsveta, besides this the fact that you are obliging to use your proxy in a free account is a killer feature that will make user escape but anyway:

If I want to use one of my custom http proxy I get always error even if the proxy is working correctly. I have got error 400 from the proxy, it seems that apify is not making the proper calls.

Besides this the interface is blocking adding the socks5:// protocol even if in your documentation is stated so: https://apify.com/apify/cheerio-scraper#proxy-configuration

Lukáš Křivka (lukaskrivka)

Hello, we are aware of this problem and looking into it now.

As for mandatory proxies, that will stay but we will provide Apify proxies for free users indefinitely soon

ollie10

Another bug I spotted is that globs are not shown once you reload the page after saving it, but they are saved correctly, anyway if you can't see them and you don't know how to use the json editor it's tricky for most users

Lukáš Křivka (lukaskrivka)

Thanks, this is a one-time issue since the Glob input type was just changed. Will be fixed asap

ollie10

You're welcome, regards

Lukáš Křivka (lukaskrivka)

Socks are not supported, we will fix the docs. As I said, very soon free accounts will be granted free proxies forever

ollie10

A little bit of a mess over there, sorry to say: removing features, documentation incorrect... me I'm an hobbist but how companies can build a reliable product with this kind of premises...

Many thanks so I will look forward for the developments

Lukáš Křivka (lukaskrivka)

You are right. Most devs build new actors with Crawlee and Apify SDK directly so the doc issue escaped. As for changes, we are aware of the bad execution of these events, it is not a norm.

Add comment

Developer

Apify

Actor metrics

400 monthly users
99.8% runs succeeded
0.4 days response time
Created in Apr 2019
Modified about 1 month ago

Categories

Developer tools

For creators

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

62.9k

Website Content Crawler

apify/website-content-crawler

Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.

Apify

13k

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

3.2k

Merge, Dedup & Transform Datasets

lukaskrivka/dedup-datasets

The ultimate dataset processor. Extremely fast merging, deduplications & transformations all in a single run.

Lukáš Křivka

1.7k

Actor fail manager

lukaskrivka/actor-fail-manager

Automatically triggered on a failed run to analyze if the run should be resurrected and to create an error report for the author.

Lukáš Křivka

2.6k

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

603

Website Screenshot Generator

apify/screenshot-url

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Apify

2.4k

Anti Captcha Recaptcha

petr_cermak/anti-captcha-recaptcha

🧰 Actor for solving Google reCAPTCHA using the anti-captcha.com service. You need to have an anti-captcha subscription.

Petr Cermak

1.3k

Page Scraping Analyzer

apify/page-analyzer

Performs analysis of a webpage to figure out the best way how to scrape its data. Provide a URL and data points to find and get back a detailed dashboard showing how the data can be scraped. Works with initial and rendered HTML, JavaScript variables and dynamically loaded data.

Apify

1.1k