Web Scraper avatar
Web Scraper

Pricing

Pay per usage

Go to Store
Web Scraper

Web Scraper

Developed by

Apify

Apify

Maintained by Apify

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

4.5 (23)

Pricing

Pay per usage

856

Total users

88K

Monthly users

4.5K

Runs succeeded

>99%

Issues response

8.1 days

Last modified

a month ago

CT

Error : net::ERR_TUNNEL_CONNECTION_FAILED

Closed

campsider_tools opened this issue
20 days ago

Hello,

We have recently been observing recurring occurrences of the following error in several of our tasks that rely on the "web-scraper" actor:

net::ERR_TUNNEL_CONNECTION_FAILED

This issue seems to occur intermittently, sometimes multiple times on the same page (each page may be visited up to twice), which in turn leads to incomplete or reduced output in the corresponding runs.

Our task is configurated with :

  • a proxy that changes with each request.
  • a retry policy is set to 1.
  • the maximum concurrency is set to 1.

The problem isn't on our side, as we're running tasks directly from console.apify.com. Tthe targeted website has no updates

We would appreciate any insights into whether this issue is known and currently being addressed on your side, or if there are configuration adjustments we could do to prevent this problem

Thank you in advance for your support.

Best regards,

jindrich.bar avatar

Hello, and thank you for your interest in this Actor!

net::ERR_TUNNEL_CONNECTION_FAILED is an internal Chrome error that gets fired on a proxy error. Proxies (actually, websites and web scrapers in general) can be unreliable, which is why maxRequestRetries is set to 3 by default. By setting maxRequestRetries to 1, you instruct the crawler to fail after one retry, which may not be enough in some cases.

If you prefer complete results over crawling speed, try reverting the maxRequestRetries option to the default value.

I'll close this issue now, but feel free to ask questions, if you have any. Cheers!

CT

campsider_tools

14 days ago

Thank you your answer i'm going to update the maxRequestRetries to 3