Similarweb Scraper avatar
Similarweb Scraper
Try for free

7 days trial then $25.00/month - No credit card required now

View all Actors
Similarweb Scraper

Similarweb Scraper

m0uka/similarweb-scraper
Try for free

7 days trial then $25.00/month - No credit card required now

A simple but powerful scraper for similarweb.com. Retrieve website popularity information and get it in a JSON/XML/CSV/Excel/HTML table format. Get data such as total visits, traffic sources, competitors, top countries, company info, etc..

User avatar

Not working anymore

Closed

Nathan Barbeux (productive_scythe) opened this issue
2 years ago

Seems, that the page changed layout, bacause it does not return any result. Tried both https://www.similarweb.com/website/google.com/ as well google.com input values. Is there any chance to update?

User avatar

economical_linnet

2 years ago

Hello, it does not work I got this erorr: 2022-11-21T12:55:47.942Z ERROR CheerioCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {"url":"https://similarweb.com/website/https://www.zeta-shoes.com/en","retryCount":1,"id":"P44n9ZdbHI7VWb6"} 2022-11-21T12:55:47.945Z TypeError: Cannot read properties of null (reading 'substr') 2022-11-21T12:55:47.948Z at parseAppData (/usr/src/app/src/routes.js:13:22) 2022-11-21T12:55:47.949Z at exports.handleStart (/usr/src/app/src/routes.js:26:21) 2022-11-21T12:55:47.949Z at CheerioCrawler.handlePageFunction [as userProvidedHandler] (/usr/src/app/main.js:38:28) 2022-11-21T12:55:47.950Z at /usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:516:62 2022-11-21T12:55:47.951Z at wrap (/usr/src/app/node_modules/@apify/timeout/index.js:73:33) 2022-11-21T12:55:47.952Z at /usr/src/app/node_modules/@apify/timeout/index.js:88:13 2022-11-21T12:55:47.952Z at AsyncLocalStorage.run (node:async_hooks:319:14) 2022-11-21T12:55:47.953Z at /usr/src/app/node_modules/@apify/timeout/index.js:87:25 2022-11-21T12:55:47.954Z at new Promise (

User avatar

cinnabar_bat

2 years ago

I guess we waiting the fix from Lukas, but it's like 21 days he don't show up

User avatar

Hi, sorry for the huge delay, I'll work on a fix now

User avatar

Turns out they implemented aggressive bot detection, so I'll have to rework the entire scraper, so it might take a while.. I will try to figure out a possible solution.

User avatar

rayyala-owner

a year ago

Hi Lukas were you able to fix the issue?

User avatar

julian_risos

a year ago

I still have the same issue as this this thread currently.

2023-01-11T07:22:40.659Z DEBUG CheerioCrawler:SessionPool: Created new Session - session_XO6v8cYNuF 2023-01-11T07:22:41.032Z DEBUG Page opened. {"url":"https://similarweb.com/website/maybank.com"} 2023-01-11T07:22:41.151Z ERROR CheerioCrawler: Request failed and reached maximum retries {"id":"amRDy7Ft4GmPm3D","url":"https://similarweb.com/website/maybank.com","method":"GET","uniqueKey":"https://similarweb.com/website/maybank.com"} 2023-01-11T07:22:41.153Z TypeError: Cannot read properties of null (reading 'substr') 2023-01-11T07:22:41.155Z at parseAppData (/usr/src/app/src/routes.js:13:22) 2023-01-11T07:22:41.156Z at exports.handleStart (/usr/src/app/src/routes.js:26:21) 2023-01-11T07:22:41.158Z at CheerioCrawler.handlePageFunction [as userProvidedHandler] (/usr/src/app/main.js:38:28) 2023-01-11T07:22:41.160Z at /usr/src/app/node_modules/apify/build/crawlers/cheerio_crawler.js:516:62 2023-01-11T07:22:41.161Z at wrap (/usr/src/app/node_modules/@apify/timeout/index.js:73:33) 2023-01-11T07:22:41.163Z at /usr/src/app/node_modules/@apify/timeout/index.js:88:13 2023-01-11T07:22:41.165Z at AsyncLocalStorage.run (node:async_hooks:319:14) 2023-01-11T07:22:41.166Z at /usr/src/app/node_modules/@apify/timeout/index.js:87:25 2023-01-11T07:22:41.168Z at new Promise (

User avatar

travel

a year ago

I just tried, I get same error as above an no results

User avatar

Hi everyone, it's Misa from Apify. We're currently working on an update of this scraper and will inform you once it's done. Sorry for any inconvenience you've experienced with the scraper malfunctioning in the meantime.

User avatar

Hey, it should be fixed.

Developer
Maintained by Apify
Actor metrics
  • 50 monthly users
  • 99.7% runs succeeded
  • Created in May 2022
  • Modified 5 months ago