No credit card required

Similarweb Scraper

tri_angle/similarweb-scraper

No credit card required

A simple but powerful scraper for similarweb.com. Retrieve website popularity information and get it in a JSON/XML/CSV/Excel/HTML table format. Get data such as total visits, traffic sources, competitors, top countries, company info, etc..

Do you want to learn more about this Actor?

Get a demo

Back to issues Create new issue

Crawling erros

Closed

gainful_governor opened this issue

Hi there, I've raised this previously and was told that this its not a bug, but I can't see how this can't be deemed a bug when it fails this many times in a row and runs for 15+ minutes (when it usually takes ~ 1 minute

Tri⟁angle (tri_angle)

Hi there, if you check the log of this run, you can see the actor is facing captchas. What you can do is to decrease the number of retries in the input, the default is 10 retries.

gainful_governor

thanks for your reply .... what is the best strategy to get around the captcha? would repeated runs ever solve this? is the captcha based on the fingerprint of the current IP + container?

Tri⟁angle (tri_angle)

You can start another run with residential proxy, this works well (at least now). Here is more info about dealing with anti-scraping protection, if you would consider building your own actors in the future https://docs.apify.com/academy/anti-scraping and our discord community: https://discord.com/invite/jyEM2PRvMU

gainful_governor

thanks for your help :-) will try residential proxy

Add comment

Developer

Tri⟁angle

Actor metrics

122 monthly users
24 stars
99.9% runs succeeded
6 hours response time
Created in May 2022
Modified 2 days ago

Categories

Business

Marketing

SEO tools

Advanced Similarweb Scraper

riccee/advanced-similarweb-scraper

Advanced scraper for Simlarweb. Gets data such as companies' ranks, total visits, competitor analysis, etc. Also offers the option to scrape all top-level employees from all scraped companies.

Joseph Schady

Similarweb Advanced Scraper

curious_coder/similarweb-advanced-scraper

Enrich website details in bulk from Similarweb by scraping all available data at an affordable cost. Get traffic info, Competitors, Audience insights, etc

Curious Coder

288

Similarweb Quick Scraper

mscraper/similarweb-quick-scraper

A quick scraper for Similarweb. Get needed data instantly for domains of your choice. Export accumulated data into formats such as HTML, JSON, or Excel.

mscraper

445

Similarweb Speedy Scraper (with full page Screenshot)

karamelo/similarweb-speedy-scraper-with-full-page-screenshot

Powerful Detailed and Fast Scraper for similarweb.com. Scrape comprehensive domain traffic insights, websites comparing data, sitemaps, and trending domains. for any website with detailed information. Includes full page screenshot.

karamelo

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

17.1k

330

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

14.8k

188

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

58.4k

485

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

24.5k

189

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

78.4k

549

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add one or more Instagram usernames and extract number of followers&follows, URLs, bio, posts, likes, counts, related profiles, captions, highlight reels. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

37.7k

203