7 days trial then $25.00/month - No credit card required now

Similarweb Scraper

tri_angle/similarweb-scraper

7 days trial then $25.00/month - No credit card required now

A simple but powerful scraper for similarweb.com. Retrieve website popularity information and get it in a JSON/XML/CSV/Excel/HTML table format. Get data such as total visits, traffic sources, competitors, top countries, company info, etc..

Back to issues Create new issue

Not getting any results

Closed

eventvesta opened this issue

I am sending a list of ~860 websites to scrape and letting the actor run for ~1 hour before aborting and getting no results.

Not sure what is going wrong.

eventvesta

It looks like the crawler is getting stopped by Captcha on every page.

Míša Fialová (misa)

Hey, our apologies for the delay in responding, would you mind resharing the run link as the one you've provided has expired, so we can further look into it? Thank you!

eventvesta

Unfortunately the run no longer exists, as runs are purged after 7 days (see screenshot).

We were attempting to crawl ~860 urls. Here is a sample of 20 of those urls:

1[
2 "https://www.similarweb.com/website/columbustelegram.com/#overview",
3 "https://www.similarweb.com/website/kctoday.6amcity.com/#overview",
4 "https://www.similarweb.com/website/omahamagazine.com/#overview",
5 "https://www.similarweb.com/website/journalstar.com/#overview",
6 "https://www.similarweb.com/website/klkntv.com/#overview",
7 "https://www.similarweb.com/website/journalstar.com/#overview",
8 "https://www.similarweb.com/website/fun4alamokids.com/#overview",
9 "https://www.similarweb.com/website/wmfe.org/#overview",
10 "https://www.similarweb.com/website/kmov.com/#overview",
11 "https://www.similarweb.com/website/heraldtribune.com/#overview",
12 "https://www.similarweb.com/website/rvaonthecheap.com/#overview",
13 "https://www.similarweb.com/website/news-journalonline.com/#overview",
14 "https://www.similarweb.com/website/tallahassee.com/#overview",
15 "https://www.similarweb.com/website/tallahasseemagazine.com/#overview",
16 "https://www.similarweb.com/website/emeraldcoastmagazine.com/#overview",
17 "https://www.similarweb.com/website/850businessmagazine.com/#overview",
18 "https://www.similarweb.com/website/myhorrynews.com/#overview",
19 "https://www.similarweb.com/website/miaminewtimes.com/#overview",
20 "https://www.similarweb.com/website/sun-sentinel.com/#overview",
21 "https://www.similarweb.com/website/spiritofomaha.com/#overview"
22]

Míša Fialová (misa)

Hi, I tried inputting the list of URLs you've provided and managed to scrape data as expected, so please try re-running your input and let us know should you experience any further issues.

Add comment

Developer

Tri⟁angle

Actor metrics

56 monthly users
8 stars
99.9% runs succeeded
Created in May 2022
Modified about 1 month ago

Categories

Business

Marketing

SEO tools

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

69.2k

198

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

16.4k

201

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

43.5k

133

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

9.4k

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

3.3k

Instagram Reel Scraper

apify/instagram-reel-scraper

Scrape data from Instagram reels. Just add one or more Instagram usernames and get your data in seconds including hashtags, mentions, comments, images, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Apify

45.2k

🏯 Tweet Scraper V2 (Pay Per Result) - X / Twitter Scraper

apidojo/tweet-scraper

⚡️ Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.30 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!

API Dojo

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including text, hashtags, mentions, comments, images, URLs, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

15.7k

Facebook Pages Scraper

apify/facebook-pages-scraper

Facebook scraping tool to crawl and extract basic data from one or multiple Facebook Pages. Extract Facebook page name, page URL address, category, likes, check-ins, and other public data. Download data in JSON, CSV, Excel and use it in apps, spreadsheets, and reports.

Apify

17k