Pay $10.00 for 1,000 results

Similarweb Quick Scraper

mscraper/similarweb-quick-scraper

Pay $10.00 for 1,000 results

A quick scraper for Similarweb. Get needed data instantly for domains of your choice. Export accumulated data into formats such as HTML, JSON, or Excel.

Developer

mscraper

Actor Metrics

26 monthly users
No reviews yet
19 bookmarks
>99% runs succeeded
Created in Jun 2023
Modified 7 months ago

Categories

SEO tools

Back to issues Create new issue

Failed urls

Closed

world33 opened this issue

Hello,

Are the failed URLs inevitably occuring in each actor run due to bad proxies and the strong bot mitigation protections similarweb is adopting?
Is there any way to rerun the actor only with the failed URLs once a a run is concluded? Or is there a way to download also the failed urls to easily and quickly identify them and start a new run only with those? It is very time consuming to identify the failed urls otherwise. Maybe when downloading the results it would be possible to add a field called Status where it says Succeded or Failed for each URLs initially added to the input field. At the moment only the succeded URLs are exported if I am not wrong. Thanks

mscraper (mscraper)

Now, they only block IPs. I recommend to try Oxylabs or Brightdata, and pass their proxy string as a custom proxy.
I will save all failed keywords to the key-value store to simplify the process. I can't add it to the dataset because it will cost the same as a successful result.

I will let you know here when I add it.

mscraper (mscraper)

Ok, now an actor saves the failed-websites.json file to the key-value store if any websites fail. I added it as the last step of the actor before the end.

world33

Thank you very much!

Add comment

Similarweb Scraper

tri_angle/similarweb-scraper

A simple but powerful scraper for similarweb.com. Retrieve website popularity information and get it in a JSON/XML/CSV/Excel/HTML table format. Get data such as total visits, traffic sources, competitors, top countries, company info, etc..

Tri⟁angle

2.7k

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

55k

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

4.1k

Google Images Scraper

hooli/google-images-scraper

Scrape image details from images.google.com. Add your query and number of images and extract image details such as image URL, image source, description, image dimensions, thumbnail, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Hooli

1.4k

Fast Google Scraper

hooli/easy-google-scraper

Make collecting data from Google easy with Fast Google Scraper. Extract organic and paid Google search engine results pages, then download your data as HTML table, JSON, CSV, Excel, or XML.

Hooli

YouTube Channel Latest Post Date Tracker

newbs/Latest-Post-Time-Youtube

YouTube Channel Latest Post Tracker is a tool for tracking the most recent posts on YouTube channels. It scrapes and provides the latest update dates for selected youtube channels, ideal for monitoring content updates or video uploads on youtube efficiently.

Newbs

Articles Extractor

web.harvester/articles-extractor

A high-performance, versatile tool for extracting structured data from online articles and web content. Leveraging advanced HTML parsing, this actor efficiently scrapes and processes articles to extract critical information including titles, authors, publication dates, content, metadata, and more.

Web Harvester

407

Bing Search Scraper

tri_angle/bing-search-scraper

Scrape search results from Bing.com. You can get the total number of results, organic results, paid results, people also ask, related queries and more.

Tri⟁angle

Linkedin Employees Scraper

caprolok/linkedin-employees-scraper

Effortlessly gather LinkedIn URLs and names of employees in bulk. Ideal for HR and recruitment, this tool quickly provides essential contact information, simplifying talent search and networking opportunities.

Caprolok

961

Ultimate Screenshot

dz_omar/ultimate-screenshot

Ultimate Screenshot allows you to extract data in formats like JPEG, PNG, PDF, GIF, and MP4. It supports device emulation, including iPhones, Android phones, tablets, and desktops, or uses a default resolution of 1920x1080 for accurate, versatile screenshots and videos.