Pay $2.00 for 1,000 results

Deep Website Content Crawler

6sigmag/deep-website-content-crawler

Pay $2.00 for 1,000 results

Scrape Failed Killer! A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

Developer

David Deng

Actor Metrics

37 Monthly users
No reviews yet
8 bookmarks
>99% runs succeeded
Created in Oct 2024
Modified 4 months ago

Categories

Lead generation

Back to issues Create new issue

If one of websites hangs, the whole task fails

Open

Nikita Sviridenko (nikita-sviridenko) opened this issue

Could you please add a per-website timeout? And an error column so that we know which of them will fail?

This way other websites will be fetched successfully.

Nikita Sviridenko (nikita-sviridenko)

F.e. from this list 7 websites were imported and then it timed out after 5 min:

1{
2  "startUrls": [
3    "pearl-executivesearch.com",
4    "e-epitech.com",
5    "taylormaderecrutement.com",
6    "carrhure.com",
7    "focus-recrutement.fr",
8    "vauban-recrutement.fr",
9    "beager.com",
10    "job-tourisme.fr",
11    "eurojob-consulting.com",
12    "headhuntingfactory.com",
13    "pentabell.com",
14    "la-releve.com",
15    "haussmann-es.com",
16    "prismemploi.eu",
17    "briks.fr",
18    "88jobs.com",
19    "archibat.com",
20    "solantis.fr",
21    "selescope.com",
22    "ltd-international.com",
23    "sintel-recrutement.fr",
24    "macanders.eu",
25    "jobmania.fr",
26    "quadra-consultants.com",
27    "recruter.tn",
28    "grantalexander.com",
29    "fedlegalimports.com",
30    "asap.work",
31    "synapse-executive.com",
32    "dlsi-emploi.com",
33    "easypartner.fr",
34    "vendome-associes.com",
35    "now-consulting.fr",
36    "recrutop.com",
37    "myrecrutement.eu",
38    "skillwise.fr",
39    "ccld.com",
40    "alerys.fr",
41    "talysio.fr",
42    "gif-emploi.fr",
43    "gif-emploi.fr",
44    "tasterh.fr",
45    "axxis-interimetrecrutement.com",
46    "abg.asso.fr",
47    "lynkus.fr",
48    "morganmallet.agency",
49    "highdev.com",
50    "achil.io",
51    "fedafrica.com",
52    "aperlead.com"
53  ]
54}

Nikita Sviridenko (nikita-sviridenko)

It'll be also helpful to see in the logs which websites are already scraped.

Add comment

📩📍 Google Maps Email Extractor

lukaskrivka/google-maps-with-contact-details

Extract Google Maps contact details. Scrape websites of Google Maps places for contact details and get email addresses, website, location, address, zipcode, phone number, social media links. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Lukáš Křivka

19k

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

78.6k

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

19.1k

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor and lead scraper to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

28.8k

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

32.3k

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including text, hashtags, mentions, comments, images, URLs, likes, locations, and metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

27.3k

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add one or more Instagram usernames and extract number of followers&follows, URLs, bio, posts, likes, counts, related profiles, captions, highlight reels. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

44.9k

Facebook Groups Scraper

apify/facebook-groups-scraper

Extract data from one or multiple public Facebook groups. Get group and post URLs, post text, comments, timestamp, likes and comments count, and basic commentator info. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

8.7k

Facebook Comments Scraper

apify/facebook-comments-scraper

Extract data from hundreds of Facebook comments from one or multiple Facebook posts. Get comment text, timestamp, likes count and basic commenter info. Download the data in JSON, CSV, Excel and use it in apps, spreadsheets, and reports.

Apify

8.7k

Google Maps Scraper

compass/crawler-google-places

Extract data from thousands of Google Maps locations and businesses. Get Google Maps data including reviews, reviewer details, images, contact info, opening hours, location, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

87.5k