Dark Web Scraper

Hey Shay Cohen!

Thank you very much for asking the question. Let me answer them one by one;

1. Yes, you need to get the list of the URLs that you'd like to scrape.
2. By default, the scraper encounters retrieves the information of any Phone number, Email, URL, API token, API Secret Key, SSH Key, RSA Key, Crypto wallet address, and many more important data.

Please let me know if you need anything else. Best

Hi Thanks! 1/ from where do I get the lists? do you dont have it Prebuilt? 2/ so in the json it contain all the data that scrap from the url right?

Thanks

Hey Shay Cohen!

1. This would be up to you. If you'd like to come up with a list of URLs, I'd suggest you to research it on the web and come up with a list. Because the URLs can be changed very easily and we don't store anything by default.
2. That's true.

best

1/ Do you have a clue from where can i get those lists? 2/ if i scarp data can we compere from previous scraps to get only new data?

1. I'd say it would be better to check the open web to find out these URLs. Unfortunately, the resources vary a lot. But for Dark Web 101, https://thehiddenwiki.org/ would be a good start.

2. Unfortunately, it doesn't compare the previous runs and get only the new data.

Best

Thank you, Where can i find example of all the json result?

Hey Shay,

The example result can be found on the actor information through this link: https://apify.com/epctex/darkweb-scraper#dark-web-export

Best

Hi, I saw it but you mention that you collect many things like tokens and etc, how can i see a full json result, i want to do some tests and then use the solution, but i need to prepare things from my hand

Now I understood, sorry for the misunderstanding. The following object contains all the possible fields that can come through the actor:

1{
2	"links": "array of strings",
3	"phones": "array of strings",
4	"emails": "array of strings",
5	"cryptoAddresses": {
6		"BTC Wallet": "array of strings"
7		"DASH Wallet": "array of strings"
8		"Monero Wallet": "array of strings"
9		"Ethereum Wallet": "array of strings"
10	},
11	misc: {
12		"Mailgun Private Key": "array of strings",
13		"Heroku API key": "array of strings",
14		"Slack API token": "array of strings",
15		"MailChimp API Key": "array of strings",
16		"Github Personal Access Token": "array of strings",
17		"Github OAuth Access Token": "array of strings",
18		"Github App Token": "array of strings",
19		"Github Refresh Token": "array of strings",
20		"GitHub Access Token": "array of strings",
21		"Stripe Restricted API Token": "array of strings",
22		"Stripe Standard API Token": "array of strings",
23		"PayPal/Braintree Access Token": "array of strings",
24		"Google OAuth Access Key": "array of strings",
25		"Google OAuth ID": "array of strings",
26		"StackHawk API Key": "array of strings",
27		"NuGet API Key": "array of strings",
28		"SendGrid Token": "array of strings",
29		"AWS Access Key": "array of strings",
30		"Google Cloud Platform API Key": "array of strings",
31		"New Relic Admin API Key": "array of strings",
32		"New Relic Insights Key": "array of strings",
33		"New Relic REST API Key": "array of strings",
34		"New Relic Synthetics Location Key": "array of strings",
35		"Google FCM Server Key": "array of strings",
36		"Discord Webhook": "array of strings",
37		"Cloudinary Credentials": "array of strings",
38		"Bitly Secret Key": "array of strings",
39		"Amazon SNS Topic": "array of strings",
40		"PyPI Upload Token": "array of strings",
41		"Shopify Private App Access Token": "array of strings",
42		"Shopify Custom App Access Token": "array of strings",
43		"Shopify Access Token": "array of strings",
44		"Shopify Shared Secret": "array of strings",
45		"Dynatrace Token": "array of strings",
46		"MongoDB Cloud Connection String": "array of strings",
47		"Riot Games Developer API Key": "array of strings",
48		"SerpAPI": "array of strings",
49		"AWS Manager ID": "array of strings",
50		"AWS MWS key": "array of strings",
51		"Facebook Client ID": "array of strings",
52		"AWS S3 Bucket": "array of strings",
53		"Twitter Secret Key": "array of strings",
54		"Twitter Client ID": "array of strings",
55		"LinkedIn Secret Key": "array of strings",
56		"Slack": "array of strings",
57		"EC": "array of strings",
58		"Google API key": "array of strings",
59		"Heroku API key": "array of strings",
60		"MailChimp API key": "array of strings",
61		"Mailgun API key": "array of strings",
62		"PayPal Braintree access token": "array of strings",
63		"Picatic API key": "array of strings",
64		"Stripe API key": "array of strings",
65		"Square access token": "array of strings",
66		"Square OAuth secret": "array of strings",
67		"Twilio API key": "array of strings",
68		"Generic Credential": "array of strings",
69		"Slack Token": "array of strings",
70		"RSA private key": "array of strings",
71		"SSH (DSA) private key": "array of strings",
72		"SSH (EC) private key": "array of strings",
73		"PGP private key block": "array of strings",
74		"Amazon AWS Access Key ID": "array of strings",
75		"Amazon MWS Auth Token": "array of strings",
76		"AWS API Key": "array of strings",
77		"Facebook Access Token": "array of strings",
78		"GitHub": "array of strings",
79		"Generic API Key": "array of strings",
80		"Generic Secret": "array of strings",
81		"Google API Key": "array of strings",
82		"Google Cloud Platform API Key": "array of strings",
83		"Google Cloud Platform OAuth": "array of strings",
84		"Google Drive API Key": "array of strings",
85		"Google Drive OAuth": "array of strings",
86		"Google (GCP) Service-account": "array of strings",
87		"Google Gmail API Key": "array of strings",
88		"Google Gmail OAuth": "array of strings",
89		"Google OAuth Access Token": "array of strings",
90		"Google YouTube API Key": "array of strings",
91		"Google YouTube OAuth": "array of strings",
92		"Heroku API Key": "array of strings",
93		"MailChimp API Key": "array of strings",
94		"Mailgun API Key": "array of strings",
95		"Password in URL": "array of strings",
96		"PayPal Braintree Access Token": "array of strings",
97		"Picatic API Key": "array of strings",
98		"Stripe API Key": "array of strings",
99		"Stripe Restricted API Key": "array of strings",
100		"Square Access Token": "array of strings",
101		"Square OAuth Secret": "array of strings",
102		"Twilio API Key": "array of strings",
103		"Twitter Access Token": "array of strings",
104		"Twitter username": "array of strings",
105		"Instagram username": "array of strings",
106	}
107}

Hi, If i need a bash script that run a curl and get all the result and insert/update the db this is something can you do for me with extra money?