DNB Companies Scraper

DNB Companies Scraper

DNB Companies Scraper is an Apify actor for filtering and extracting data about companies from DNB. It allows you to extract all companies' information. It is build on top of Apify SDK and you can run it both on Apify platform and locally.

• Input
• Output
• Compute units consumption
• Epilogue

Input

Input is a JSON object with the following properties:

1{
2    "searchTerm": SEARCH_TERM,
3    "revenueMin": REVENUE_MIN_FILTER,
4    "numberOfEmployeesMin": NUMBER_OF_EMPLOYEES_MIN_FILTER,
5    "yearStartFrom": YEAR_START_FROM_FILTER,
6    "countryIn": COUNTRY_IN_FILTER,
7    "industryIn": INDUSTRY_IN_FILTER,
8    "maxItems": MAXIMUM_ITEMS,
9    "proxyConfiguration": APIFY_PROXY_CONFIG,
10}

Example:

1{
2  "searchTerm": "Apple",
3  "revenueMin": 1000000,
4  "numberOfEmployeesMin": 100,
5  "yearStartFrom": 1980,
6  "countryIn": "Australia, Germany, Russian, Spain, Japan, United, France",
7  "industryIn": "Computer, Retail",
8  "maxItems": 10,
9  "proxyConfiguration": {
10    "useApifyProxy": true,
11    "apifyProxyCountry": "US"
12  }
13}

Output

Output is stored in a dataset. Each item is an information about a company. Example:

1{
2  "url": "https://www.dnb.com/business-directory/company-profiles.apple_pty_limited.1b1106bd70f6fd219ede98d370ae92ce.html",
3  "name": "APPLE PTY LIMITED",
4  "description": "APPLE PTY LIMITED is located at 367 GEORGE STREET in SYDNEY, NSW and offers Other Electronic Parts and Equipment Merchant Wholesalers products and services.",
5  "website": "www.apple.com.au",
6  "numberOfEmployees": 4137,
7  "telephone": "289878000",
8  "addressLocality": "SYDNEY",
9  "addressCountry": "Australia",
10  "addressRegion": "NSW",
11  "streetAddress": "367 GEORGE STREET",
12  "postalCode": "2000",
13  "type": "Parent",
14  "role": "Proprietorship",
15  "industry": [
16    "Household Appliances and Electrical and Electronic Goods Merchant Wholesalers",
17    "Professional and Commercial Equipment and Supplies Merchant Wholesalers",
18    "Electronics and Appliance Stores",
19    "Other Miscellaneous Store Retailers",
20    "Electronic parts and equipment, nec",
21    "Computers, peripherals, and software",
22    "Computer and software stores",
23    "Miscellaneous retail stores, nec"
24  ],
25  "principals": [
26    {
27      "name": "ANTHONY JAMES KING",
28      "position": "Chief Executive Officer"
29    },
30    {
31      "name": "PETER RONALD DENWOOD",
32      "position": "Director"
33    },
34    {
35      "name": "ANTHONY JAMES KING",
36      "position": "Director"
37    },
38    {
39      "name": "REBECCA-JANE MCCALLUM",
40      "position": "Secretary"
41    },
42    {
43      "name": "PAUL ANTHONY WHITTINGHAM",
44      "position": "Director and Secretary"
45    }
46  ],
47  "breadcrumb": [
48    "HOME",
49    "BUSINESS DIRECTORY",
50    "WHOLESALE TRADE",
51    "MERCHANT WHOLESALERS, DURABLE GOODS",
52    "HOUSEHOLD APPLIANCES AND ELECTRICAL AND ELECTRONIC GOODS MERCHANT WHOLESALERS",
53    "AUSTRALIA",
54    "NEW SOUTH WALES",
55    "SYDNEY"
56  ],
57  "revenue": 8060000000,
58  "fiscalYearEnd": "SEP",
59  "yearStart": 1982,
60  "incorporated": 1982
61}

Compute units consumption

Keep in mind that it is much more efficient to run one longer scrape (at least one minute) than more shorter ones because of the startup time.

The average consumption is 1 Compute unit for 1000 actor pages scraped

Using proxies

DNB now aggressively blocks scrapers. Currently, the only reliable solution to this problem is to use residential proxies or provide your own proxy servers. Datacenter proxies will not work.

Apify residential proxies

The Apify platform provides residential proxies if you have a paid subscription. These proxies are only available to be run within actors on the Apify platform, not externally. If you are interested in using residential proxies for this scraper, contact support@apify.com via email or in-app chat to get the proxies enabled.

Epilogue

Thank you for trying my actor. I will be very glad for a feedback that you can send to my email dtrungtin@gmail.com.