Dun & Bradstreet Scraper avatar

Dun & Bradstreet Scraper

Try for free

3 days trial then $25.00/month - No credit card required now

View all Actors
Dun & Bradstreet Scraper

Dun & Bradstreet Scraper

epctex/dnb-scraper
Try for free

3 days trial then $25.00/month - No credit card required now

Effortlessly extract valuable company information, financial projections, industry insights, and more from the extensive Dun & Bradstreet commercial database. Dive deep into the D&B Data Cloud, Business Directory, articles, companies, and industries with customized search terms.

SC

Actor doesn't return all requested companies and also returns companies which were not requested

Closed

scrape_crawl opened this issue
a year ago

These companies were requested:

https://www.dnb.com/business-directory/company-profiles.mobis_india_limited.5ed64f1f15ee79861b7e876f46215921.html https://www.dnb.com/business-directory/company-profiles.bayerische_motoren_werke_aktiengesellschaft.b2f60216a8986dd0132b03ae1d926227.html https://www.dnb.com/business-directory/company-profiles.abb_india_limited.5111bc20d5f711deed549f6f3055025a.html https://www.dnb.com/business-directory/company-profiles.apple_india_private_limited.4e84ec79d60a2d86b2e8bf8a954d3430.html https://www.dnb.com/business-directory/company-profiles.skoda_auto_volkswagen_india_private_limited.3860c3970e9396c6ee0015e531a1b641.html https://www.dnb.com/business-directory/company-profiles.fossil_india_private_limited.f8b75c7f8c10d0c21d4c2a51f79348a3.html https://www.dnb.com/business-directory/company-profiles.shahi_exports_private_limited.cdff6adae29ea68d4320f2a5775afe4f.html https://www.dnb.com/business-directory/company-profiles.Chiquita_Brands_International_S%C3%A0rl.68c45c0b4a91d33d59f615effa8a46cf.html https://www.dnb.com/business-directory/company-profiles.nokia_solutions_and_networks_india_private_limited.4d2ee87fde4fa249803ab560525bfd09.html https://www.dnb.com/business-directory/company-profiles.dixon_technologies_(india)_limited.0133777077b385c6bcb3d06e416cafc0.html https://www.dnb.com/business-directory/company-profiles.wipro_ge_healthcare_private_limited.194f91f15ce842e1c2555112ceaa4d76.html https://www.dnb.com/business-directory/company-profiles.jaguar_land_rover_limited.522e7f8ea589b984e1ca9ba91153eeda.html https://www.dnb.com/business-directory/company-profiles.starkist_co.2db27f941ff0ec179d903e8b665e2a63.html https://www.dnb.com/business-directory/company-profiles.l_g_electronics_india_private_limited.9e94abb510a7c08e64c2a08f911c08f6.html https://www.dnb.com/business-directory/company-profiles.canfor_pulp_ltd.90c98d7f0047acbaed15c21fa34c1206.html https://www.dnb.com/business-directory/company-profiles.toyota_motor_corporation.d5974b3626590dfccf54fc17efd80a3e.html https://www.dnb.com/business-directory/company-profiles.timken_india_limited.8b67183832ace90905c59410ae7e9e4a.html https://www.dnb.com/business-directory/company-profiles.kennametal_india_limited.ee85ff476180752dac73f3719d2788ee.html https://www.dnb.com/business-directory/company-profiles.gainwell_commosales_private_limited.b07597ec3119a07a13848e09989130b0.html https://www.dnb.com/business-directory/company-profiles.kobelco_construction_equipment_india_private_limited.1017c12f6d1da0d14d2ae2bcfa65fd43.html https://www.dnb.com/business-directory/company-profiles.mrf_limited.6c310bf65807a4553c993f7f20bcb0b3.html https://www.dnb.com/business-directory/company-profiles.hewlett_packard_enterprise_india_private_limited.9f8fac1feafa827068ef6e86b0356fa8.html https://www.dnb.com/business-directory/company-profiles.goodrich_aerospace_services_private_limited.7588bf261b8808b06f57ecc36a16acba.html https://www.dnb.com/business-directory/company-profiles.phoenix_contact_india_private_limited.4eefb74d9bcc8ef59a2ba98c94a1150d.html https://www.dnb.com/business-directory/company-profiles.gm_korea_company.83599a2b7de9292b100428bbb88ffe89.html https://www.dnb.com/business-directory/company-profiles.kyungshin_industrial_motherson_private_limited.92547a182a1cf5c632a64d142d6088e8.html https://www.dnb.com/business-directory/company-profiles.merck_life_science_private_limited.9ca24d42d50bb2c365bb6240def76436.html https://www.dnb.com/business-directory/company-profiles.honda_trading_america_corp.f0e5d772e9aeab5efdff4a1fd10dd06f.html https://www.dnb.com/business-directory/company-profiles.johnson__johnson_limited.0b4833154c8ca28be8dc481d7ff6e6f9.html https://www.dnb.com/business-directory/company-profiles.siemens_healthcare_(pvt)_limited.812daf0ed94e0b01812f24713d440f2b.html https://www.dnb.com/business-directory/company-profiles.lenovo_(india)_private_limited.8145b0587000d56764395c63335ac3ca.html https://www.dnb.com/business-directory/company-profiles.joosung_sea__air_co_ltd.68fcf8f5ae07be83.html https://www.dnb.com/business-directory/company-profiles.welspun_global_brands_limited.f641bd960dc81fab2992913ea68af028.html https://www.dnb.com/business-directory/company-profiles.daimler_india_commercial_vehicles_private_limited.c2af5c4c51e55c51c761266e913aef4f.html https://www.dnb.com/business-directory/company-profiles.atlas_copco_(india)_limited.e0d4cb8761016fb3fd3d0e8ba83ab8ad.html https://www.dnb.com/business-directory/company-information.retail_trade.in.karnataka.bangalore.html?page=6 https://www.dnb.com/business-directory/company-profiles.msc_mediterranean_shipping_company_sa.7056e3fed19fb906c75edb85801b9fc4.html https://www.dnb.com/business-directory/company-profiles.stryker_global_technology_center_private_limited.ef0973e648d9c6d9162c6032416e926a.html https://www.dnb.com/business-directory/company-profiles.thai_samsung_electronics_company_limited.f7349c695b61cd20.html https://www.dnb.com/business-directory/company-profiles.eaton_power_quality_limited.670cbfcf87cf295b80629730379b412b.html https://www.dnb.com/business-directory/company-profiles.danfoss_industries_private_limited.c0745f89be2d76d12368dca66fc05043.html https://www.dnb.com/business-directory/company-profiles.union_de_bananeros_ecuatorianos_sa_ubesa.31282e8750ef9edb09c741b7f6aa48ff.html https://www.dnb.com/business-directory/company-profiles.j_b_and_brothers_private_limited.fd3489e80a184c2fce5b9b66783d24c7.html https://www.dnb.com/business-directory/company-profiles.titan_company_limited.2c23b3d8e3269120f63e585b4c43a25e.html https://www.dnb.com/business-directory/company-profiles.genesis_comtrade_private_limited.1b54f6a6256f45be2bad1b8c91243fe0.html https://www.dnb.com/business-directory/company-profiles.skf_india_limited.31d070e05188d187308b0ccfad4ceb09.html

These were found:

[0] => https://www.dnb.com/business-directory/company-profiles.wipro_ge_healthcare_private_limited.194f91f15ce842e1c2555112ceaa4d76.html [1] => https://www.dnb.com/business-directory/company-profiles.jaguar_land_rover_limited.522e7f8ea589b984e1ca9ba91153eeda.html [2] => https://www.dnb.com/business-directory/company-profiles.l_g_electronics_india_private_limited.9e94abb510a7c08e64c2a08f911c08f6.html [3] => https://www.dnb.com/business-directory/company-profiles.canfor_pulp_ltd.90c98d7f0047acbaed15c21fa34c1206.html [4] => https://www.dnb.com/business-directory/company-profiles.timken_india_limited.8b67183832ace90905c59410ae7e9e4a.html [5] => https://www.dnb.com/business-directory/company-profiles.gainwell_commosales_private_limited.b07597ec3119a07a13848e09989130b0.html [6] => https://www.dnb.com/business-directory/company-profiles.kobelco_construction_equipment_india_private_limited.1017c12f6d1da0d14d2ae2bcfa65fd43.html [7] => https://www.dnb.com/business-directory/company-profiles.kennametal_india_limited.ee85ff476180752dac73f3719d2788ee.html [8] => https://www.dnb.com/business-directory/company-profiles.toyota_motor_corporation.d5974b3626590dfccf54fc17efd80a3e.html [9] => https://www.dnb.com/business-directory/company-profiles.mrf_limited.6c310bf65807a4553c993f7f20bcb0b3.html [10] => https://www.dnb.com/business-directory/company-profiles.hewlett_packard_enterprise_india_private_limited.9f8fac1feafa827068ef6e86b0356fa8.html [11] => https://www.dnb.com/business-directory/company-profiles.goodrich_aerospace_services_private_limited.7588bf261b8808b06f57ecc36a16acba.html [12] => https://www.dnb.com/business-directory/company-profiles.phoenix_contact_india_private_limited.4eefb74d9bcc8ef59a2ba98c94a1150d.html [13] => https://www.dnb.com/business-directory/company-profiles.gm_korea_company.83599a2b7de9292b100428bbb88ffe89.html [14] => https://www.dnb.com/business-directory/company-profiles.kyungshin_industrial_motherson_private_limited.92547a182a1cf5c632a64d142d6088e8.html [15] => https://www.dnb.com/business-directory/company-profiles.merck_life_science_private_limited.9ca24d42d50bb2c365bb6240def76436.html [16] => https://www.dnb.com/business-directory/company-profiles.honda_trading_america_corp.f0e5d772e9aeab5efdff4a1fd10dd06f.html [17] => https://www.dnb.com/business-directory/company-profiles.siemens_healthcare_(pvt)_limited.812daf0ed94e0b01812f24713d440f2b.html [18] => https://www.dnb.com/business-directory/company-profiles.lenovo_(india)_private_limited.8145b0587000d56764395c63335ac3ca.html [19] => https://www.dnb.com/business-directory/company-profiles.welspun_global_brands_limited.f641bd960dc81fab2992913ea68af028.html [20] => https://www.dnb.com/business-directory/company-profiles.joosung_sea__air_co_ltd.68fcf8f5ae07be83.html [21] => https://www.dnb.com/business-directory/company-profiles.daimler_india_commercial_vehicles_private_limited.c2af5c4c51e55c51c761266e913aef4f.html [22] => https://www.dnb.com/business-directory/company-profiles.atlas_copco_(india)_limited.e0d4cb8761016fb3fd3d0e8ba83ab8ad.html

These were returned but not requested:

[0] => https://www.dnb.com/business-directory/company-profiles.kuber_trading_company.b7e0ef06e1cc94562371bfa670da1ccc.html [1] => https://www.dnb.com/business-directory/company-profiles.ohio_homes.df37fedbd274559b2b17ce5c7b10d633.html [2] => https://www.dnb.com/business-directory/company-profiles.bodycraft_salon_skin__cosmetology_private_limited.ca57774ab3d0975c99cbafa274a39954.html [3] => https://www.dnb.com/business-directory/company-profiles.shree_shyam_glass_enterprises_private_limited.4768beecdeda2c61ec07dd9017534b85.html [4] => https://www.dnb.com/business-directory/company-profiles.super_bazar.f3e6b9ba2dc161d2164c285858a46217.html [5] => https://www.dnb.com/business-directory/company-profiles.kirloskar_technologies_private_limited.6e6197c1c36c6b8e7f82391d1147dbd3.html [6] => https://www.dnb.com/business-directory/company-profiles.sainath_commercial_enterprises.e33486f24f0f9123fb28c2077bbf227d.html [7] => https://www.dnb.com/business-directory/company-profiles.arken_solutions_private_limited.27a446b47f84f2f2686af94a0cd64991.html [8] => https://www.dnb.com/business-directory/company-profiles.buildtek_polymers_private_limited.14185736009fa66ff87cdd3ab86ab6c9.html [9] => https://www.dnb.com/business-directory/company-profiles.hmt_(international)_limited.3e3a4de77d06c6cb26c0c249140f796a.html [10] => https://www.dnb.com/business-directory/company-profiles.s_s_lines.2598a94d0e97a681ce470d06cee94867.html [11] => https://www.dnb.com/business-directory/company-profiles.prakash_bhauwala__company.d83a7088921a4170630685ff35e55b83.html [12] => https://www.dnb.com/business-directory/company-profiles.galaxy_paints_and_hadwares.35a500c0df8443f6d894ec2579774351.html [13] => https://www.dnb.com/business-directory/company-profiles.chirag_infrastructure.295320ec25c95b021954e103f9aa52bb.html [14] => https://www.dnb.com/business-directory/company-profiles.srv_home_appliances_private_limited.187503a195658cffd8d8b53a5090319e.html [15] => https://www.dnb.com/business-directory/company-profiles.hindustan_drug_house.9bb1133821f2b4de1665f1a15e7d84bc.html [16] => https://www.dnb.com/business-directory/company-profiles.rr_salons_private_limited.892c785efa37d41d8bc2e8883cf7ec4c.html [17] => https://www.dnb.com/business-directory/company-profiles.varna_coats_private_limited.8db7bc2bb2b3244ebff70bd6507adbd1.html [18] => https://www.dnb.com/business-directory/company-profiles.vishaals_home_stores_retail_private_limited.0c17a090b6738781f465b7da9205dc98.html [19] => https://www.dnb.com/business-directory/company-profiles.asha_kiran_social_business_llp.d6aa347ed3f6df83602a40bb95afcb05.html [20] => https://www.dnb.com/business-directory/company-profiles.mrk_gold_llp.b99a5e1fe3a7adc8832179957fbb1c2f.html [21] => https://www.dnb.com/business-directory/company-profiles.pooja_novelty.3de86c20b5252707e1a84141d527097f.html [22] => https://www.dnb.com/business-directory/company-profiles.madhurya.4e06b235c6dd06c29a204e0f644e090b.html [23] => https://www.dnb.com/business-directory/company-profiles.vinayak_cars_private_limited.6bce61174269750c1c5ca43b4d7416ac.html [24] => https://www.dnb.com/business-directory/company-profiles.steel_udyog.b670850425b9f8416398fde649352040.html [25] => https://www.dnb.com/business-directory/company-profiles.select_trading_company.4d5a85e2fc8fd776110adbd705ac6bb6.html [26] => https://www.dnb.com/business-directory/company-profiles.sreenidhi_agencies.f715d32c14e14b30f5ff8934d5d60b65.html

And these were not found:

[https://www.dnb.com/business-directory/company-profiles.mobis_india_limited.5ed64f1f15ee79861b7e876f46215921.html] => 5626 [https://www.dnb.com/business-directory/company-profiles.bayerische_motoren_werke_aktiengesellschaft.b2f60216a8986dd0132b03ae1d926227.html] => 5645 [https://www.dnb.com/business-directory/company-profiles.abb_india_limited.5111bc20d5f711deed549f6f3055025a.html] => 5663 [https://www.dnb.com/business-directory/company-profiles.apple_india_private_limited.4e84ec79d60a2d86b2e8bf8a954d3430.html] => 5676 [https://www.dnb.com/business-directory/company-profiles.skoda_auto_volkswagen_india_private_limited.3860c3970e9396c6ee0015e531a1b641.html] => 5677 [https://www.dnb.com/business-directory/company-profiles.fossil_india_private_limited.f8b75c7f8c10d0c21d4c2a51f79348a3.html] => 5679 [https://www.dnb.com/business-directory/company-profiles.shahi_exports_private_limited.cdff6adae29ea68d4320f2a5775afe4f.html] => 5681 [https://www.dnb.com/business-directory/company-profiles.Chiquita_Brands_International_S%C3%A0rl.68c45c0b4a91d33d59f615effa8a46cf.html] => 5683 [https://www.dnb.com/business-directory/company-profiles.nokia_solutions_and_networks_india_private_limited.4d2ee87fde4fa249803ab560525bfd09.html] => 5684 [https://www.dnb.com/business-directory/company-profiles.dixon_technologies_(india)_limited.0133777077b385c6bcb3d06e416cafc0.html] => 5685 [https://www.dnb.com/business-directory/company-profiles.starkist_co.2db27f941ff0ec179d903e8b665e2a63.html] => 5688 [https://www.dnb.com/business-directory/company-profiles.johnson__johnson_limited.0b4833154c8ca28be8dc481d7ff6e6f9.html] => 5712 [https://www.dnb.com/business-directory/company-information.retail_trade.in.karnataka.bangalore.html?page=6] => 5721 [https://www.dnb.com/business-directory/company-profiles.msc_mediterranean_shipping_company_sa.7056e3fed19fb906c75edb85801b9fc4.html] => 5722 [https://www.dnb.com/business-directory/company-profiles.stryker_global_technology_center_private_limited.ef0973e648d9c6d9162c6032416e926a.html] => 5723 [https://www.dnb.com/business-directory/company-profiles.thai_samsung_electronics_company_limited.f7349c695b61cd20.html] => 5724 [https://www.dnb.com/business-directory/company-profiles.eaton_power_quality_limited.670cbfcf87cf295b80629730379b412b.html] => 5725 [https://www.dnb.com/business-directory/company-profiles.danfoss_industries_private_limited.c0745f89be2d76d12368dca66fc05043.html] => 5727 [https://www.dnb.com/business-directory/company-profiles.union_de_bananeros_ecuatorianos_sa_ubesa.31282e8750ef9edb09c741b7f6aa48ff.html] => 5728 [https://www.dnb.com/business-directory/company-profiles.j_b_and_brothers_private_limited.fd3489e80a184c2fce5b9b66783d24c7.html] => 5729 [https://www.dnb.com/business-directory/company-profiles.titan_company_limited.2c23b3d8e3269120f63e585b4c43a25e.html] => 5730 [https://www.dnb.com/business-directory/company-profiles.genesis_comtrade_private_limited.1b54f6a6256f45be2bad1b8c91243fe0.html] => 5731 [https://www.dnb.com/business-directory/company-profiles.skf_india_limited.31d070e05188d187308b0ccfad4ceb09.html] => 5732

SC

scrape_crawl

a year ago

How do I tell the scraper to only crawl the submitted initial urls and not follow other urls found in the source?

TU

tugkan

a year ago

Hey Martin,

Thank you very much for reaching out. If you only want to retrieve the first page of the listing URLs, you can add the endPage property to 1. This will apply to all the sub-pages and let you crawl over only the initial URLs. About the problem above, it would be better if we could investigate how the actor behaved while you were trying it out. If possible, can you please share your Run ID? Which can be found on the top right of the screen called "Share" while you are on the Run Detail page.

Best

SC

scrape_crawl

a year ago

Thanks for explaining - the endPage property isn't documented here https://apify.com/epctex/dnb-scraper are there any other undocumented properties? As for your question this is a similar run ID where not all requested pages were in the response but the response also including pages which were not requested: 49fFAwGLzfVCfTHKt

TU

tugkan

a year ago

Hey there,

Thank you very much for sharing the Run ID and the missing property. We will update the documentation immediately. About the Run, we investigated what might go wrong there.

In such cases, the actor enqueues all the URLs in place and starts scraping. Whenever it reaches 50 rows in output, it won't fetch the remaining start URLs. , In your case, the actor filled up this 50 limit with the ones that have been found over the listing URLs. I am not quite sure if that is the intended approach here. What exactly would you like to retrieve from the actor? Maybe we can build up an Input for achieving your goal.

Best

SO

scrape_crawl-owner

a year ago

Hi Tuğkan Cengiz, thanks for responding. In this case we only want to receive the profiles provided in the start URLs. As we were requesting 50 Profile Urls we set the maxItems property to 50 - any lower number would not follow all start urls. The Crawler should (in this case) not follow further links found on the profile pages but only scrape the provided profile.

TE

tugkan_epctex

a year ago

Hey Martin,

Thank you very much for explaining.

As mentioned, the main problem is the links that have been included in the Start URLs. Meaning that, if you provide the company profiles only, then the actor will retrieve that amount of rows in place and not touch any internal links. So in this case, you don't even need to use Max Items.

However, when a start Url includes a link with https://www.dnb.com/business-directory/company-information https://www.dnb.com/business-directory/company-information.household_appliances_and_electrical_and_electronic_goods_merchant_wholesalers.in.tamil_nadu.kanchipuram.html, the actor counts it as a listing page and starts scraping it over. Therefore, if a listing page is included, the actor will proceed forward with it until it hits the Max Items parameter. That is the main logic of the actor and unfortunately, it cannot be changed.

My suggestion for you to achieve your goal; would only provide the links which start with https://www.dnb.com/business-directory/company-profiles https://www.dnb.com/business-directory/company-profiles.mobis_india_limited.5ed64f1f15ee79861b7e876f46215921.html. These links are being counted as DnB Company Profiles as they should be. The couple of links you mentioned in your Input are the ones that contain many companies. That's where your goal is getting distracted.

Please let me know if I can help you with anything else,

Developer
Maintained by Community
Actor metrics
  • 24 monthly users
  • 13 stars
  • 96.9% runs succeeded
  • 15 hours response time
  • Created in Mar 2021
  • Modified about 24 hours ago