No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.
unable to get data from script even though works on Chome Console
Closed
url: https://singmalls.app/en/malls/triple-one-somerset/merchants/triple-one-somerset-allu
Problem: I looked at your tutorial and used that to create the below code. Unable to get a) website and b) description fields from the script, even though it works when using the Chrome Console. Also unable to get the images from the url, but i am still trying to figure this out.
Code is below: async function pageFunction(context) {
1const $ = context.jQuery; 2const storeName = $('h2').first().text(); 3const store_number = $('p').first().text(); 4const store_location = $("[class^='LocationDetails']").text(); 5const store_contact = $("[class^='tel']").text(); 6const store_openinghrs = $("[class^='MerchantOpeningHours_container']").text(); 7const store_desc = $("[class^='MerchantDetail_descriptionContainer__']").text(); 8const store_website = $("[class^='MerchantDataItem_websiteInset']").text(); 9const googleMapsLink = $("a[href^='https://maps.google.com']"); 10const googleMapsUrl = googleMapsLink.attr('href'); 11 12// Select the slider wrapper 13const slider = $("[class^='MerchantDetail_hero__']"); 14 15// Initialize an array to store the URLs 16const imageUrls = []; 17 18// Find all img tags within the slider and extract their src attributes 19slider.find('img').each(function() { 20 const imageUrl = $(this).attr('src'); 21 imageUrls.push(imageUrl); 22}); 23 24return { 25 url: context.request.url, 26 storeName, 27 store_number, 28 store_location, 29 store_contact, 30 store_openinghrs, 31 store_desc, 32 store_website, 33 googleMapsUrl, 34 imageUrls 35};
}
Hello and thank you for your interest in this Actor!
You almost got it - the only problem is that for the missing items, you're using mobile selectors (perhaps because the open Chrome Console shrank your viewport, forcing the website to render the mobile version). Replace the store_desc and store_website definitions with the following lines:
1const store_desc = $("[class^=MerchantDetailDesktop_description]").text(); 2const store_website = $("[class^=MerchantDataItemDesktop_website] > a").attr('href');
For the image URLs, you can use the following code:
1const image_urls = $(".slider img").map(function() { 2 return $( this ).attr('src'); 3 }).get();
Did this solve your problem? I'll close this issue now, but feel free to ask any additional questions if you have some. Happy scraping!
Apify- 3.7k monthly users
- 98.8% runs succeeded
- 3.6 days response time
- Created in Mar 2019
- Modified about 1 month ago
