Shopify Scraper avatar
Shopify Scraper
Try for free

No credit card required

View all Actors
Shopify Scraper

Shopify Scraper

pocesar/shopify-scraper
Try for free

No credit card required

Automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc.

What does Shopify Scraper do?

Using this tool, you can automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc.

Need to find product pairs between Shopify and another online shop?

Use the AI Product Matcher. This AI model allows you to compare items from different web stores, identifying exact matches and comparing real-time data obtained via web scraping. With the AI Product Matcher, you can use scraped product data to monitor product matches across the industry, implement dynamic pricing for your website, replace or complement manual mapping, and obtain realistic estimates against your competition for upcoming promo campaigns.

Most importantly, it is relatively easy to get started with (just follow this guide) and it can match thousands of product pairs.

Extend Scraper and Output Function

Extend output function allows to filter the items that are output:

1async ({ item, customData }) => {
2    if (!item.title.includes('cuisine')) {
3        return null; // omit the output
4    }
5
6    delete item.additional; // remove data from output
7
8    item.requestId = customData.requestId; // add data from the outside
9
10    return item;
11}

Extend scraper function allows you to interact with scraper phases:

1async ({ label, url, filter, fns, filteredSitemapUrls, customData }) => {
2    switch (label) {
3        case 'FILTER_SITEMAP_URL': {
4            // product url, like .../products/cooking-for-dummies-2002-289854
5            filter(
6                url.includes('cooking') || url.includes(customData.filter)
7            );
8            break;
9        }
10        case 'SETUP': {
11            // filteredSitemapUrls is a `Set` instance and can be edited in-place
12            filteredSitemapUrls.add('https://example.com/secret-unlisted-sitemap.xml');
13            filteredSitemapUrls.forEach((sitemapURL) => {
14                if (!sitemapURL.includes('en-us')) {
15                    filteredSitemapUrls.delete(sitemapURL);
16                }
17            });
18            break;
19        }
20    }
21}

License

Apache 2.0

Developer
Maintained by Community
Actor metrics
  • 50 monthly users
  • 98.4% runs succeeded
  • 31.7 days response time
  • Created in Jul 2021
  • Modified 11 months ago
Categories