Shopify Scraper

Shopify Scraper provide a way to crawl product details of site build on Shopify.

Features

1. Support to fetch all collections of Shopify site;
2. Fetch products of any collection or all the products of the site;
3. Fetch single product by product url;

To check if this site is a Shopify site, open the url `https://<shopname>/admin' and see if this site redirected to the shopify admin ui.

The price values are multiplied by 100 to avoiding floating point calculations.

TODO

1. Support tag filter for collection;

Input Parameters

The input of this scraper shoule be JSON formated. Fields are:

Field	Type	Description
startUrls	Array	Start URLs of Shopify site to start the scraper. Category page, product group page or product page urls are all supported.
proxy	Object	Select proxies to be used by your crawler.

Here is the example:

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops" }
  ],
  "proxy": { "useApifyProxy": true }
}

Input Examples

• fetch collections of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch all products of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch products of one collection

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch product detail

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/products/gymshark-vision-long-sleeve-crop-top-black-aw21" }
  ],
  "proxy": { "useApifyProxy": true }
}

Data storage

Shopify site scraper stores the product data to default data set in JSON format.

{
  "source": {
    "id": "4857453543626",
    "canonicalUrl": "https://www.gymshark.com/products/gymshark-fraction-crop-top-light-green-white-logo",
    "retailer": "Gymshark | Be a visionary.",
    "language": "en",
    "currency": "USD",
    "createdUTC": 1613989955000,
    "updatedUTC": 1640058905000,
    "publishedUTC": 1614279501000
  },
  "title": "Gymshark Fraction Crop Top - Light Green",
  "description": "<meta charset=\"utf-8\"><strong data-mce-fragment=\"1\">IN YOUR LOCKER</strong><span data-mce-fragment=\"1\"></span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">A slice of unique athleisure, the Fraction Crop Top is tasteful, charming and convenient. Available in a range of versatile colours, style any workout or rest day outfit with the finishing touch of this short sleeve crop top.</span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Cropped t-shirt</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Slight dropped shoulder</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Crew neck</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Rolled hem to sleeve</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Large cut off Gymshark logo at hem</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- 95% Cotton, 5% Elastane</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- We've cut down on our use of swing tags, so this product comes without one</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Model is </span><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">5'9\" and wears a size L<br></span>- Video model is 5'8\" and wears size XS<br data-mce-fragment=\"1\"><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">- SKU: GLCT1844-CLM</span><br>",
  "brand": "Gymshark | Be a visionary.",
  "categories": [
    "Womens Crop Top"
  ],
  "tags": [
    "25/02/21",
    "all-products",
    "crop-tops",
    "essentials",
    "filter-colour: Green",
    "filter-size:l",
    "filter-size:m",
    "filter-size:s",
    "filter-size:xl",
    "filter-size:xs",
    "filter-size:xxl",
    "home-workout-clothes",
    "instock:s",
    "instock:xs",
    "outlet",
    "retention-collection",
    "short-sleeve",
    "sizeguide:top",
    "SS21",
    "t-shirts-tops",
    "Womens"
  ],
  "variants": [
    {
      "id": "32686177550538",
      "title": "Extra Small",
      "sku": "GLCT1844-CLM-XS",
      "options": [
        "Extra Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177583306",
      "title": "Small",
      "sku": "GLCT1844-CLM-S",
      "options": [
        "Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177616074",
      "title": "Medium",
      "sku": "GLCT1844-CLM-M",
      "options": [
        "Medium"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177648842",
      "title": "Large",
      "sku": "GLCT1844-CLM-L",
      "options": [
        "Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177681610",
      "title": "Extra Large",
      "sku": "GLCT1844-CLM-XL",
      "options": [
        "Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "39785973285066",
      "title": "Extra Extra Large",
      "sku": "GLCT1844-CLM-XXL",
      "options": [
        "Extra Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    }
  ],
  "medias": [
    {
      "id": "15928233361610",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.A-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233394378",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.B-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233427146",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.C-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233459914",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D1-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233492682",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D2-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233525450",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D3-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    }
  ],
  "options": [
    {
      "type": "Size",
      "values": [
        {
          "id": "Extra Small",
          "name": "Extra Small"
        },
        {
          "id": "Small",
          "name": "Small"
        },
        {
          "id": "Medium",
          "name": "Medium"
        },
        {
          "id": "Large",
          "name": "Large"
        },
        {
          "id": "Extra Large",
          "name": "Extra Large"
        },
        {
          "id": "Extra Extra Large",
          "name": "Extra Extra Large"
        }
      ]
    }
  ]
}