Shopify Scraper

Shopify Scraper provide a way to crawl product details of site build on Shopify.

ChangeLogs

02/03/2025

• use new method to escape site gateway forbid
• added support of passthrough currency
• added support of parse InStock/OutOfStock status for single product (most cases are supported)

Features

1. Support to fetch all collections of Shopify site;
2. Fetch products of any collection or all the products of the site;
3. Fetch single product by product url;
4. Support of specify the currency of the product;

To check if this site is a Shopify site, open the url `https://

The price values are multiplied by 100 to avoiding floating point calculations.

TODO

1. Support tag filter for collection;

Input Parameters

The input of this scraper shoule be JSON formated. Fields are:

Here is the example:

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops" }
  ],
  "proxy": { "useApifyProxy": true }
}

Input Examples

• fetch collections of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch all products of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch products of one collection

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops?currency=USD" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch product detail

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/products/gymshark-vision-long-sleeve-crop-top-black-aw21" }
  ],
  "proxy": { "useApifyProxy": true }
}

• fetch product detail with specified currency

{
  "startUrls": [
    { "url": "https://www.fashionnova.com/products/von-dutch-short-sleeve-pj-boxer-pant-set-pink?currency=GBP" }
  ],
  "proxy": { "useApifyProxy": true }
}

Data storage

Shopify site scraper stores the product data to default data set in JSON format.

{
  "source": {
    "id": "4857453543626",
    "canonicalUrl": "https://www.gymshark.com/products/gymshark-fraction-crop-top-light-green-white-logo",
    "retailer": "Gymshark | Be a visionary.",
    "language": "en",
    "currency": "USD",
    "createdUTC": 1613989955000,
    "updatedUTC": 1640058905000,
    "publishedUTC": 1614279501000
  },
  "title": "Gymshark Fraction Crop Top - Light Green",
  "description": "<meta charset=\"utf-8\"><strong data-mce-fragment=\"1\">IN YOUR LOCKER</strong><span data-mce-fragment=\"1\"></span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">A slice of unique athleisure, the Fraction Crop Top is tasteful, charming and convenient. Available in a range of versatile colours, style any workout or rest day outfit with the finishing touch of this short sleeve crop top.</span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Cropped t-shirt</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Slight dropped shoulder</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Crew neck</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Rolled hem to sleeve</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Large cut off Gymshark logo at hem</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- 95% Cotton, 5% Elastane</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- We've cut down on our use of swing tags, so this product comes without one</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Model is </span><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">5'9\" and wears a size L<br></span>- Video model is 5'8\" and wears size XS<br data-mce-fragment=\"1\"><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">- SKU: GLCT1844-CLM</span><br>",
  "brand": "Gymshark | Be a visionary.",
  "categories": [
    "Womens Crop Top"
  ],
  "tags": [
    "25/02/21",
    "all-products",
    "crop-tops",
    "essentials",
    "filter-colour: Green",
    "filter-size:l",
    "filter-size:m",
    "filter-size:s",
    "filter-size:xl",
    "filter-size:xs",
    "filter-size:xxl",
    "home-workout-clothes",
    "instock:s",
    "instock:xs",
    "outlet",
    "retention-collection",
    "short-sleeve",
    "sizeguide:top",
    "SS21",
    "t-shirts-tops",
    "Womens"
  ],
  "variants": [
    {
      "id": "32686177550538",
      "title": "Extra Small",
      "sku": "GLCT1844-CLM-XS",
      "options": [
        "Extra Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177583306",
      "title": "Small",
      "sku": "GLCT1844-CLM-S",
      "options": [
        "Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177616074",
      "title": "Medium",
      "sku": "GLCT1844-CLM-M",
      "options": [
        "Medium"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177648842",
      "title": "Large",
      "sku": "GLCT1844-CLM-L",
      "options": [
        "Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177681610",
      "title": "Extra Large",
      "sku": "GLCT1844-CLM-XL",
      "options": [
        "Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "39785973285066",
      "title": "Extra Extra Large",
      "sku": "GLCT1844-CLM-XXL",
      "options": [
        "Extra Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    }
  ],
  "medias": [
    {
      "id": "15928233361610",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.A-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233394378",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.B-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233427146",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.C-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233459914",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D1-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233492682",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D2-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233525450",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D3-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    }
  ],
  "options": [
    {
      "type": "Size",
      "values": [
        {
          "id": "Extra Small",
          "name": "Extra Small"
        },
        {
          "id": "Small",
          "name": "Small"
        },
        {
          "id": "Medium",
          "name": "Medium"
        },
        {
          "id": "Large",
          "name": "Large"
        },
        {
          "id": "Extra Large",
          "name": "Extra Large"
        },
        {
          "id": "Extra Extra Large",
          "name": "Extra Extra Large"
        }
      ]
    }
  ]
}