Shopify Scraper

autofacts/shopify

Shopify online store collection and product data extractor. Supports realtime price/stock monitor. Crawl product list or single product in a structured form, including title,description,price,sku, etc.

Free trial for 7 days

Then $5/month

No credit card required now

Autofactor

Modified10 months ago
Users118
Runs1,564

Shopify Scraper

Free trial for 7 days

Then $5/month

Shopify Scraper provide a way to crawl product details of site build on Shopify.

Features

Support to fetch all collections of Shopify site;
Fetch products of any collection or all the products of the site;
Fetch single product by product url;

To check if this site is a Shopify site, open the url `https://<shopname>/admin' and see if this site redirected to the shopify admin ui.

The price values are multiplied by 100 to avoiding floating point calculations.

TODO

Support tag filter for collection;

Input Parameters

The input of this scraper shoule be JSON formated. Fields are:

Field	Type	Description
startUrls	Array	Start URLs of Shopify site to start the scraper. Category page, product group page or product page urls are all supported.
proxy	Object	Select proxies to be used by your crawler.

Here is the example:

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops" }
  ],
  "proxy": { "useApifyProxy": true }
}

Input Examples

fetch collections of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections" }
  ],
  "proxy": { "useApifyProxy": true }
}

fetch all products of shopify site

{
  "startUrls": [
    { "url": "https://uk.gymshark.com" }
  ],
  "proxy": { "useApifyProxy": true }
}

fetch products of one collection

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/collections/crop-tops" }
  ],
  "proxy": { "useApifyProxy": true }
}

fetch product detail

{
  "startUrls": [
    { "url": "https://uk.gymshark.com/products/gymshark-vision-long-sleeve-crop-top-black-aw21" }
  ],
  "proxy": { "useApifyProxy": true }
}

Data storage

Shopify site scraper stores the product data to default data set in JSON format.

{
  "source": {
    "id": "4857453543626",
    "canonicalUrl": "https://www.gymshark.com/products/gymshark-fraction-crop-top-light-green-white-logo",
    "retailer": "Gymshark | Be a visionary.",
    "language": "en",
    "currency": "USD",
    "createdUTC": 1613989955000,
    "updatedUTC": 1640058905000,
    "publishedUTC": 1614279501000
  },
  "title": "Gymshark Fraction Crop Top - Light Green",
  "description": "<meta charset=\"utf-8\"><strong data-mce-fragment=\"1\">IN YOUR LOCKER</strong><span data-mce-fragment=\"1\"></span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">A slice of unique athleisure, the Fraction Crop Top is tasteful, charming and convenient. Available in a range of versatile colours, style any workout or rest day outfit with the finishing touch of this short sleeve crop top.</span><br data-mce-fragment=\"1\"><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Cropped t-shirt</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Slight dropped shoulder</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Crew neck</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Rolled hem to sleeve</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Large cut off Gymshark logo at hem</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- 95% Cotton, 5% Elastane</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- We've cut down on our use of swing tags, so this product comes without one</span><br data-mce-fragment=\"1\"><span data-mce-fragment=\"1\">- Model is </span><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">5'9\" and wears a size L<br></span>- Video model is 5'8\" and wears size XS<br data-mce-fragment=\"1\"><meta charset=\"utf-8\"><span data-mce-fragment=\"1\">- SKU: GLCT1844-CLM</span><br>",
  "brand": "Gymshark | Be a visionary.",
  "categories": [
    "Womens Crop Top"
  ],
  "tags": [
    "25/02/21",
    "all-products",
    "crop-tops",
    "essentials",
    "filter-colour: Green",
    "filter-size:l",
    "filter-size:m",
    "filter-size:s",
    "filter-size:xl",
    "filter-size:xs",
    "filter-size:xxl",
    "home-workout-clothes",
    "instock:s",
    "instock:xs",
    "outlet",
    "retention-collection",
    "short-sleeve",
    "sizeguide:top",
    "SS21",
    "t-shirts-tops",
    "Womens"
  ],
  "variants": [
    {
      "id": "32686177550538",
      "title": "Extra Small",
      "sku": "GLCT1844-CLM-XS",
      "options": [
        "Extra Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177583306",
      "title": "Small",
      "sku": "GLCT1844-CLM-S",
      "options": [
        "Small"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "InStock"
      }
    },
    {
      "id": "32686177616074",
      "title": "Medium",
      "sku": "GLCT1844-CLM-M",
      "options": [
        "Medium"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177648842",
      "title": "Large",
      "sku": "GLCT1844-CLM-L",
      "options": [
        "Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "32686177681610",
      "title": "Extra Large",
      "sku": "GLCT1844-CLM-XL",
      "options": [
        "Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    },
    {
      "id": "39785973285066",
      "title": "Extra Extra Large",
      "sku": "GLCT1844-CLM-XXL",
      "options": [
        "Extra Extra Large"
      ],
      "price": {
        "current": 1500,
        "previous": 2500,
        "stockStatus": "OutOfStock"
      }
    }
  ],
  "medias": [
    {
      "id": "15928233361610",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.A-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233394378",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.B-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233427146",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.C-Edit_AS.jpg?v=1613989955",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233459914",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D1-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233492682",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D2-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    },
    {
      "id": "15928233525450",
      "type": "Image",
      "url": "https://cdn.shopifycdn.net/s/files/1/0156/6146/products/TRAININGCROPTEECOOLMINT.D3-Edit_AS.jpg?v=1613997217",
      "variantIds": [],
      "alt": ""
    }
  ],
  "options": [
    {
      "type": "Size",
      "values": [
        {
          "id": "Extra Small",
          "name": "Extra Small"
        },
        {
          "id": "Small",
          "name": "Small"
        },
        {
          "id": "Medium",
          "name": "Medium"
        },
        {
          "id": "Large",
          "name": "Large"
        },
        {
          "id": "Extra Large",
          "name": "Extra Large"
        },
        {
          "id": "Extra Extra Large",
          "name": "Extra Extra Large"
        }
      ]
    }
  ]
}