Shopify Products Scraper By Collection & Keyword

Shopify Products Scraper — Extract Products and Collections as JSON

Shopify Products Scraper By Collection & Keyword pulls product and collection data straight from any public Shopify storefront's own catalogue endpoints — scoped to the whole store, to named collections, to the store's own in-store product search, or to a hand-picked list of product URLs. Every product row can carry the collections it belongs to, and every collection can be emitted as its own row alongside the products. Output is typed, normalized JSON — no HTML, no CSS selectors, no parsing step before the data reaches your pipeline or an LLM. Point it at a storefront and get a category-mapped product catalogue back.

What is Shopify Products Scraper By Collection & Keyword?

It's a targeted Shopify catalogue scraper: instead of dumping every product on a store, it lets you choose the scope — whole store, specific collections, an in-store keyword search, or specific product URLs — and joins collection membership onto each product row so you know which categories a product sits in. No Shopify account or API key is required; it reads the storefront's own public JSON endpoints (products.json, collections.json, search/suggest.json), the same ones a shopper's browser loads.

• Scrape a store's whole published product catalogue
• Scrape products scoped to one or more named collections
• Scrape whatever the store's own product search returns for your keywords
• Scrape a specific list of product URLs
• Join collection membership (which collections, how many, the primary one) onto every product row
• Emit one row per collection, with reported vs. observed product counts
• Export as JSON or CSV (or XML/Excel) straight from the Apify dataset

What data does Shopify Products Scraper By Collection & Keyword collect?

Each run can return two distinct row shapes — product rows and collection rows — plus an optional collection-membership join carried on every product row.

Data Type	Key Fields	JSON Field Names
Products	title, price, vendor, tags, product type	title, price, compare_at_price, vendor, product_type, tags, handle, product_url
Collection membership (joined onto product rows)	which collections, how many, the primary one	collections, collectionTitles, collectionCount, primaryCollection, membershipComplete
Collections	id, handle, reported vs. observed product counts	collectionId, handle, productsCountReported, productsCountObserved, productsCountDelta

Need more Shopify data?

If you also need verified store-owner contact details and lead scoring, API-Empire publishes Shopify Merchant Scraper (verified contacts and lead rating). For stock photography and product-change tracking on a store, there's Shopify Scraper (stock images and product updates). And for a broader catalogue export without the collection/keyword scoping this Actor adds, see Shopify Store Scraper (product catalogue data).

How does Shopify Products Scraper By Collection & Keyword differ from the official Shopify API?

Shopify's own Admin API and Storefront API both require a per-store access token issued by that store's owner — there is no official endpoint for reading products across many arbitrary third-party stores you don't own or administer. This Actor instead reads the same public catalogue endpoints a shopper's browser already loads (products.json, collections.json, the storefront search), which need no login and no per-store authorization, across as many storefronts as you list.

Feature	Shopify Admin/Storefront API	Shopify Products Scraper
Access setup	Per-store access token, issued by that store's owner (shopify.dev, checked 2026-08-15)	No token — reads each storefront's public catalogue endpoints
Cross-store queries	Not supported without individual authorization per store (shopify.dev, checked 2026-08-15)	✅ Any number of public storefronts in one run
Collection membership join	Requires separate calls per collection, stitched by the caller	✅ Built in, attached to every product row
In-store keyword search	Not exposed by the Admin API	✅ Runs the storefront's own search/suggest endpoint
Output format	Raw REST/GraphQL JSON, caller normalizes	Normalized rows, stable field names
Setup time	App creation, OAuth or custom-app token, scope approval	Fill in a store URL and run

The official APIs are the right choice if you own the store and need to write data back (create products, update inventory) or need customer-account-scoped data. Use this scraper when you need read-only catalogue data from stores you don't administer, across more than one storefront at once.

Why do developers and teams scrape Shopify product catalogues?

For AI engineers and agent builders

Product catalogues are common RAG and agent-tool data: index title, body_html (inside full_data.product), tags and price into a vector store for a shopping assistant, or wire this Actor as a tool call so an agent can look up live pricing and stock for a named store before answering. Because every row is typed JSON with stable field names, there's no HTML to strip before it reaches the model's context window.

For pricing and competitive-intelligence analysts

Track a competitor's price and compare_at_price across their whole catalogue or a specific collection over repeated runs to see markdown patterns, new SKUs (product_id), and category (collectionTitles) shifts — without a Shopify account.

For dropshippers and sourcing teams

Scope a run to a supplier's collections or search their store for a keyword, and get title, price, vendor, tags and every collection a product sits in back as one row — useful for building a sourcing shortlist or checking overlap between two stores' catalogues.

For developers building data products

Feed productUrls or collectionUrls on a schedule to keep a downstream catalogue, price-tracker, or search index in sync with a set of storefronts, using product_id and handle as stable keys across runs.

How to scrape Shopify products (step by step)

1. Open Shopify Products Scraper By Collection & Keyword on the Apify Store and click Try for free (or Run, if you already have it saved).
2. Add at least one store, collection, or product source — a store URL in startUrls, a collectionUrl/collectionHandle, or a productUrl. None of these fields is individually required; the run just needs at least one populated.
3. Set catalogueSource to choose the scope: store for the whole catalogue, collections for named collections, keyword for the store's own search, or productUrls for a hand-picked list.
4. Click Start in the Apify Console.
5. When the run finishes, download the dataset as JSON, CSV, XML, or Excel, or pull it with the Apify API/apify_client.

What to do when Shopify changes its structure

The scraper is maintained against Shopify's public storefront endpoints, and the output schema — the field names and types on every row — stays stable, so downstream integrations keep working even if the underlying page markup changes.

⬇️ Input

Parameter	Required	Type	Description	Example Value
startUrls	No	array	One or more Shopify storefront URLs. Collection and product URLs are resolved to their store automatically, so this can stay empty if you only use those.	["https://deathwishcoffee.com"]
catalogueSource	No	string (enum)	Where products come from: store (whole catalogue, default), collections (named collections only), keyword (in-store search only), productUrls (specific product URLs only).	"collections"
collectionHandles	No	array	Collection handles to pull products from, used when catalogueSource is collections.	["accessories", "coffee-mugs"]
collectionUrls	No	array	Full collection URLs. The store is taken from the URL, so a store URL is optional when you use this.	["https://deathwishcoffee.com/collections/accessories"]
attachCollectionMembership	No	boolean	Adds collections, collectionTitles, collectionCount and primaryCollection to every product row. Off = a faster run, those fields left empty. Default true.	true
emitCollectionRows	No	boolean	Adds one row of type collection per collection, with id, handle, title, description, image, dates and product counts. Default true.	true
maxCollectionsToScan	No	integer, min 0	Caps how many of the store's collections are read when building membership. 0 = all. When the cap is hit, membershipComplete is false on every product row for that store. Default 150.	150
searchTerms	No	array	Keywords run through the store's own product search, used when catalogueSource is keyword. Each term is a separate search, recorded on every resulting row.	["mug", "cold brew"]
maxMatchesPerTerm	No	integer, min 1, max 10	Matches kept per term. The storefront's own product search never returns more than 10 per term regardless of this value. Default 10.	5
productUrls	No	array	Full product URLs, used when catalogueSource is productUrls. The store is taken from the URL.	["https://deathwishcoffee.com/products/ride-or-die-mug"]
maxProductsPerSource	No	integer, min 0	Caps products returned per store, per collection, or per search term. 0 = no cap. Default 200.	200
maxItems	No	integer, min 0	Legacy alias kept for input built against the original Shopify Products Scraper. Identical to maxProductsPerSource, which wins when both are set.	0
proxyConfiguration	No	object	Apify Proxy settings for reaching the storefront. Leave as-is unless a store is unreachable on the default route.	{ "useApifyProxy": false }

Example input:

{
  "startUrls": ["https://deathwishcoffee.com"],
  "catalogueSource": "collections",
  "collectionHandles": ["coffee-mugs"],
  "collectionUrls": [],
  "attachCollectionMembership": true,
  "emitCollectionRows": true,
  "maxCollectionsToScan": 150,
  "searchTerms": [],
  "maxMatchesPerTerm": 10,
  "productUrls": [],
  "maxProductsPerSource": 200,
  "maxItems": 0,
  "proxyConfiguration": { "useApifyProxy": false }
}

⚠️ The most common input mistake: setting catalogueSource to collections or keyword without filling in collectionHandles/collectionUrls or searchTerms. The run doesn't fall back to the whole store in that case — it logs a warning and scopes nothing, so no products come back.

⬆️ Output

Every result is a typed JSON row pushed to the Actor's default dataset, exportable as JSON, CSV, XML, Excel, or HTML table. Two row shapes share the same dataset, distinguished by type.

Scraped product

{
  "type": "product",
  "isChild": false,
  "store_url": "https://deathwishcoffee.com",
  "product_url": "https://deathwishcoffee.com/products/ride-or-die-mug",
  "product_id": 6741932589144,
  "handle": "ride-or-die-mug",
  "title": "Ride or Die Mug",
  "vendor": "Death Wish Coffee",
  "product_type": "Mugs",
  "price": "14.99",
  "compare_at_price": "19.99",
  "tags": ["mug", "accessories", "black"],
  "discoveredVia": "collection",
  "discoveryKey": "coffee-mugs",
  "collections": ["coffee-mugs", "accessories", "all"],
  "collectionTitles": ["Coffee Mugs", "Accessories", "All Products"],
  "collectionCount": 3,
  "primaryCollection": "coffee-mugs",
  "membershipComplete": true,
  "publishedAt": "2023-04-11T12:00:00-04:00",
  "updatedAt": "2026-07-30T09:14:22-04:00",
  "total_found": 42,
  "successful": 7,
  "scrapedAt": "2026-08-15T10:03:41Z",
  "full_data": {
    "product": {
      "id": 6741932589144,
      "title": "Ride or Die Mug",
      "handle": "ride-or-die-mug",
      "body_html": "<p>15oz ceramic mug...</p>",
      "vendor": "Death Wish Coffee",
      "product_type": "Mugs",
      "tags": "mug, accessories, black",
      "published_at": "2023-04-11T12:00:00-04:00",
      "updated_at": "2026-07-30T09:14:22-04:00",
      "variants": [
        {
          "id": 40123456789,
          "title": "Default Title",
          "price": "14.99",
          "compare_at_price": "19.99",
          "sku": "MUG-ROD-001",
          "inventory_quantity": 214
        }
      ],
      "images": [
        { "id": 30123456789, "src": "https://cdn.shopify.com/.../ride-or-die-mug.jpg" }
      ],
      "options": [{ "name": "Title", "values": ["Default Title"] }]
    }
  }
}

total_found is the number of products the current batch (this collection, this search term, or the whole-store crawl) returned after any cap was applied — not the store's grand total. successful is a running 1-based count of product rows emitted for this store so far in the run. full_data.product carries the storefront's complete raw product object exactly as Shopify returns it, so nothing is dropped even where it isn't broken out into a top-level field.

Scraped collection

{
  "type": "collection",
  "isChild": true,
  "store_url": "https://deathwishcoffee.com",
  "collectionId": 278912345678,
  "handle": "coffee-mugs",
  "title": "Coffee Mugs",
  "productsCountReported": 45,
  "productsCountObserved": 42,
  "productsCountDelta": 3,
  "collectionUrl": "https://deathwishcoffee.com/collections/coffee-mugs",
  "description": "<p>Mugs built for the coffee you actually drink.</p>",
  "imageUrl": "https://cdn.shopify.com/.../coffee-mugs.jpg",
  "publishedAt": "2022-01-05T08:00:00-05:00",
  "updatedAt": "2026-06-19T11:42:03-04:00",
  "scrapedAt": "2026-08-15T10:03:41Z"
}

Collection rows are also mirrored into a separate named dataset for the run (<runId>-collections) when the run has an accessible run ID, in addition to sitting in the main dataset — so you can pull just the collections without filtering by type.

Every dataset row, product or collection, is billed under a single pay-per-event, row_result — there's no separate charge per field or per page, and no uncharged accounting rows are pushed to the dataset. If a store fails outright (password-gated, taken down, or temporarily unreachable), no rows are pushed for it at all; check the run log or the key-value store's OUTPUT record for that store's status instead of expecting an empty-result row.

How does Shopify Products Scraper By Collection & Keyword compare to other Shopify scrapers?

Feature	Shopify Products Scraper By Collection & Keyword	Generic alternative
Output format	Typed JSON, stable field names	❌ Often raw HTML or ad-hoc keys
Collection membership on product rows	✅ Built in, toggleable	❌ Usually a separate manual join
Scope control	✅ Store, collections, keyword search, or product-URL list, in one Actor	❌ Typically one fixed scope
In-store keyword search	✅ Uses the storefront's own search endpoint	❌ Not commonly offered
Proxy/anti-bot handling	✅ Paced requests, automatic Apify Proxy escalation, retries	Varies
Schema stability across runs	✅ Field names and types don't change between runs	Varies

If you're building an AI agent or RAG pipeline, the output-format row is the decision-maker — parsing HTML inside an agent loop is a reliability failure mode, not a feature. A stable, typed schema is what makes catalogue data safe to feed straight into an LLM context window or a vector store.

How many results can you scrape with Shopify Products Scraper By Collection & Keyword?

There's no hard cap by default — maxProductsPerSource is 0 unless you set it, which means no cap per store/collection/search term. Two limits come from Shopify itself, not from this Actor: the listing endpoints (products.json, collections.json) return at most 250 products per page no matter what you ask for, so the Actor pages through automatically; and the storefront's own search never returns more than 10 matches per term (maxMatchesPerTerm above 10 is silently treated as 10). As an internal safety net, pagination on any single source stops after 400 pages (roughly 100,000 products) so one runaway catalogue can't run indefinitely — this only matters for exceptionally large stores.

Integrate Shopify Products Scraper By Collection & Keyword and automate your workflow

Shopify Products Scraper By Collection & Keyword works with any language or tool that can send an HTTP request.

REST API integration

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("API-Empire/shopify-products-scraper-by-collection-keyword").call(
    run_input={
        "startUrls": ["https://deathwishcoffee.com"],
        "catalogueSource": "collections",
        "collectionHandles": ["coffee-mugs"],
    }
)

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["type"], row.get("title") or row.get("handle"), row.get("price"))

Works in Python, Node.js, Go, Ruby, cURL — anything that can call the Apify API with your token.

Automation platforms (n8n, Make, LangChain)

Apify's own Apify node in n8n and Apify module in Make can run this Actor by its Actor ID, pass catalogueSource, startUrls, collectionUrls or searchTerms as node inputs, and hand the resulting dataset straight to the next step in your flow. For agent frameworks like LangChain, the Apify integration exposes any Actor — including this one — as a callable tool, so an agent can request a store's catalogue mid-conversation and get structured rows back.

Is it legal to scrape Shopify product data?

Yes — scraping publicly published product and collection data is generally legal, and Shopify Products Scraper By Collection & Keyword only reads what a storefront already serves to any visitor's browser, without logging in or bypassing any access control. This is business and product data — titles, prices, vendors, collections — not personal data about individuals, so GDPR does not apply to it; what does apply is the target store's own terms of service and, in some jurisdictions, database-rights protections on the catalogue as a compiled work. Consult legal counsel for commercial use cases, especially high-volume or resale use of a specific store's catalogue.

Frequently asked questions

Does Shopify Products Scraper By Collection & Keyword work without a Shopify account?

Yes. It reads each storefront's public catalogue endpoints (products.json, collections.json, search/suggest.json) directly — no Shopify account, app installation, or API key is needed, for the stores you're scraping or for you.

How often is the scraped data updated?

Data is fetched live on every run — there's no cache. Prices, stock-linked fields, and collection membership reflect what the storefront returns at the moment the run executes.

What happens if a product or collection has been removed?

A deleted product returns a 404 from Shopify and is skipped rather than emitted as a broken row; a request for that product simply produces no row. A collection with no readable products still gets a collection row if emitCollectionRows is on, but its productsCountObserved is left null if the collection could never be read at all, rather than reported as a false zero.

Can I scrape a password-protected Shopify store?

No. If a storefront is gated behind Shopify's own store password, the scraper detects the gate and reports the store as access-restricted rather than guessing at data — it never attempts to bypass a login wall.

Can I run this scraper on a schedule?

Yes, through the Apify Console's built-in Schedules, or the Apify API — set a cron expression and the same input runs automatically, with results in a fresh dataset each time.

Does Shopify Products Scraper By Collection & Keyword work for AI agent workflows and LLM pipelines?

Yes — it's callable as a standard HTTP endpoint through the Apify API by any agent framework. Every response is typed JSON with stable field names, so there's no HTML-parsing step between the scraper and an LLM's context window.

How does Shopify Products Scraper By Collection & Keyword handle anti-bot protection?

Requests are paced with a shared minimum interval, retried with exponential backoff on retriable statuses (429, 503, and similar), and automatically escalated from a direct connection to Apify Proxy (and then to residential proxy) if a storefront starts blocking. None of this bypasses a password gate — it only helps against ordinary rate-limiting and IP-based throttling.

Does Shopify Products Scraper By Collection & Keyword return data in a format LLMs can use directly?

Yes. Output is typed, normalized JSON with stable field names and no HTML — pass it directly into an LLM context window, index it into a vector store, or route it through an agent tool call.

Can I use Shopify Products Scraper By Collection & Keyword without managing proxies?

Yes. Apify Proxy is used automatically when a storefront needs it; you only need to touch proxyConfiguration if a specific store is unreachable on the default route.

What happens when Shopify changes its structure or blocks the scraper?

The scraper is maintained against Shopify's public storefront endpoints, and the output schema stays stable — field names and types don't change on your end even if the underlying page markup does.

💬 Your feedback

Found a bug, or missing a field you need on a product or collection row? We want to know. Open an issue on the Actor's Apify Store page, or reach out through Apify Console support, and it'll go straight into the maintenance queue for this Actor.