# Shopware Products Scraper (`trovevault/shopware-products-scraper`) Actor Use this actor to collect public Shopware catalog data from storefront pages. It discovers product pages through product sitemaps and storefront HTML, normalizes titles, prices, stock signals, SKUs, images, categories, and product URLs for price monitoring, competitor analysis, and market research. - **URL**: https://apify.com/trovevault/shopware-products-scraper.md - **Developed by:** [Trove Vault](https://apify.com/trovevault) (community) - **Categories:** E-commerce, AI, Agents - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $0.85 / 1,000 products This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Shopware Products Scraper: Scrape Public Product Data from Shopware Stores Use this actor to collect public Shopware catalog data from storefront pages that expose products through SEO URLs, sitemaps, and structured markup. It discovers product pages through product sitemaps and storefront HTML, then normalizes titles, prices, stock signals, SKUs, images, categories, and product URLs for price monitoring, competitor analysis, and market research. ### Why use Shopware Products Scraper? - Use Shopware Products Scraper to collect normalized product data from one store or many stores without building a private integration first. - Use Shopware Products Scraper when you need repeatable output fields such as URL, title, SKU, brand, price, currency, availability, image, categories, and description. - Use Shopware Products Scraper when your workflow depends on the Apify API, scheduling, dataset exports, and pipeline fields such as `datasetId` and `runId`. ### What data can Shopware Products Scraper extract? | Field | Description | | --- | --- | | `url` | Product page URL that was scraped | | `title` | Product title from structured data or page markup | | `sku` | SKU or product code when it is visible publicly | | `brand` | Brand or manufacturer label | | `price` | Parsed product price | | `currency` | Currency code such as USD or EUR | | `available` | In-stock signal normalized to true or false | | `image` | Primary product image URL | | `categories` | Breadcrumb or category labels | | `sourceType` | Whether the record came from JSON-LD, meta tags, or a mix | | `discoveredVia` | Whether the URL was found through API, sitemap, or HTML | | `error`, `errorType`, `message`, `reason`, `fix` | Structured soft-fail fields for blocked, timeout, not-found, and network errors | ### Which use cases fit Shopware Products Scraper? #### Price monitoring Use Shopware Products Scraper to track visible prices, compare-at prices, and stock signals across public Shopware product pages. #### Competitor analysis Use Shopware Products Scraper to compare titles, prices, brands, and category structures across several stores in one run. #### Catalog QA and market research Use Shopware Products Scraper to spot weak product pages, missing images, missing descriptions, and assortment changes while supporting broader market research projects. ### Why use Shopware Products Scraper instead of alternatives? | Feature | Alternatives | This Actor | | --- | --- | --- | | Public storefront scraping | Often needs store-specific scripts | One reusable actor pattern | | Output structure | Raw HTML or partial JSON | Normalized dataset fields | | Monitoring | Manual reruns | Ready for schedules and API workflows | | Error handling | Whole run fails or hides bad pages | Structured soft-fail rows | ### How do you use Shopware Products Scraper? 1. Add one or more store root URLs in `storeUrls`. 2. Start with `maxProducts` at 3 to 5 for a smoke test, then raise it for production runs. 3. Keep `discoveryMode` on `AUTO` unless you are debugging sitemap or HTML discovery. 4. Run the actor and export the dataset through the Apify Console or API. ### How does Shopware Products Scraper work? 1. Shopware Products Scraper normalizes storefront roots and validates the input. 2. Shopware Products Scraper discovers product URLs through public API, sitemap, and HTML paths when they are available. 3. Shopware Products Scraper extracts Product JSON-LD, product meta tags, breadcrumbs, images, stock signals, and pricing data from each product page. 4. Shopware Products Scraper emits product rows and structured error rows, then writes a `RUN_SUMMARY` record to the key-value store. ### What input does Shopware Products Scraper accept? | Field | Type | Default | Description | | --- | --- | --- | --- | | `storeUrls` | array of strings | demo store prefill | Store root URLs or domains to scrape | | `maxProducts` | integer | 25 | Maximum number of products to extract per store | | `discoveryMode` | string | `AUTO` | Chooses between auto, sitemap-only, and HTML-only discovery | | `requestTimeoutSecs` | integer | 30 | Timeout used for storefront requests | | `includeRawJsonLd` | boolean | false | Includes the parsed Product JSON-LD object in the output | | `proxyConfiguration` | object | Apify Proxy off | Applies proxy settings to discovery, product pages, and supported API calls | | `datasetId` | string | blank | Appends results to an existing dataset as well as the run dataset | | `runId` | string | blank | Associates results with a previous step in a pipeline | #### Example input for a quick test ```json { "storeUrls": [ "https://demo.themeware.design/modern/en/" ], "maxProducts": 10, "discoveryMode": "AUTO" } ```` #### Example input for a pipeline run ```json { "storeUrls": [ "https://demo.themeware.design/modern/en/" ], "maxProducts": 25, "discoveryMode": "AUTO", "datasetId": "your-shared-dataset-id", "runId": "previous-run-id" } ``` ### What does the output from Shopware Products Scraper look like? Shopware Products Scraper writes one dataset item per product and uses the same dataset for structured error rows. | Output use case | Main fields | | --- | --- | | Product analysis | `title`, `price`, `currency`, `brand`, `sku`, `available` | | URL mapping | `url`, `canonicalUrl`, `slug`, `categories` | | Discovery QA | `sourceType`, `discoveredVia` | | Soft-fail debugging | `error`, `errorType`, `reason`, `fix` | ```json { "store": "demo.themeware.design", "platform": "Shopware", "title": "Product page with large image", "url": "https://demo.themeware.design/modern/en/Product-page-with-large-image/TWT10012", "canonicalUrl": "https://demo.themeware.design/modern/en/Product-page-with-large-image/TWT10012", "slug": "TWT10012", "sku": "TWT10012", "brand": "Themeware Demo", "price": 99.95, "compareAtPrice": null, "currency": "EUR", "available": true, "image": "https://demo.themeware.design/modern/en/sample-image.jpg", "images": [ "https://demo.themeware.design/modern/en/sample-image.jpg" ], "imageCount": 1, "categories": [ "Shop" ], "description": "Product page with large image product page extracted from a public Shopware storefront.", "sourceType": "jsonld", "discoveredVia": "sitemap", "attributes": null, "lastModified": null, "rawJsonLd": null, "scrapedAt": "2026-04-21T09:00:00.000Z", "error": false, "errorType": null, "message": null, "reason": null, "fix": null } ``` ### How can you use Shopware Products Scraper via the Apify API? Yes. Shopware Products Scraper works through the Apify API like any other actor. ```bash curl -X POST "https://api.apify.com/v2/acts/trovevault~shopware-products-scraper/runs?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "storeUrls": ["https://demo.themeware.design/modern/en/"], "maxProducts": 10, "discoveryMode": "AUTO" }' ``` ### What are the limitations of Shopware Products Scraper? - Shopware Products Scraper only works with data exposed publicly on the storefront. - Shopware Products Scraper depends on the store exposing enough public product links for discovery. - Shopware Products Scraper can still be blocked by rate limits, bot protection, or regional filtering. ### FAQ #### Does it work on all Shopware stores? No. Shopware Products Scraper works best on stores that expose product sitemaps, JSON-LD product blocks, or stable product page markup. #### Can I scrape multiple stores at once? Yes. Add multiple root URLs in `storeUrls` and Shopware Products Scraper will process them in one run. #### How do I monitor for changes over time? Schedule Shopware Products Scraper in Apify and append each run to a shared dataset or export destination. #### What if the site returns a 403 error? Shopware Products Scraper keeps the run successful and emits a structured row with `errorType: "BLOCKED"`. #### How do I schedule the actor to run automatically? Create a schedule in the Apify Console and attach your preferred input. #### Can I use this via the Apify API? Yes. The curl example above is enough to start a run and collect the dataset programmatically. #### Can I use this through an MCP Server? Yes. If your MCP workflow can trigger Apify actors or consume Apify datasets, you can use Shopware Products Scraper in a broader automation. #### Is scraping Shopware data legal? That depends on your jurisdiction, your use case, the site terms, and the type of data you collect. Shopware Products Scraper is intended for public product data, but you should review the target site terms and get legal guidance when needed. ### Related actors If you also work across storefront platforms, see these related TroveVault actors: Magento and Adobe Commerce Products Scraper, BigCommerce Products Scraper, PrestaShop Products Scraper, Salesforce Commerce Cloud Products Scraper. ### Changelog - `0.1`: First TroveVault-aligned release of Shopware Products Scraper with public storefront discovery, normalized product output, structured soft-fail rows, and daily test input. ### Feedback Open an issue if you need another discovery path, output field, or storefront pattern supported by Shopware Products Scraper. # Actor input Schema ## `storeUrls` (type: `array`): One or more Shopware storefront domains or root URLs to scrape. The actor uses each value as the store entry point, discovers product pages through public storefront discovery paths, and extracts URLs, titles, prices, stock signals, SKUs, images, and categories. Accepts bare hostnames like `demo.themeware.design` or full URLs like `https://demo.themeware.design/modern/en/`. Add multiple stores to compare assortments side by side in one run. Use storefront roots instead of product page URLs for the best coverage. ## `maxProducts` (type: `integer`): Maximum number of products to extract per store. The actor stops after this many product detail pages return usable product data. Use small values like 3 to 5 for quick smoke tests and daily checks, and larger values like 25 to 200 for fuller catalog snapshots. Higher values return more products but take longer and increase the chance of hitting weaker storefront pages. ## `discoveryMode` (type: `string`): Controls how product page URLs are discovered. `AUTO` uses the strongest public path first for Shopware, then falls back to other public sources until it reaches the requested cap. `SITEMAP_ONLY` is best when the storefront publishes clean product sitemaps. `HTML_ONLY` is useful for debugging pages when sitemap coverage is weak. Recommended: `AUTO` for most runs. ## `requestTimeoutSecs` (type: `integer`): HTTP timeout used for storefront requests. The actor applies this limit to sitemap downloads, HTML pages, and API discovery calls. Use 20 to 30 seconds for most stores and increase it when the storefront is slow or far from your region. Higher values reduce false timeouts but make failed requests take longer to finish. ## `includeRawJsonLd` (type: `boolean`): When enabled, the actor includes the parsed Product JSON-LD object in each successful dataset item when the storefront exposes it. This is useful for QA, debugging, and schema comparison across stores. Leave it disabled for smaller datasets unless you need the raw source object. ## `proxyConfiguration` (type: `object`): Proxy settings for storefront requests. The actor applies this configuration to discovery requests, sitemap downloads, product pages, and supported API calls. Enable Apify Proxy, ideally residential groups, when the target store returns HTTP 403, 429, 526, Cloudflare blocks, or country-specific bot defenses. Leave it disabled for open storefronts to keep runs simpler and lower cost. ## `datasetId` (type: `string`): ID of an existing Apify dataset to append results to, in addition to the default run dataset. Leave blank to use only the default run dataset. ## `runId` (type: `string`): ID of an existing Apify actor run to associate results with. Used for pipeline chaining and multi-step workflows. Leave blank for standalone runs. ## Actor input object example ```json { "storeUrls": [ "https://demo.themeware.design/modern/en/" ], "maxProducts": 3, "discoveryMode": "AUTO", "requestTimeoutSecs": 30, "includeRawJsonLd": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `dataset` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "storeUrls": [ "https://demo.themeware.design/modern/en/" ], "maxProducts": 3 }; // Run the Actor and wait for it to finish const run = await client.actor("trovevault/shopware-products-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "storeUrls": ["https://demo.themeware.design/modern/en/"], "maxProducts": 3, } # Run the Actor and wait for it to finish run = client.actor("trovevault/shopware-products-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "storeUrls": [ "https://demo.themeware.design/modern/en/" ], "maxProducts": 3 }' | apify call trovevault/shopware-products-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=trovevault/shopware-products-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopware Products Scraper", "description": "Use this actor to collect public Shopware catalog data from storefront pages. It discovers product pages through product sitemaps and storefront HTML, normalizes titles, prices, stock signals, SKUs, images, categories, and product URLs for price monitoring, competitor analysis, and market research.", "version": "0.1", "x-build-id": "HCLph3ZVo5oyUjycJ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/trovevault~shopware-products-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-trovevault-shopware-products-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/trovevault~shopware-products-scraper/runs": { "post": { "operationId": "runs-sync-trovevault-shopware-products-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/trovevault~shopware-products-scraper/run-sync": { "post": { "operationId": "run-sync-trovevault-shopware-products-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "storeUrls" ], "properties": { "storeUrls": { "title": "Store URLs", "type": "array", "description": "One or more Shopware storefront domains or root URLs to scrape. The actor uses each value as the store entry point, discovers product pages through public storefront discovery paths, and extracts URLs, titles, prices, stock signals, SKUs, images, and categories. Accepts bare hostnames like `demo.themeware.design` or full URLs like `https://demo.themeware.design/modern/en/`. Add multiple stores to compare assortments side by side in one run. Use storefront roots instead of product page URLs for the best coverage.", "items": { "type": "string" } }, "maxProducts": { "title": "Max Products", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of products to extract per store. The actor stops after this many product detail pages return usable product data. Use small values like 3 to 5 for quick smoke tests and daily checks, and larger values like 25 to 200 for fuller catalog snapshots. Higher values return more products but take longer and increase the chance of hitting weaker storefront pages.", "default": 25 }, "discoveryMode": { "title": "Discovery Mode", "enum": [ "AUTO", "SITEMAP_ONLY", "HTML_ONLY" ], "type": "string", "description": "Controls how product page URLs are discovered. `AUTO` uses the strongest public path first for Shopware, then falls back to other public sources until it reaches the requested cap. `SITEMAP_ONLY` is best when the storefront publishes clean product sitemaps. `HTML_ONLY` is useful for debugging pages when sitemap coverage is weak. Recommended: `AUTO` for most runs.", "default": "AUTO" }, "requestTimeoutSecs": { "title": "Request Timeout (seconds)", "minimum": 5, "maximum": 120, "type": "integer", "description": "HTTP timeout used for storefront requests. The actor applies this limit to sitemap downloads, HTML pages, and API discovery calls. Use 20 to 30 seconds for most stores and increase it when the storefront is slow or far from your region. Higher values reduce false timeouts but make failed requests take longer to finish.", "default": 30 }, "includeRawJsonLd": { "title": "Include Raw JSON-LD", "type": "boolean", "description": "When enabled, the actor includes the parsed Product JSON-LD object in each successful dataset item when the storefront exposes it. This is useful for QA, debugging, and schema comparison across stores. Leave it disabled for smaller datasets unless you need the raw source object.", "default": false }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Proxy settings for storefront requests. The actor applies this configuration to discovery requests, sitemap downloads, product pages, and supported API calls. Enable Apify Proxy, ideally residential groups, when the target store returns HTTP 403, 429, 526, Cloudflare blocks, or country-specific bot defenses. Leave it disabled for open storefronts to keep runs simpler and lower cost.", "default": { "useApifyProxy": false } }, "datasetId": { "title": "Dataset ID (optional)", "type": "string", "description": "ID of an existing Apify dataset to append results to, in addition to the default run dataset. Leave blank to use only the default run dataset." }, "runId": { "title": "Run ID (optional)", "type": "string", "description": "ID of an existing Apify actor run to associate results with. Used for pipeline chaining and multi-step workflows. Leave blank for standalone runs." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```