# Shopify & Woocomerce Products Drop-Shipping Scrapper (`yasserj/shopify-woocomerce-products-drop-shipping-scrapper`) Actor Effortlessly pull products from any Shopify or WooCommerce shop. This actor handles the heavy lifting: it scrapes deep product details, translates content into your target language, and generates a ready-to-upload Shopify CSV. Optimized for high-volume catalog scraping and currency conversion. - **URL**: https://apify.com/yasserj/shopify-woocomerce-products-drop-shipping-scrapper.md - **Developed by:** [Yasser](https://apify.com/yasserj) (community) - **Categories:** E-commerce, Agents, AI - **Stats:** 5 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Next-Gen E-Commerce Product Scraper A powerful, robust, and versatile web scraping solution for extracting product data from various e-commerce websites. Built specifically for Apify, this actor leverages Puppeteer to handle JavaScript-rendered content, automatically traverses domains, translates content, converts currencies, and generates ready-to-use Shopify CSVs. ### 🌟 Key Features - **Multi-Domain Crawling**: Capable of discovering product links across entire websites just by providing the root domains. - **AI-Powered Categorization**: Automatically generates detailed Shopify product categories using OpenAI (optional, requires API key). - **Automated Translation**: Translates titles and descriptions natively to a target language (e.g. `EN`, `DE`, `AR`). - **Currency Conversion**: Dynamically converts local prices to a target currency using live exchange rates, with customizable markups and margins. - **Smart Stock and Price Extraction**: Deep DOM parsing logic to identify stock status, variations, and pricing using heuristic analysis. - **Shopify-Ready Exports**: The scraper seamlessly exports directly to the Apify dataset using standard Shopify CSV columns (`Handle`, `Title`, `Body (HTML)`, `Vendor`, `Type`, `Tags`, `Image Src`, `Variant Price`, etc.), ready for immediate import to your Shopify store. - **Resilience & Proxy Support**: Built-in support for Apify Proxies to bypass rate limits and anti-bot systems. ### 🎯 Use Cases - **Drop-Shipping Automation**: Effortlessly scrape competitor catalogs, apply price markups, and instantly import the items to your Shopify store. - **Market Research**: Aggregate pricing and availability across multiple regional domains in a normalized currency. - **Content Localization**: Scrape foreign e-commerce sites and dynamically translate descriptions for local markets. ### ⚙️ Input Parameters The actor is highly configurable via JSON or the Apify UI: | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | **`urls`** | Array | Yes | List of domain URLs to process (e.g., `["https://example-shop.com"]`). | | **`targetCountry`** | Select | Yes | Target country context (e.g., `EG`, `DE`, `NG`). Default: `EG`. | | **`concurrency`** | Integer | No | Number of domains to process simultaneously (1-5). Default: `1`. | | **`targetLanguage`** | String | No | Translates titles and descriptions to this language code (e.g., `EN`, `DE`). Default: `EN`. | | **`baseCurrency`** | String | No | The source currency (e.g., `EGP`, `EUR`). Default: `AUTO`. | | **`targetCurrency`** | String | No | Currency to convert final prices to (e.g., `USD`, `NGN`). | | **`priceMultiplier`** | Number | No | Apply a markup multiplier to final prices. Default: `1`. | | **`exchangeRateMargin`** | Number | No | Adds a safety margin to the exchange rate. Default: `0`. | | **`checkStockInfo`** | Boolean | No | Determines whether the scraper should deeply verify stock. Default: `true`. | | **`openaiApiKey`** | String | No | Optional API key to enable AI-powered Shopify categorization. | | **`proxyConfiguration`** | Proxy | No | Apify Proxy configuration to avoid IP bans. | #### Example Input configuration ```json { "urls": [ "https://www.example.com" ], "targetCountry": "DE", "concurrency": 2, "targetLanguage": "EN", "baseCurrency": "EUR", "targetCurrency": "USD", "priceMultiplier": 1.5, "checkStockInfo": true } ```` ### 📊 Output Format The actor pushes results directly to the default **Apify Dataset** as soon as a product is fully processed. The output adheres to standard **Shopify CSV format** fields to ensure maximum compatibility. Example extracted fields: - `Handle` (URL slug) - `Title` (Translated to target language) - `Body (HTML)` (Cleaned product description) - `Vendor` (Extracted domain/brand name) - `Product Category` (Generated via AI if enabled) - `Tags` - `Variant Price` (Automatically converted with applied markups) - `Variant Inventory Tracker` (shopify) - `Variant Inventory Qty` (Extracted stock levels) - `Image Src` (High-res image link) - `Image Position` (1) You can easily download the results as a `CSV` file from the Apify Console and upload them directly into your Shopify dashboard. ### 🛠 Troubleshooting - **No Products Found**: Some target websites might use extremely obscure DOM architectures for products. - **Timeouts**: If you set concurrency too high, the Apify Actor may run out of memory. Try keeping `concurrency` close to `1` or `2` for complex sites. - **Missing Stock Data**: Sometimes stock statuses are hidden behind network requests that require deeper analysis or specialized proxy configurations. If you like the actor or need custom modifications, feel free to reach out or fork the logic! # Actor input Schema ## `urls` (type: `array`): List of domain URLs to process. ## `targetCountry` (type: `string`): Country code for scraping defaults (e.g. EG, DE, NG). ## `concurrency` (type: `integer`): Number of domains to process concurrently. ## `targetLanguage` (type: `string`): Language to translate product titles and descriptions to (e.g. EN, DE, AR). ## `baseCurrency` (type: `string`): The currency of the scraped website (e.g. EGP, EUR). Defaults to AUTO to detect from data, or USD. ## `targetCurrency` (type: `string`): The currency to convert prices to (e.g. NGN, USD). Leave empty to skip conversion. ## `priceMultiplier` (type: `number`): Multiply the final price by this amount. Default is 1 (no markup). ## `exchangeRateMargin` (type: `number`): Add this value to the exchange rate. Default is 0. ## `checkStockInfo` (type: `boolean`): If disabled, products will default to in-stock with quantity 10. ## `allowTranslation` (type: `boolean`): If checked, product data will be translated to the Target Language. If unchecked, translation will be skipped. ## `openaiApiKey` (type: `string`): Optional: Provide your OpenAI API key to use GPT for product categorization. ## `proxyConfiguration` (type: `object`): Select proxies to be used by your crawler. ## Actor input object example ```json { "urls": [ "https://americanfurnitureegypt.com/" ], "targetCountry": "EG", "concurrency": 1, "targetLanguage": "EN", "baseCurrency": "AUTO", "priceMultiplier": 1, "exchangeRateMargin": 0, "checkStockInfo": true, "allowTranslation": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "https://americanfurnitureegypt.com/" ] }; // Run the Actor and wait for it to finish const run = await client.actor("yasserj/shopify-woocomerce-products-drop-shipping-scrapper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": ["https://americanfurnitureegypt.com/"] } # Run the Actor and wait for it to finish run = client.actor("yasserj/shopify-woocomerce-products-drop-shipping-scrapper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "https://americanfurnitureegypt.com/" ] }' | apify call yasserj/shopify-woocomerce-products-drop-shipping-scrapper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=yasserj/shopify-woocomerce-products-drop-shipping-scrapper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopify & Woocomerce Products Drop-Shipping Scrapper", "description": "Effortlessly pull products from any Shopify or WooCommerce shop. This actor handles the heavy lifting: it scrapes deep product details, translates content into your target language, and generates a ready-to-upload Shopify CSV. Optimized for high-volume catalog scraping and currency conversion.", "version": "0.0", "x-build-id": "9q7OnopPc7R0fICMy" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/yasserj~shopify-woocomerce-products-drop-shipping-scrapper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-yasserj-shopify-woocomerce-products-drop-shipping-scrapper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/yasserj~shopify-woocomerce-products-drop-shipping-scrapper/runs": { "post": { "operationId": "runs-sync-yasserj-shopify-woocomerce-products-drop-shipping-scrapper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/yasserj~shopify-woocomerce-products-drop-shipping-scrapper/run-sync": { "post": { "operationId": "run-sync-yasserj-shopify-woocomerce-products-drop-shipping-scrapper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "urls" ], "properties": { "urls": { "title": "URLs to scrape", "type": "array", "description": "List of domain URLs to process.", "items": { "type": "string" } }, "targetCountry": { "title": "Target Country", "enum": [ "EG", "DE", "NG", "US" ], "type": "string", "description": "Country code for scraping defaults (e.g. EG, DE, NG).", "default": "EG" }, "concurrency": { "title": "Concurrency", "minimum": 1, "maximum": 5, "type": "integer", "description": "Number of domains to process concurrently.", "default": 1 }, "targetLanguage": { "title": "Target Language", "type": "string", "description": "Language to translate product titles and descriptions to (e.g. EN, DE, AR).", "default": "EN" }, "baseCurrency": { "title": "Base Currency", "type": "string", "description": "The currency of the scraped website (e.g. EGP, EUR). Defaults to AUTO to detect from data, or USD.", "default": "AUTO" }, "targetCurrency": { "title": "Target Currency", "type": "string", "description": "The currency to convert prices to (e.g. NGN, USD). Leave empty to skip conversion." }, "priceMultiplier": { "title": "Price Multiplier / Markup", "type": "number", "description": "Multiply the final price by this amount. Default is 1 (no markup).", "default": 1 }, "exchangeRateMargin": { "title": "Exchange Rate Margin", "type": "number", "description": "Add this value to the exchange rate. Default is 0.", "default": 0 }, "checkStockInfo": { "title": "Check Stock Info", "type": "boolean", "description": "If disabled, products will default to in-stock with quantity 10.", "default": true }, "allowTranslation": { "title": "Allow Translation", "type": "boolean", "description": "If checked, product data will be translated to the Target Language. If unchecked, translation will be skipped.", "default": true }, "openaiApiKey": { "title": "OpenAI API Key", "type": "string", "description": "Optional: Provide your OpenAI API key to use GPT for product categorization." }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Select proxies to be used by your crawler." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```