# Price-Comparison Crawler (`xidik_inc/aabw-price-comparison`) Actor Crawls marketplaces (eBay + best-effort Google Shopping) and any product URL (via JSON-LD / OpenGraph) and returns normalized, deduped competitor listings for price comparison. - **URL**: https://apify.com/xidik\_inc/aabw-price-comparison.md - **Developed by:** [Khidayotullo Salakhitdinov](https://apify.com/xidik_inc) (community) - **Categories:** E-commerce, Developer tools, Automation - **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Price Comparison Crawler **Price-check any product across marketplaces in one run.** Give it product keywords (or product URLs) and it returns **normalized, deduped competitor listings** โ€” `title`, `price`, `currency`, `seller`, `rating`, `url`, `marketplace` โ€” ready to sort, compare, or feed into a pricing model. - ๐Ÿ”Ž Search **eBay** and **Google Shopping** by keyword - ๐Ÿ”— Or extract a price straight off **any product URL** (schema.org JSON-LD โ†’ OpenGraph โ†’ price meta) - ๐Ÿงน One clean, deduped schema across every source โ€” prices parsed to both major units and integer minor units, currency detected - ๐Ÿ†“ Free to use โ€” you pay Apify only for the platform compute/proxy you consume ### Input ```json { "queries": ["the ordinary niacinamide 10% serum"], "marketplaces": ["ebay", "google_shopping"], "startUrls": [{ "url": "https://example-shop.com/product/123" }], "country": "sg", "maxItemsPerQuery": 20, "proxyConfiguration": { "useApifyProxy": false } } ```` - **queries** + **marketplaces** โ€” runs each query through each search adapter. - **startUrls** โ€” pulls a price straight off any product page via schema.org JSON-LD โ†’ OpenGraph โ†’ price meta tags (no per-site adapter needed). - **country** โ€” region hint (eBay TLD, Google `gl`/`hl`). ### Sources | Adapter | Engine | Reliability | |---|---|---| | `ebay` | HTTP (server-rendered) | Solid from datacenter IPs โ€” the default. | | `google_shopping` | HTTP | **Best-effort** โ€” Google often serves a consent/bot wall to datacenter IPs; use a residential proxy. Returns `[]` cleanly when blocked. | | product URLs (JSON-LD) | HTTP | Works on most e-commerce product pages that emit schema.org `Product`. | ### Output One dataset row per listing: ```json { "title": "The Ordinary Niacinamide 10% + Zinc 1% 30ml", "price": 12.9, "priceMinor": 1290, "priceText": "S$12.90", "currency": "SGD", "seller": "beautystore_sg", "rating": 4.8, "reviewCount": 231, "availability": "InStock", "image": "https://โ€ฆ", "marketplace": "ebay", "query": "the ordinary niacinamide 10% serum", "country": "sg", "url": "https://www.ebay.com/itm/โ€ฆ", "scrapedAt": "2026-โ€ฆ" } ``` Field names follow tolerant families (`title` / `price` / `url` / `rating` / `seller`) so the output drops cleanly into a downstream price-comparison normalizer. ### Run locally ```bash npm install apify run --input='{"queries":["airpods pro 2"],"marketplaces":["ebay"],"country":"us"}' ``` ### Extending Add a marketplace by dropping a `src/adapters/.js` that exports `buildRequests(query, country)` and `parse($)`, then register it in `src/adapters/index.js`. JS-heavy marketplaces (Lazada/Shopee) need a `PlaywrightCrawler` + residential proxy โ€” documented as the next adapter tier. # Actor input Schema ## `queries` (type: `array`): Product names / keywords to price-check across the selected marketplaces. ## `marketplaces` (type: `array`): Search adapters to run for each query. eBay is reliable from datacenter IPs; Google Shopping is best-effort (may hit a consent wall without a residential proxy). ## `startUrls` (type: `array`): Specific product pages to extract a price from directly (via JSON-LD / OpenGraph / price meta tags). Works on most e-commerce sites and needs no marketplace adapter. ## `country` (type: `string`): Region hint passed to the search adapters (eBay TLD, Google gl/hl). ## `maxItemsPerQuery` (type: `integer`): Cap on normalized listings kept per query per marketplace. ## `proxyConfiguration` (type: `object`): Apify Proxy configuration. Recommended for Google Shopping and for higher volumes. ## Actor input object example ```json { "queries": [ "the ordinary niacinamide 10% serum" ], "marketplaces": [ "ebay" ], "country": "sg", "maxItemsPerQuery": 20, "proxyConfiguration": { "useApifyProxy": false } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "the ordinary niacinamide 10% serum" ] }; // Run the Actor and wait for it to finish const run = await client.actor("xidik_inc/aabw-price-comparison").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["the ordinary niacinamide 10% serum"] } # Run the Actor and wait for it to finish run = client.actor("xidik_inc/aabw-price-comparison").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "the ordinary niacinamide 10% serum" ] }' | apify call xidik_inc/aabw-price-comparison --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=xidik_inc/aabw-price-comparison", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Price-Comparison Crawler", "description": "Crawls marketplaces (eBay + best-effort Google Shopping) and any product URL (via JSON-LD / OpenGraph) and returns normalized, deduped competitor listings for price comparison.", "version": "0.1", "x-build-id": "Njf82QZA5ifw4Fyzq" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/xidik_inc~aabw-price-comparison/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-xidik_inc-aabw-price-comparison", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/xidik_inc~aabw-price-comparison/runs": { "post": { "operationId": "runs-sync-xidik_inc-aabw-price-comparison", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/xidik_inc~aabw-price-comparison/run-sync": { "post": { "operationId": "run-sync-xidik_inc-aabw-price-comparison", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "queries": { "title": "Product queries", "type": "array", "description": "Product names / keywords to price-check across the selected marketplaces.", "items": { "type": "string" } }, "marketplaces": { "title": "Marketplaces to search", "type": "array", "description": "Search adapters to run for each query. eBay is reliable from datacenter IPs; Google Shopping is best-effort (may hit a consent wall without a residential proxy).", "items": { "type": "string", "enum": [ "ebay", "google_shopping" ] }, "default": [ "ebay" ] }, "startUrls": { "title": "Product URLs (optional)", "type": "array", "description": "Specific product pages to extract a price from directly (via JSON-LD / OpenGraph / price meta tags). Works on most e-commerce sites and needs no marketplace adapter.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "country": { "title": "Country", "enum": [ "us", "gb", "sg", "my", "ph", "th", "vn", "id", "au" ], "type": "string", "description": "Region hint passed to the search adapters (eBay TLD, Google gl/hl).", "default": "sg" }, "maxItemsPerQuery": { "title": "Max items per query", "minimum": 1, "maximum": 200, "type": "integer", "description": "Cap on normalized listings kept per query per marketplace.", "default": 20 }, "proxyConfiguration": { "title": "Proxy", "type": "object", "description": "Apify Proxy configuration. Recommended for Google Shopping and for higher volumes.", "default": { "useApifyProxy": false } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```