# Amazon Product Scraper (`excitable_pickle/amazon-competitor-price-radar`) Actor Use this Amazon scraper to collect data based on URL and country from the Amazon website. Extract product information without using the Amazon API, including reviews, prices, descriptions, and Amazon Standard Identification Numbers (ASINs). Download data in various structured formats. - **URL**: https://apify.com/excitable\_pickle/amazon-competitor-price-radar.md - **Developed by:** [Pavan](https://apify.com/excitable_pickle) (community) - **Categories:** E-commerce - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README

Amazon Product Scraper Logo

## Amazon Product Scraper ### What is Amazon Product Scraper and how does it work? Amazon Product Scraper is a web scraping tool that enables you to extract product data from Amazon by using Amazon's subcategory URLs or product URLs. For example, this is an Amazon URL for the Computer Monitors subcategory: `https://www.amazon.com/s?i=specialty-aps&bbn=16225007011&rh=n%3A16225007011%2Cn%3A1292115011` Amazon subcategories typically have `/s` in the URL after the Amazon domain, so make sure your URL looks something like the example above. So just add one or more Amazon URLs in the input field and select the maximum number of items you want to scrape. Then download the data from the output schema. You can even get this data via API without having to log in to the Apify platform. ### Why scrape Amazon products? Scraping Amazon products can help you to: ➡️ Monitor the performance of categories and subcategories on Amazon so you can put the performance of your products into context. ➡️ Discover up-and-coming brands and products to benchmark your performance within its category based on views, conversions, and reviews. ➡️ Fine-tune your advertising and messaging. ➡️ Use Amazon data to give you an edge in competitive intelligence. For more inspiration, check out [how web scraping is transforming e-commerce analytics](https://apify.com/industries/ecommerce-and-retail). ### How many results can you scrape with Amazon Product Scraper? Amazon Product Scraper can return over 100 000 results on average. However, you have to keep in mind that scraping amazon.com has many variables to it and may cause the results to fluctuate case by case. There’s no one-size-fits-all-use-cases number. The maximum number of results may vary depending on the complexity of the input, location, and other factors. Some of the most frequent cases are: - website gives a different number of results depending on the type/value of the input - website has an internal limit that no scraper can cross - scraper has a limit that we are working on improving Therefore, while we regularly run Actor tests to keep the benchmarks in check, the results may also fluctuate without our knowing. The best way to know for sure for your particular use case is to do a test run yourself. ### How much will scraping Amazon Products cost you? When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary significantly. That's why the best course of action is to run a test scrape with a small sample of input data and limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do. [Watch this video](https://www.youtube.com/watch?v=kOy8C6_5ZU4) for a few helpful tips. And don't forget that choosing a higher plan will save you money in the long run. https://www.youtube.com/watch?v=kOy8C6\_5ZU4 ### 7 pages Amazon limitation Amazon sometimes restricts search terms to a maximum of 7 pages, please validate the number of pages in the URL. To overcome this limit, try to use one of the regular categories and combine it with the search terms. E.g. instead of just searching for `Diamond rings`, find `Jewelry` or `Women's rings` category and combine them with the search term. ### Is it legal to scrape Amazon? It is legal to scrape publicly available data such as product descriptions, prices, or ratings. Read our blog post on [the legality of web scraping](https://blog.apify.com/is-web-scraping-legal/) to learn more. ### How do I scrape products on Amazon? This simple [step-by-step guide](https://blog.apify.com/step-by-step-guide-to-scraping-amazon/#how-do-i-scrape-data-on-amazon) shows you how to use Amazon Product Scraper. ### Want more Amazon scraping options? Try these 👇 - [Amazon Best Sellers Scraper](https://apify.com/mihails/amazon-bestsellers-scraper) - [Amazon Reviews Scraper](https://apify.com/lhotanok/amazon-reviews-scraper) - [Amazon ASINs Scraper](https://apify.com/lhotanok/amazon-asins-scraper) ### Need to find product pairs between Amazon and another online shop? Use the [AI Product Matcher](https://apify.com/equidem/ai-product-matcher). This AI model allows you to compare items from different web stores, identifying exact matches and comparing real-time data obtained via web scraping. With the AI Product Matcher, you can use scraped product data to monitor product matches across the industry, implement dynamic pricing for your website, replace or complement manual mapping, and obtain realistic estimates against your competition for upcoming promo campaigns. Most importantly, it is relatively easy to get started with (just follow [this guide](https://blog.apify.com/product-matching-ai-pricing-intelligence-web-scraping/)), and it can check thousands of pairs of products. ### Input options When running Amazon Product Scraper, you need to configure what you want to scrape and how it should be extracted. This input is provided either as a JSON file or in the editor on the Apify platform. Most input fields have reasonable default values. For detailed descriptions and examples for all input fields, please visit the dedicated [Input tab](https://apify.com/vaclavrut/amazon-crawler/input-schema). Please note the following when using Amazon Product Scraper: - Price information might be missing if the delivery country has no sellers. Setting a specific "apifyProxyCountry" in proxy settings should help with this. - There could still be differences in pricing depending on the US state. - Amazon also shows different offers based on your proxy location. ### Sample output from Amazon Product Scraper ```json { "title": "SanDisk 1TB Extreme microSDXC UHS-I Memory Card with Adapter - Up to 190MB/s, C10, U3, V30, 4K, 5K, A2, Micro SD Card- SDSQXAV-1T00-GN6MA", "url": "https://www.amazon.com/dp/B09X7MPX8L", "asin": "B09X7MPX8L", "inStock": true, "inStockText": "Only 8 left in stock - order soon. Only 8 left in stock - order soon.", "brand": "SanDisk", "price": { "value": 145.5, "currency": "$" }, "listPrice": { "value": 299.99, "currency": "$" }, "shippingPrice": null, "stars": 4.8, "starsBreakdown": { "5star": 0.86, "4star": 0.09, "3star": 0.02, "2star": 0.01, "1star": 0.01 }, "reviewsCount": 36704, "answeredQuestions": 151, "breadCrumbs": "Electronics › Computers & Accessories › Computer Accessories & Peripherals › Memory Cards › Micro SD Cards", "thumbnailImage": "https://m.media-amazon.com/images/I/716kSUlHouL.__AC_SX300_SY300_QL70_FMwebp_.jpg", "description": null, "features": [ "Save time with card offload speeds of up to 190MB/s powered by SanDisk QuickFlow Technology (Up to 190MB/s read speeds, engineered with proprietary technology to reach speeds beyond UHS-I 104MB/s, requires compatible devices capable of reaching such speeds. Based on internal testing; performance may be lower depending upon host device interface, usage conditions and other factors. 1MB=1,000,000 bytes. SanDisk QuickFlow Technology is only available for 64GB, 128GB, 256GB, 400GB, 512GB, and 1TB capacities. 1GB=1,000,000,000 bytes and 1TB=1,000,000,000,000 bytes. Actual user storage less.)", "Pair with the SanDisk Professional PRO-READER SD and microSD to achieve maximum speeds (sold separately)", "Up to 130MB/s write speeds for fast shooting (Based on internal testing; performance may be lower depending upon host device interface, usage conditions and other factors. 1MB=1,000,000 bytes.)", "4K and 5K UHD-ready with UHS Speed Class 3 (U3) and Video Speed Class 30 (V30) (Compatible device required. Full HD (1920x1080), 4K UHD (3840 x 2160), and 5K UHD (5120 X 2880) support may vary based upon host device, file attributes and other factors. See HD page on SanDisk site. UHS Speed Class 3 (U3) designates a performance option designed to support real-time video recording with UHS-enabled host devices. Video Speed Class 30 (V30), sustained video capture rate of 30MB/s, designates a performance option designed to support real-time video recording with UHS-enabled host devices. See the SD Association’s official website.)", "Rated A2 for faster loading and in-app performance (A2 performance is 4000 read IOPS, 2000 write IOPS. Results may vary based on host device, app type and other factors)" ], "variantAsins": [], "reviewsLink": "/SanDisk-Extreme-microSDXC-Memory-Adapter/product-reviews/B09X7MPX8L?reviewerType=all_reviews", "delivery": "Thursday, January 26", "fastestDelivery": "Sunday, January 22", "returnPolicy": "Eligible for Return, Refund or Replacement within 30 days of receipt Eligible for Return, Refund or Replacement within 30 days of receipt", "support": "Free Amazon tech support included", "variantAttributes": [], "priceVariants": null, "seller": { "name": "Direct Suppliers US", "id": "A210SJF12S88M5", "url": "/gp/help/seller/at-a-glance.html/ref=dp_merchant_link?ie=UTF8&seller=A210SJF12S88M5&asin=B09X7MPX8L&ref_=dp_merchant_link&isAmazonFulfilled=1", "reviewsCount": null, "averageRating": null }, "bestsellerRanks": null, "locationText": "Select your address" } ``` ### Integrations and Amazon Product Scraper Last but not least, Amazon Product Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Or you can use webhooks to carry out an action whenever an event occurs, e.g. get a notification whenever Instagram API Scraper successfully finishes a run. ### Using Amazon Product Scraper with the Apify API The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more. To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client PyPI package. Check out the Apify API reference docs for full details or click on the API tab for code examples. ### Error items When the scraper cannot retrieve data for a given input — for example a product URL returns a 404 or a start URL is malformed — it pushes an **error item** to the dataset instead of silently skipping it. Normal output items are never affected; you can tell them apart by the presence of an `error` field. #### Error item structure ```json { "error": "product_not_found", "errorDescription": "Loaded a 404 page. The product does not exist.", "input": "https://www.amazon.com/dp/B0XXXXXXXX", "url": "https://www.amazon.com/dp/B0XXXXXXXX" } ``` #### Error codes reference | `error` | Meaning | | -------------------------------- | -------------------------------------------------------------- | | `invalid_url` | One or more start URLs were malformed or not recognized | | `invalid_input` | Actor failed due to bad configuration (run is also terminated) | | `product_not_found` | Product URL returns a 404 page | | `shortened_url_invalid` | Amazon shortened URL could not be resolved | | `bestsellers_category_not_found` | Bestsellers category page could not be found | | `no_relevant_reviews_found` | Reviews exist but none match the active filters | | `no_results_found` | No results were found for the given input | *** ### Not your cup of tea? Build your own scraper Amazon Product Scraper doesn’t exactly do what you need? You can always build your own! We have various [scraper templates](https://apify.com/templates) in Python, JavaScript, and TypeScript to get you started. Alternatively, you can write it from scratch using our [open-source library Crawlee](https://crawlee.dev/). You can keep the scraper to yourself or make it public by adding it to Apify Store (and [find users](https://blog.apify.com/make-regular-passive-income-developing-web-automation-actors-b0392278d085/) for it). Or let us know if you need a [custom scraping solution](https://apify.com/custom-solutions). ### Your feedback We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for xxx or simply found a bug, please create an issue on the Actor’s [Issues tab](https://console.apify.com/actors/BG3WDrGdteHgZgbPK/issues) in Apify Console. ### Connect to Claude / AI Agent (MCP) Connect this Actor directly to Claude Desktop, Cursor, or any MCP client via the Model Context Protocol endpoint: ```text https://mcp.apify.com/?token=YOUR_APIFY_TOKEN_HERE&tools=actors,docs,excitable_pickle/amazon-competitor-price-radar ``` #### Claude Copy-Paste Prompt ```text MCP Server URL: https://mcp.apify.com/?token=YOUR_APIFY_TOKEN_HERE&tools=actors,docs,excitable_pickle/amazon-competitor-price-radar Apify Token: YOUR_APIFY_TOKEN_HERE Execute Actor: excitable_pickle/amazon-competitor-price-radar Task: "Execute search with parameters, extract structured results, and present key intelligence in a markdown table." ``` # Actor input Schema ## `categoryOrProductUrls` (type: `array`): Use a link to the Amazon category you want to scrape, e.g. video games - https://www.amazon.com/s?bbn=16225016011\&rh=n%3A20972781011, https://www.amazon.com/dp/B0B3BVWJ6Y. You can also filter within the category with almost any filter available, i.e. price, manufacturer, rating, etc. By using a different URL you can also choose a different Amazon country, e.g. Germany: https://www.amazon.de/. You don't need to set a proxy country to do that. ## `maxItemsPerStartUrl` (type: `integer`): Enter the maximum number of results you want to scrape per each star URL. This will let the scraper know when to stop. ## `language` (type: `string`): Language to use on Amazon. **Disclaimer**: each Amazon domain has a different set of supported languages. If you choose a language that is not supported by the domain, the scraper will use the default language of the domain. ## `proxyCountry` (type: `string`): You can set proxy country to **avoid geo-blocking** and **access region-specific content**. Amazon shows you the products that can be shipped to your address based on the proxy you use. *By default*, the scraper **automatically selects** the **proxy country** based on the used Amazon website domain, ensuring low blocking rates and results relevant to the base domain. ## `maxSearchPagesPerStartUrl` (type: `integer`): Enter the maximum number of search pages to scrape for each start URL. E.g. only scrape the first 5 pages. This field can also be used with the `maxItemsPerStartUrl` option, the scraper will stop after either one of these limits is reached. ## `maxProductVariantsAsSeparateResults` (type: `integer`): Enter the maximum number of product variants you want to scrape per each product, outputted as separate results. **Please beware** that **you can already get** most of the **variant details** changes, in the **`variantDetails` field**, which is outputted by default. Note that this **will increase the number of requests** and **extend the scraping time**. ## `maxOffers` (type: `integer`): Enter the maximum number of offers you want to scrape per each product. This will let the scraper know when to stop. Note that this **will increase the number of requests** and **extend the scraping time**. ## `scrapeSellers` (type: `boolean`): Enable this option to extract further information about sellers of the product. Scrapes the featured offer seller (main seller). This option goes hand in hand with the `maxOffers` option bby scraping the seller of each offer. If you set `maxOffers` to 0, only the featured offer seller will be scraped. Note that this **will increase the number of requests** and **extend the scraping time**. ## `useCaptchaSolver` (type: `boolean`): If enabled the scraper will automatically solve captchas thrown by Amazon. This will decrease the amount of request retries and increase the speed of the scraper.

***IMPORTANT*** - This option works well only for the `'.com'` Amazon domain, but even for that one Amazon doesn't show a few product fields after solving a captcha (specifically: 'attributes', 'manufacturer attributes', and 'bestseller ranks') ## `scrapeProductVariantPrices` (type: `boolean`): Enable this option to extract prices of different variations of a product. Useful when you need prices for each variant. Note that this **will increase the number of requests** and **extend the scraping time**. ## `scrapeProductDetails` (type: `boolean`): If enabled, the scraper will extract each found product from the category page in detail (this is the default behavior). If disabled, the scraper will only extract the quick product information from the category page. Useful for faster and more lightweight searches for products. ## `countryCode` (type: `string`): The country code to use as the delivery location. ## `zipCode` (type: `string`): The zip code to use as the delivery location. This is useful for more precise location settings, especially in larger countries. ## `locationDeliverableRoutes` (type: `array`): The list of routes/page types, for which the deliverable location settings should be applied. - This is useful for speeding up the scraping process by not applying the location settings to all pages, for which you don't need the localized data for. - For example, you can set it to `['SEARCH', 'PRODUCT']` to apply the location settings only to search and product pages. - By default, all pages are included. ## Actor input object example ```json { "categoryOrProductUrls": [ { "url": "https://www.amazon.com/s?k=keyboard" } ], "maxItemsPerStartUrl": 100, "language": "en", "proxyCountry": "AUTO_SELECT_PROXY_COUNTRY", "maxSearchPagesPerStartUrl": 9999, "maxOffers": 0, "scrapeSellers": false, "useCaptchaSolver": false, "scrapeProductVariantPrices": false, "scrapeProductDetails": true, "countryCode": "US", "zipCode": "10001", "locationDeliverableRoutes": [ "PRODUCT", "SEARCH", "OFFERS" ] } ``` # Actor output Schema ## `products` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "categoryOrProductUrls": [ { "url": "https://www.amazon.com/s?k=keyboard" } ], "maxItemsPerStartUrl": 100, "proxyCountry": "AUTO_SELECT_PROXY_COUNTRY", "maxSearchPagesPerStartUrl": 9999, "maxProductVariantsAsSeparateResults": 0, "maxOffers": 0, "locationDeliverableRoutes": [ "PRODUCT", "SEARCH", "OFFERS" ] }; // Run the Actor and wait for it to finish const run = await client.actor("excitable_pickle/amazon-competitor-price-radar").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "categoryOrProductUrls": [{ "url": "https://www.amazon.com/s?k=keyboard" }], "maxItemsPerStartUrl": 100, "proxyCountry": "AUTO_SELECT_PROXY_COUNTRY", "maxSearchPagesPerStartUrl": 9999, "maxProductVariantsAsSeparateResults": 0, "maxOffers": 0, "locationDeliverableRoutes": [ "PRODUCT", "SEARCH", "OFFERS", ], } # Run the Actor and wait for it to finish run = client.actor("excitable_pickle/amazon-competitor-price-radar").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "categoryOrProductUrls": [ { "url": "https://www.amazon.com/s?k=keyboard" } ], "maxItemsPerStartUrl": 100, "proxyCountry": "AUTO_SELECT_PROXY_COUNTRY", "maxSearchPagesPerStartUrl": 9999, "maxProductVariantsAsSeparateResults": 0, "maxOffers": 0, "locationDeliverableRoutes": [ "PRODUCT", "SEARCH", "OFFERS" ] }' | apify call excitable_pickle/amazon-competitor-price-radar --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,excitable_pickle/amazon-competitor-price-radar" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/XX4SsuyYgcvC3zn0D/builds/iKidbi4Q2LgnSx8I3/openapi.json