# Amazon Commerce Intelligence (`optirefine/amazon-commerce-intelligence`) Actor

Observe public Amazon search/product/review pages with normalized records, price snapshots, source provenance and clear access/coverage warnings.

- **URL**: https://apify.com/optirefine/amazon-commerce-intelligence.md
- **Developed by:** [OptiRefine](https://apify.com/optirefine) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 product details extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Amazon Commerce Intelligence — Products, Reviews & Price Monitoring

**Publisher:** OptiRefine\
**Actor slug:** `amazon-commerce-intelligence`\
**Release:** `0.1.0` (tested starter implementation, not a verified live-Amazon release)\
**Runtime:** Node.js 22, no npm dependencies

Observe publicly accessible Amazon product information, search cards and reviews across multiple marketplaces. Produce normalized datasets with **ASIN identity, provenance, visibility/completeness metadata, buyer-facing prices, featured sellers and incremental price snapshots**. Access-control failures generate explicit machine-readable error records instead of silently returning empty data.

> **Testing status:** Node.js parser and workflow tests pass using curated HTML fixtures and local mock APIs. This project has **not** been tested against live Amazon pages, Apify's actual cloud deployment, real Apify Proxy traffic, or live pay-per-event billing. Amazon markup and access controls change frequently. Review the [release blockers](#before-publishing-to-apify-store) before charging customers.

### Capabilities in v0.1.0

| Mode | What this build does | Caveats |
|---|---|---|
| `product` | Extracts title, brand, ASIN, currency/price, list price, stock text, rating, review count, features, breadcrumbs, visible variants, main image, Buy Box seller. | Fields missing in public HTML stay null/empty; no browser rendering. |
| `search` | Keywords and Amazon search URLs, result card ASIN/title/price/rating, limited pagination; optional detail enrichment. | Public search HTML only; results, ranks and sponsorship labels depend on markup. |
| `category` | Takes Amazon category URLs or numeric browse-node IDs and parses public result cards. | No recursive category tree traversal. |
| `reviews` | Public review-page records, visible local filters, reviewer-PII minimization, visibility summary. | May yield zero reviews if sign-in is required; **not all reviews**. |
| `monitor` | Compares product observations against `baselineDatasetId`; emits changed product rows and `price_snapshot` rows. | Does **not** recreate historical prices before the first observation. |
| `bestsellers` | Can accept ranking/category URLs and attempt normal public result-card parsing. | Dedicated chart-specific ranking parser is **not** implemented. |

All `recordType` values use `schemaVersion: "1.0"`: `product`, `search_result`, `review`, `review_coverage`, `offer`, `price_snapshot`, `error`.

**Known feature gaps relative to the source research:** no Crawlee Request Queue or persistent in-run resume; no Playwright fallback; no multi-offer expansion; no actual delivery-postcode setting; no full cross-locale review-date support; no official SP-API/Creators API provider; no seller-page crawl; no advanced adaptive concurrency or 1,000-target live performance benchmark. Variants/featured offers rely on visible product HTML. Plan a second release for these before describing the Actor as feature-complete.

### Install or deploy on Apify

1. Extract the ZIP into a local project directory.
2. Install Apify CLI separately if needed: `npm install -g apify-cli`.
3. Sign in using `apify login`, then run `apify push` **from the extracted project folder**. This uploads the Actor source and lets Apify build from the included Dockerfile. Alternatively connect a GitHub repo containing the extracted project as an Actor source.
4. Open the Actor in Apify Console, verify the **Input** and **Output** tabs, and run a tiny test using one ASIN and `maxRequests: 5` before any broader workloads.
5. Examine the default Dataset rows and key-value store record `OUTPUT`. Verify with human eyes that actual price, rating, ASIN, locale and source coverage are correct, and note any `ROBOTS_DISALLOWED`, `ACCESS_CHALLENGE` or `LOGIN_REQUIRED` rows.
6. **Do not publish a monetized Store listing until successful live runs and billing tests have been confirmed.**

The Apify platform supplies the Actor token and default storage IDs automatically. **Do not paste an Apify token into the Actor input or commit one to source.**

#### Example: product details (Canada)

```json
{
  "mode": "product",
  "marketplace": "CA",
  "asins": ["B08N5WRWNW"],
  "includeReviews": false,
  "includeOffers": true,
  "maxRequests": 5,
  "maxRequestsPerMinute": 20,
  "robotsPolicy": "respect",
  "privacyMode": "minimal"
}
```

#### Example: keyword search

```json
{
  "mode": "search",
  "marketplace": "US",
  "keywords": ["usb-c charger"],
  "maxPages": 2,
  "maxProducts": 30,
  "includeProductDetails": false,
  "maxRequests": 50,
  "maxRequestsPerMinute": 20
}
```

#### Example: public reviews

```json
{
  "mode": "reviews",
  "marketplace": "US",
  "asins": ["B08N5WRWNW"],
  "reviewRating": "all",
  "maxReviewsPerAsin": 20,
  "maxPages": 2,
  "privacyMode": "minimal"
}
```

The review limit is a **maximum**, not a guarantee: Amazon can limit what is publicly viewable. If data is not exposed publicly, expect a `review_coverage` record with a limitation or an access error, rather than fabricated reviews.

#### Example: price changes / history

1. Run product mode to create a baseline dataset; retain its ID.
2. Run `monitor` with the same ASINs and `baselineDatasetId` set to that dataset ID:

```json
{
  "mode": "monitor",
  "marketplace": "US",
  "asins": ["B08N5WRWNW"],
  "baselineDatasetId": "REPLACE_WITH_PREVIOUS_DATASET_ID",
  "maxRequests": 10
}
```

3. For ongoing monitoring, use Apify Schedules and update the baseline dataset ID or implement a durable baseline manager. Each changed observation is recorded as a `price_snapshot`, not an inferred historical price. When nothing changed, the summary reports `unchanged` even though the dataset may be empty.

**Important:** A run without `baselineDatasetId` treats observed products as new. This MVP does not update its own baseline ID across scheduled runs.

### Source access, privacy and behavior

- Only exact, allowlisted `https://` Amazon marketplace domains are requested. Redirects to outside hosts are rejected. The included marketplace map covers US, CA, UK, DE, FR, IT, ES, JP, IN, AU, MX, BR, NL, PL, SE, SG, AE, SA and TR.
- Requests identify themselves as `Agent/OptiRefineAmazonCommerceIntelligence` in the `User-Agent` header.
- `robotsPolicy: "respect"` is the default. Robots rules are fetched for the relevant marketplace and the Actor fails closed when they cannot be read. Some product and review pages may therefore be inaccessible in default mode.
- `robotsPolicy: "report_only"` requires `authorizedAccessConfirmed: true` and is intended only for separately authorized access. Changing the setting is not evidence of legal permission and does not bypass access challenges.
- CAPTCHA, 403, 429, 503 and login-only conditions are **classified, not bypassed**. No CAPTCHA solver, stealth/human simulation, credential scraping or login flow is included.
- Reviewer display names and profile links are excluded by default (`privacyMode: "minimal"`). `full` allows these fields when publicly available. You are responsible for a lawful basis, retention and downstream privacy controls.
- Review rating, date, verified status and keyword filters apply to records actually observed, not to an imagined complete Amazon review archive. Some locales may not expose enough information to reliably apply every filter.
- `deliveryLocation` is only recorded as an annotation in price snapshots. It **does not** manipulate the retail website's shipping location.
- Optional `proxyConfiguration` accepts Apify Proxy for geo-routing. This uses an HTTP CONNECT tunnel; real residential proxy operation has **not** been verified here. Proxy charges accrue on the Apify user's plan and are separate from Actor event fees.

#### Optional Apify Proxy object

```json
{
  "useApifyProxy": true,
  "apifyProxyGroups": ["RESIDENTIAL"],
  "apifyProxyCountry": "CA"
}
```

Use proxy settings only where allowed by the relevant service terms and actual authorization; this Actor does not rotate proxies around blocks or CAPTCHA challenges.

### Dataset outputs

Each dataset row has `recordType` and `schemaVersion` to support downstream ingestion. Source metadata includes the public URL, observed timestamp, page type, HTTP status, locale, and access method. Products include a `completeness` object. Review runs emit `review_coverage` for every successfully parsed first review page. Errors include a `target`, stable `code`, category, retryability, HTTP status and captured timestamp. A run summary lives in the default key-value store under `OUTPUT`.

Example error codes: `INVALID_INPUT`, `UNSUPPORTED_URL`, `UNSUPPORTED_MARKETPLACE`, `PRODUCT_NOT_FOUND`, `PAGE_LAYOUT_UNRECOGNIZED`, `ACCESS_CHALLENGE`, `LOGIN_REQUIRED`, `RATE_LIMITED`, `ROBOTS_DISALLOWED`, `ROBOTS_UNAVAILABLE`, `NETWORK_TIMEOUT`, `PROXY_ERROR`, `PARSE_PARTIAL`, `HTTP_ERROR`, `BILLING_ERROR`.

The Console shows one **Results and diagnostics** dataset view for all row types. Filter by `recordType` in exports. This version does not expose separate physical datasets for reviews, sellers and errors.

### Monetization setup (optional, not enabled out of the box)

This Actor includes an Apify pay-per-event charging hook, but **charging is off until the owner configures it**. To avoid unbudgeted billing and surprises, take these steps before making any listing paid:

1. In Apify Console, choose **Pay Per Event** and configure these exact event names. Example launch prices (USD, suggestions only):

   | Event | Example USD / event | Typical record |
   |---|---:|---|
   | `search_result` | $0.0015 | One search result card |
   | `product_detail` | $0.0060 | One product record |
   | `review` | $0.0030 | One review record |
   | `offer` | $0.0010 | One visible featured offer |
   | `history_snapshot` | $0.0010 | One changed observation |

2. In the **Actor owner's environment variables** (not customer input), set `PPE_ENABLED=1` and the JSON string below, keeping it synchronized with the actual Console event prices:

   ```text
   PPE_EVENT_PRICES_JSON={"search_result":0.0015,"product_detail":0.006,"review":0.003,"offer":0.001,"history_snapshot":0.001}
   ```

3. The Actor uses Apify's `ACTOR_MAX_TOTAL_CHARGE_USD` run limit to prevent explicitly configured charges from going over the user's budget. Unknown event prices fail billing rather than charging at an unverified price. Verify a tiny paid test run and your revenue/usage dashboard before publishing.

4. Prices are examples, **not profit guarantees**. Benchmark Amazon access availability and proxy/network usage before setting final rates. The Actor may produce no paid records on blocked or robots-disallowed paths.

**Billing caveat:** Direct Apify REST charging and storage writes are not an atomic transaction. If a storage or billing call fails in the middle of a run, inspect the run and its event log before retrying. A fully production-hardened Actor should use Apify's SDK charging manager and persistent per-record checkpoints.

### Local tests (offline, no Amazon traffic)

Node.js 22+ is required. There are no npm dependencies to install.

```bash
npm test
```

The Node.js built-in test runner exercises the HTML parsers, currency formats, privacy, robots rules, marketplace validation, redirects, pagination, CAPTCHA classification, per-target errors, price changes, local dataset storage, a simulated Apify REST API, and the **actual `src/main.js` entrypoint** with mocked public pages. Mocked HTML is illustrative and is *not* proof of current live Amazon compatibility.

To run locally against live pages (subject to robots rules and terms), edit `storage/key_value_stores/default/INPUT.json` and execute:

```bash
node src/main.js
```

Inspect `storage/datasets/default/*.json` and `storage/key_value_stores/default/OUTPUT.json`. Delete/recreate local default dataset storage between distinct runs to avoid confusing old records with current ones.

### Before publishing to Apify Store

This is a **tested MVP**, not yet a feature-complete Store launch:

- \[x] Node.js syntax and offline automated tests
- \[x] Real local dataset/KV storage, mocked platform REST API, true entrypoint smoke test
- \[x] Product/search/review/monitor parsing and error classification on fixtures
- \[x] CAPTCHA-detection and robots respect on fixtures
- \[ ] Successful *actual* Apify build from this Dockerfile
- \[ ] Successful, authorized live Amazon US and CA product/search/review tests with manual accuracy verification
- \[ ] Live Apify Proxy testing where authorized
- \[ ] Live PPE event/budget/retry verification in a sandbox-priced Actor
- \[ ] Live locale HTML fixtures for all advertised marketplaces
- \[ ] Request-queue restart/resume and large-scale cost benchmark
- \[ ] Full implementation of all advanced features in the research specification

Amazon's automated-agent rules require agent identification and prohibit circumventing CAPTCHA and similar access measures. This project follows those guardrails but that alone does not establish permission to extract or redistribute the content. The customer and Actor owner must review Amazon's terms and applicable laws before deployment.

### Files

- `src/main.js` — Apify/local entrypoint
- `src/actor.js` — mode router, workflows, normalization, incremental diffs and errors
- `src/parsers.js` — product, search and public-review parsers
- `src/robots.js` — robots policy and challenge-safe behavior
- `src/network.js` — HTTP client, allowlisted redirects and optional proxy tunnel
- `src/storage.js` — local files and Apify REST storage/event adapters
- `src/html.js` — dependency-free basic HTML tree parser
- `src/marketplaces.js` — marketplace definitions and URL validation
- `.actor/` — input/output/dataset schemas and deployment metadata
- `tests/` — fixture-based unit/integration tests
- `examples/` — starter inputs

### References

- Apify Actor setup: https://docs.apify.com/actors/development/quick-start/locally
- Apify input schema: https://docs.apify.com/actors/development/actor-definition/input-schema/specification/v1
- Apify output schema: https://docs.apify.com/actors/development/actor-definition/output-schema
- Apify charge API: https://docs.apify.com/api/v2/post-charge-run
- Amazon Agent Policy: https://sellercentral.amazon.com/help/hub/reference/external/GS83KH2MA7HM69PH

**No affiliation or endorsement by Amazon or Apify is implied.**

# Changelog

This Actor's version history is a separate document: https://apify.com/optirefine/amazon-commerce-intelligence/changelog.md

# Actor input Schema

## `mode` (type: `string`):

Choose the workflow. Monitor compares newly observed values with a previous dataset when provided.

## `marketplace` (type: `string`):

Amazon site to query for ASINs and keywords; URLs select their own marketplace.

## `asins` (type: `array`):

Product ASINs (10 alphanumeric characters).

## `keywords` (type: `array`):

Keywords for Amazon search or product discovery.

## `startUrls` (type: `array`):

Amazon product, category, search or review URLs.

## `categoryUrls` (type: `array`):

Public Amazon category or bestseller URLs (URL strings or objects).

## `browseNodeIds` (type: `array`):

Numeric category browse-node IDs.

## `maxPages` (type: `integer`):

Only follow publicly accessible navigation.

## `maxProducts` (type: `integer`):

Global cap of emitted search cards.

## `maxReviewsPerAsin` (type: `integer`):

Upper bound, not a guarantee; only public reviews can be collected.

## `maxRequests` (type: `integer`):

Hard cap for page requests, excluding robots.txt checks.

## `maxRequestsPerMinute` (type: `integer`):

Conservative rate cap across all marketplace requests.

## `maxRequestRetries` (type: `integer`):

Retries transient network failures only; never retries CAPTCHA and access challenges.

## `requestTimeoutMs` (type: `integer`):

Page request timeout (1000–60000 recommended).

## `includeProductDetails` (type: `boolean`):

Also visit each public product page found in search results.

## `includeReviews` (type: `boolean`):

Fetch public review pages for products found in product mode.

## `includeVariants` (type: `boolean`):

Parse visible variant ASINs from product page.

## `includeOffers` (type: `boolean`):

Emit the visible Buy Box offer; not a complete offer marketplace.

## `includeImages` (type: `boolean`):

Emit image URLs when visible.

## `reviewRating` (type: `string`):

Applied locally to visible reviews; does not expand review access.

## `reviewVerifiedOnly` (type: `boolean`):

Filter to reviews marked verified purchase in public HTML.

## `reviewKeyword` (type: `string`):

Case-insensitive keyword match against visible review title/body.

## `reviewDateFrom` (type: `string`):

ISO YYYY-MM-DD. Applies locally to dates that could be parsed.

## `reviewDateTo` (type: `string`):

ISO YYYY-MM-DD. Applies locally to dates that could be parsed.

## `reviewSort` (type: `string`):

Public review-page sort selection.

## `privacyMode` (type: `string`):

Minimal excludes reviewer names and profile links.

## `robotsPolicy` (type: `string`):

Respect blocks disallowed paths (default). Report only requires confirmed authorization.

## `complianceMode` (type: `boolean`):

Identify as an automated agent; do not solve CAPTCHAs or bypass challenges.

## `authorizedAccessConfirmed` (type: `boolean`):

Only relevant when disabling compliance defaults or choosing report_only.

## `incremental` (type: `boolean`):

Uses baselineDatasetId to compare previous observations.

## `baselineDatasetId` (type: `string`):

Previous run dataset ID for comparing product observations. Requires API read access.

## `deliveryLocation` (type: `object`):

Optional {"country":"CA","postalCode":"M5V3L9"}; included in snapshots for reference only; does NOT set Amazon delivery destination.

## `proxyConfiguration` (type: `object`):

Optional Apify proxy settings for authorized geo-localization. User is charged separately for proxy traffic. No challenge evasion. Example: {"useApifyProxy":true,"apifyProxyGroups":\["RESIDENTIAL"],"apifyProxyCountry":"CA"}.

## `debug` (type: `boolean`):

On failed HTTP access, write diagnostic status, page title, and response hash to DEBUG_FAILURE\_\* key-value records. No raw HTML or secrets stored. Does not bypass site restrictions.

## Actor input object example

```json
{
  "mode": "product",
  "marketplace": "US",
  "maxPages": 2,
  "maxProducts": 100,
  "maxReviewsPerAsin": 20,
  "maxRequests": 400,
  "maxRequestsPerMinute": 30,
  "maxRequestRetries": 1,
  "requestTimeoutMs": 15000,
  "includeProductDetails": false,
  "includeReviews": false,
  "includeVariants": true,
  "includeOffers": false,
  "includeImages": true,
  "reviewRating": "all",
  "reviewVerifiedOnly": false,
  "reviewSort": "recent",
  "privacyMode": "minimal",
  "robotsPolicy": "respect",
  "complianceMode": true,
  "authorizedAccessConfirmed": false,
  "incremental": false,
  "debug": false
}
```

# Actor output Schema

## `results` (type: `string`):

Structured product, search_result, review, offer, snapshot and error records.

## `runSummary` (type: `string`):

Status, record counts and warnings.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("optirefine/amazon-commerce-intelligence").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("optirefine/amazon-commerce-intelligence").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call optirefine/amazon-commerce-intelligence --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,optirefine/amazon-commerce-intelligence"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/sqaj5NjajoEp9eqWO/builds/uF41JSw3KgRWugCym/openapi.json