# Changelog of Amazon Products Scraper & Price Tracker (`myth84/amazon-search-scraper`) Actor - **URL**: https://apify.com/myth84/amazon-search-scraper/changelog.md - **Full Actor documentation**: https://apify.com/myth84/amazon-search-scraper.md ### Changelog History of `amazon-search-scraper` (folder `actors/amazon-search`). Versions are the Apify build numbers. #### Unreleased (quality score: input documented for AI agents) - Brief `docs/cloud-tasks/2026-10-20_studio-quality-score-insights.md` (2026-10-21): the input schema passes the shared checks of `@scraper-studio/core/testing` (`runInputQualityChecks`, `runSearchTextChecks`), the rules of Apify's quality score and of what an AI agent receives through the Apify MCP server. - Descriptions rewritten (default stated with its reason, example of the format for free text): `searchQueries` (the default only serves runs started without any keyword or URL), `startUrls` (example URL, default empty), `marketplace`, `maxResults` (says the form prefill), `sortBy`, `priceMin`, `priceMax` (examples, `priceMax` at least `priceMin`), `department` (allowed characters), `includeSponsored`, `localPrices`, `maxPages`, `monitorPrices`, `onlyChanges`, `monitorStoreName` (example, used only when monitoring is on), `proxyConfiguration`, `maxRequests`, `maxAttemptsPerPage`, `delayMs`, `debugStoreHtml`. - Bounds added to the form: `maximum` 1,000,000,000 on `priceMin` and `priceMax` (the code accepts any non-negative number; no realistic price reaches the bound, yen included). - `maxResults` is prefilled with 20 in the form (the default stays 100): the prefilled input Apify tests daily is exactly `inputs/daily-check.json` (20 products, wireless earbuds, amazon.com), which is unchanged, so the live daily-check task needs no update. - `store.json` gains `searchSynonyms` (Amazon scraper, Amazon price tracker, Amazon price monitor, Amazon search results, ASIN), all within the first 3,000 README characters: the README opening now reads "Use it as an Amazon scraper and Amazon price monitor…" (line 1 and title unchanged). README input table: prefill of `maxResults`, example values for `priceMin` and `priceMax`. - No code change: rows, logs and charges are identical. #### Unreleased - **Price monitoring** (brief `docs/cloud-tasks/2026-10-20_amazon-search-extend-monitor.md`, 2026-10-20, modelled on the eBay Actor, same option and column names): `monitorPrices` (off by default), `onlyChanges` (off by default) and `monitorStoreName` (default `amazon-price-monitor`, letters, digits and dashes, lower-cased); previous prices kept in the customer's named key-value store, one record (`amazon-monitor/1`) per search identity: keywords, or the customer URL without its tracking parameters (page 1, parameters sorted), marketplace, sort, price range, department and the country the prices are seen from (`viewCountryCode`: the marketplace's country with `localPrices`, else the proxy's), so that a search read with and without local prices has two references; a product is identified by its ASIN inside the record. `changeType` / `previousPrice` / `priceDelta` / `priceDeltaPercent` filled (price change only in the same currency); with `onlyChanges` only the new products and the price changes are pushed, hence charged; a monitoring run that compared products and found no change succeeds with an empty dataset (`isEmptyRunFailure`). `RUN_STATS.monitor` (searches, baseline, compared, new, price changes, unchanged, written, records written and skipped), `RUN_STATS.monitorBaseline`, counters `monitorCompared`, `monitorNew`, `monitorPriceChanged`, `monitorUnchanged`, `monitorDuplicatesNotWritten` in `RUN_STATS.extra`. - **Deviations from the eBay model**, because Amazon runs two searches at once and shares ASINs across marketplaces: (1) the pass finds the search of a row through the row itself (set in `normalize`), not through "the task being crawled"; (2) whether a search ended cleanly is decided from its own pages (last page requested = last page parsed, no row refused by the charger, stopped by its last page, its cap or `maxPages` rather than by the run's budget), not from the run's shared error counter, so that a failing search does not block the record of the other; (3) in monitoring mode deduplication is per search (`|`): two searches sharing a product both compare it and both records hold it whatever the order of the parallel searches; the dataset still holds it once per marketplace (ASINs reserved before the charger is awaited; `monitorDuplicatesNotWritten`); (4) a product shown without a price keeps its last known price in the record, so that a price that disappears for a day and comes back lower is reported as a drop; (5) an unreadable store fails the run before any request, an unwritable one after the crawl, with an explicit `AmazonMonitorStoreError` message. - **Monitoring off: output unchanged.** The charger receives the rows directly (no extra async layer, same interleaving of the two parallel searches); checked once against the 0.1 crawler on every fixture with 503 retries injected (1 and 2 searches at once, sponsored skipped, per-search cap, without local prices: identical rows, pushes, log lines and statistics) and kept as tests (rows equal to the normalizer's on a fixture page, identical output with `monitor: null`). `RUN_STATS` gains the zero counters and `monitor: null`. - **Store texts on the "amazon price tracker" niche** (`docs/research/2026-10-20_synthese-recherche-visibilite.md`): title and SEO title **Amazon Products Scraper & Price Tracker**, README H2 with the same words; description and SEO description say "Amazon price tracker" and price drops; README opening with "Amazon price tracker", "Amazon price monitor" and "price drop alerts". README first screen of PR #50 (no-login line, What you get / For whom / How much with the Free-plan price of `pricing/2026-10-20_pay-per-event.autopilot.json`: $2.16 per 1,000 products, Apify's $5 monthly credit covers about 2,314; 10-column output snippet), *Price monitoring* section, input table, *Use it from your AI assistant*, *Alternatives and switching*, *More scrapers* limited to the studio's public Actors (eBay, AliExpress, Etsy: `autopilot/state.json`, test), non-affiliation line at the end; the FAQ keeps 10 entries (freshness merged into the monitoring entry, export into support). `readmeVisibility` Store check on. - Input form section *Price monitoring*; dataset view **Changes**; monitoring columns described in `dataset-fields.json`; example tasks `amazon-wireless-earbuds-price-drops-us` and `amazon-robot-vacuum-price-monitor-us` (10 in total, the platform limit); tests on the monitor module, the crawler pass (reference run, changes only, price drop and new product, every product, cap, missing price, separate searches and price countries, shared products with two searches at once, fail-closed cases, per-search decision with two searches at once, unknown record, unreadable and unwritable store), the input and the Store files. #### 0.1 — 2026-10-20 - First version, written offline on the committed fixtures of the audit (`docs/audits/amazon.md`, brief `docs/cloud-tasks/2026-10-20_amazon-build.md`, 12 `/s/query` streams captured by the probe on amazon.com, .co.uk, .de, .fr and .co.jp plus the 503 page). **Source: Amazon's internal pagination call `POST /s/query`**, never the HTML page `/s` (Akamai interstitial, AWS WAF and "Continue shopping" checkpoint in the audit). The response is split on `&&&`, every fragment parsed with `JSON.parse` (a truncated last fragment is ignored, the others are counted in `pageInfo.invalidFragments`); read: `data-search-metadata` (total, page, keywords, department alias), every `data-main-slot:search-result-N` in order, the pagination strip (last page = highest page number of the strip: 20 for a list, 7 for the grids of 48–60, 10 on .co.jp; "next" disabled), `data-title-and-meta`. Every card is loaded on its own in Cheerio (1.0.0-rc.12, external dependency), never the 0.5–2.3 MB stream as one document. A response without `dispatch` fragment, or with neither metadata nor product, raises `AmazonParseError`; a stream with metadata and no product is a valid empty page (end of results). - **Transport, without touching the core**: the generic crawler (`crawlPagedSearches`) builds the public URL `https:///s?…&page=N`; the Actor's `fetch` wrapper sends instead `POST /s/query?k=…&page=N&qid=&ref=sr_pg_N` + the other parameters, body `{"customer-action":"pagination"}` (page 1 included, `page=1` explicit), `Accept: text/html,image/webp,*/*`, `Content-Type: application/json`, `X-Requested-With: XMLHttpRequest`, `Origin`, `Referer` (the public URL, without `qid`), the marketplace's `Accept-Language`, with got-scraping (default transport of `httpRequest`) and `maxRetries: 0`. Every new session's cookie jar holds the display-currency preference `i18n-prefs=` of every marketplace domain (preference cookies only). - **Attempts**: an answer is usable when it is 2xx and its stream holds `data-search-metadata`; otherwise (503, HTML page, empty body, network error) the page is retried at once on a fresh session (new proxy session, new cookie jar, new got-scraping session token) **with a new `qid`** (Amazon caches part of its 503 per exact URL), after `retryDelayMs` (1 s), up to `maxAttemptsPerPage` (input, default 5, 1–10); the fresh session is kept for the next pages. After the last attempt the answer goes back to the generic crawler, whose single rotation per search makes **one last attempt** (the wrapper gives it one attempt only, then the search is given up): at most `maxAttemptsPerPage + 1` requests for one page. Every attempt counts in `requests`, bytes, `blocked` and `maxRequests`. The 503 page carries "automated access to" (a strong marker of the core): its verdict is re-labelled `http-5xx` so that the logs say what happened. Extra markers of `detectBlock`: `Sorry! Something went wrong!`, `Tut uns Leid!`, `/errors/validateCaptcha`, `bm-verify`, `gokuProps`, `/checkpoint/` (none is in a served stream: test on every fixture). A proxy refusal (407, tunnel error) is retried once on a fresh session, then fails the run with an explicit message (`AmazonProxyConfigurationError`, distinct for the residential proxy of local prices). - **Local prices**: for a marketplace other than amazon.com, with the default proxy configuration and `localPrices` on (default), every session of that marketplace uses the Apify residential proxy of its country (`groups: ['RESIDENTIAL']`, `countryCode` GB for UK, else the code; one proxy configuration per country, created on first use) and `viewCountryCode` is that country; amazon.com stays on the datacenter tier (`viewCountryCode` US). With `localPrices: false` or a customer proxy, rows of a non-US marketplace say the proxy country (US by default) and the run logs that prices may be converted or missing. The residential traffic (`RUN_STATS.extra.residentialBytes`) is not part of the runner's cost estimate, which knows one tier per run: it is logged separately (≈ 0.04–0.10 $ per 1,000 products measured in the audit, < 6 % of the BRONZE reference). - Output schema `ecommerce-product/1.1`: `price` = first non-struck `.a-price .a-offscreen` of the price block; currency read in the string (ISO code first: "USD 12.41", "25,47 USD"; then `£`, `€`, `¥`/`￥`, `$` = the marketplace's dollar), null without price (never assumed); local separators ("$1,100.00", "1.234,56 €", "￥1,280": a single separator followed by three digits groups thousands); struck price kept only in the same currency and above the price, typed `list` / `typical` / `was` (`strike` for an unknown label) from its label; `discountPercent` to one decimal; `rating` from the visible figure, else the stars' accessible text (last number in "5つ星のうち3.9"); `reviewCount` exact from the accessible text of the count link ("46,784 ratings", "3 460 évaluations"), else the rounded visible form ("(46.7K)", "(2.8万)"); `soldCount` null, "bought in past month" in `attributes.boughtPastMonth` (lower bound) and `boughtPastMonthLabel`; `isChoice` = `badgeType === 'amazons-choice'` when the card has a status or best-seller badge, null without badge; `isBestSeller` from the `BEST_SELLER` badge (or "ベストセラー1位"); `isSponsored` from the `AdHolder` class or the sponsored label; title without the sponsored prefix ("Sponsored Ad -", "Gesponserte Anzeige –", "Publicité sponsorisée -"…); `brand` only from a separate brand heading; `deliveryEstimate` = text of the delivery block; `shippingFree` true when it starts with free delivery, false when a delivery cost is read (same currency as the price), null otherwise; `inStock` true with a price, false for "Currently unavailable" and its translations; `stockQuantity` from "Only N left in stock"; `imageUrl` without the size segment; `url` = `https:///dp/`; `searchRank` across pages, sponsored products counted only when kept; `searchQuery` = Amazon's `metadata.keywords`. - **Seller and personal data**: the seller id of the featured offer (`input[name="merchantId"]`, `data-csa-c-merchant-id`) is reduced to a boolean in the parser by comparison with the retail ids of Amazon's 8 verified marketplaces (an Amazon EU store selling on another EU marketplace is Amazon too); `storeType: 'platform'` + `storeName: 'Amazon'` when it is one of them, `unknown` otherwise (third-party sellers may be persons; amazon.ca's own id is to be read at the first run); the id itself is never exported (`storeId` null). Session ids, CSRF tokens and add-to-cart properties are never read; every row goes through `stripPersonalData` (tests: third-party id, injected `sid` / `sessionId` / e-mail). - **Decisions and deviations from the brief**: (1) `store.json › seoDescription` shortened to 149 characters (the brief's text had 165, above the 155 accepted by `scripts/release-actor.mjs`); (2) following the Store-text rule of 2026-10-20 (`docs/cloud-tasks/2026-10-20_studio-promote-store-texts.md`), the README section *Marketplaces and local prices* explains local prices without naming any proxy tier — the tiers are in the input form (section *Proxy*) and in this changelog; (3) extra flat attributes beyond the brief's list, read from the fixtures: `deal` ("Limited time deal" badges), `reviewHighlight` ("Top Reviewed for Comfort") and `bestSellerRank` (amazon.co.jp category rank, "ベストセラー4位" is not a best-seller badge: `isBestSeller` stays false); (4) `shippingFree` is false when a delivery cost is shown (the brief said null), as in the Walmart Actor; (5) the French sponsored prefix is "Publicité sponsorisée -" in the captures (added to the brief's list); (6) colour swatches are counted from `.s-color-swatch-outer-circle` (the `data-cy="color-swatch"` of the brief is not in the cards); (7) the last page is the highest number of the pagination strip (the disabled item is only the far end of a long strip); (8) concurrency 2 (one session per search), `retryDelayMs` 1 s, not exposed in the input. - Known limits (audit §9–10): ≈ 72–77 % of the attempts served on the datacenter tier (to measure on ≥ 100 pages at the validation run, with the share of 503 on page 1); amazon.ca not served during the audit; `newest` and `popularity` sorts not exercised; no postal code / delivery location; no product page, review, seller or best-seller list; memory of 256 MB to confirm on streams of 2 MB; Amazon's price filter lets a few products outside the range through (a $16.10 product listed at $25.99 in the 20–50 capture). - Store readiness: README, input schema, dataset schema with Overview / Prices / Ranking / Badges views, output schema, 8 example tasks (default proxy, at most 60 products, the unverified sort said in its description), consistency tests (README ⇔ schemas ⇔ code, Ajv replay of the 206 fixture rows), crawler tests on a fake HTTP client (POST, headers, cookie jar, pagination to the last page, deduplication, 503 then success on a new session with another `qid`, give-up after `maxAttemptsPerPage`, attempts in `maxRequests`, proxy errors, local prices for DE and nothing for US, `includeSponsored: false`), seller and injection tests.