# Changelog of Price Comparison: AliExpress, eBay, Walmart, Etsy (`myth84/marketplace-price-compare-scraper`) Actor

- **URL**: https://apify.com/myth84/marketplace-price-compare-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/myth84/marketplace-price-compare-scraper.md

### Changelog

#### Unreleased (repair: retry of a search stopped on an error)

- Brief `docs/cloud-tasks/2026-10-21_ecommerce-compare-fix-release-2.md` (2026-10-21). The control run of the release after PR #67 (`rHzIudtDukJkomoJS`, build 0.1.3, 512 MB) succeeded with 1,059 products but failed the release check `no crawler errors` (1 error). On Etsy, DataDome refused page 1 of "ceramic mug" (served after the session rotation), then page 2. The reused Etsy crawler rotates once per search, so it gave the search up at 59 of 100 listings.
- **A search that stops on an error before its share of products is retried once**, from its first page, on new sessions. The products it already returned are skipped: never written nor charged twice. The retry stops as soon as the query has its `maxResultsPerMarketplace` products. The errors of a search are those of its last attempt: an error the retry recovered is not an error of the run, as for a page recovered by the crawler's own session rotation. Log: a warning when a search is retried, one line with its outcome, and the count in the marketplace's closing line. `RUN_STATS.extra.searchesRetried` and `searchesRecovered` are new.
- Each search query now goes through its own crawl call of the marketplace's crawler, with the page and request caps of a one-keyword run (before, the keywords of a marketplace shared one request pool of the same total). The comparison now deduplicates the products across the queries of a marketplace; before, the crawler did, and it skipped the products it had seen *before* cutting a page to what the keyword still lacked. That order is kept (review of PR #69: cut first, a page lost its products after the cut, and a keyword sharing products with an earlier one could end at 19 of 20 with one more request). A crawl that may meet products already handed in (several keywords, or a retry) is uncut: the comparison skips the products an earlier crawl handed in (written, left out by the filters, or read beyond its keyword's share, as the crawler's `seen` set did), cuts the page to what the keyword lacks, before the filters, then ends the crawl at the keyword's share. A product found by two keywords still appears once, under the first.
- When no search stops on an error. One keyword: same requests, rows and log lines as before. Several keywords: same rows, ranks and requests as one crawl of every keyword, with one exception. A keyword whose pages bring no new product besides those of earlier keywords is no longer stopped after two such pages (Etsy with a price range: three pages without a new listing in the range), since each crawl knows only its own products. It reads on to its page cap (the pages its share needs, plus 1, or plus 3 with a price range or a condition), nothing charged. There is no difference while a keyword reads at most two pages (the default 20 products). Log of a several-keyword run: the reused crawler ends each keyword with `result limit reached after page N` (AliExpress, eBay and Walmart then skip that page's own line), and Etsy's warning about a price range that left out most listings quotes `of Infinity` instead of the keyword's share. Columns, SUMMARY, prices and inputs are unchanged. A search refused on every page now costs two more blocked requests, which are not charged. README: one line under "Limits and behaviour worth knowing" and the FAQ on missing products.
- Tests: the control scenario replayed on the committed Etsy fixtures (page 1 refused once, page 2 refused once). The old code gives 1 error and 18 of 26 listings; now 0 errors, with the products and ranks of an unblocked run and none twice. Fake marketplaces cover the retry rules: stop at the share, an error counted once when the retry fails again, no retry for a full search or a clean short one, none after the charge limit, deduplication across queries. The fake crawlers cut their pages as the real ones do. Real crawlers: two keywords sharing products, with and without a price range, give the rows, ranks and requests of one crawl of every keyword (the code before the repair). On AliExpress the second keyword meets 5 products the first wrote and 10 it read beyond its share; on eBay, Walmart and Etsy both keywords get the same pages. The code first reviewed in PR #69 fails both cases.

#### Unreleased (repair: eBay prices in the currency of the eBay site)

- **Symptom** (brief `docs/cloud-tasks/2026-10-21_ebay-currency-proxy-country.md`, 2026-10-21): the release control run `CpWgj7LRcws7BC7oZ` (country US, currency USD, 3 searches × 4 marketplaces, 1,100 rows, no error) had 100 eBay rows in KRW, all from "wireless earbuds" on www.ebay.com: the eBay residential proxy had no country and one session left from South Korea; the "currency USD" check failed and the release was abandoned.
- **eBay exits in the country of its site**: the eBay adapter adapts the proxy plan to the site searched (`localizeProxy`, the eBay Actor's `withMarketplaceProxyCountry`): residential exit in the site's country (US for www.ebay.com, GB for www.ebay.co.uk…), for the default proxy and for a residential group of the input without a country. A country, proxy URLs, another group or no proxy set by the customer are kept as they are; the other marketplaces are unchanged. `proxyPlanFor` takes the site as an optional third argument; the "searching" log line shows the exit country.
- **The eBay crawler's currency guard** (eBay Actor, same brief) runs in the comparison: a page in another currency is asked again on up to 3 new sessions, inside eBay's request cap; the adapter hands it the exit country of the plan.
- **Rows in another currency than their site's** stay in the dataset with the currency they show and stay out of the price figures and of the cheapest offer (existing behaviour); they are now named in the marketplace's `message` ("eBay showed 20 products in KRW instead of USD on www.ebay.com: …"), in the `message` of their search query and in the log.
- Tests: exit country of every eBay site, customer's group / country / URLs / other group / no proxy, Etsy and Walmart unchanged; the real eBay crawler on a derived KRW page (the US capture in KRW, see the eBay Actor's `derivedKrwPage`): one KRW page then dollars (USD rows, no remark), KRW pages that persist (KRW rows, remarks, cheapest offer from AliExpress).
- README: the currency FAQ says that eBay's proxy exits in the country of its site and how a product in another currency is reported.

#### Unreleased (quality score: input documented for AI agents)

- Brief `docs/cloud-tasks/2026-10-20_studio-quality-score-insights.md` (2026-10-21): `src/store.test.ts` now runs the shared checks of `@scraper-studio/core/testing` (`runInputQualityChecks` with this Actor's result limit, keywords × marketplaces × products per marketplace, and `runSearchTextChecks` on "Price Comparison"), the rules of Apify's quality score and of what an AI agent receives through the Apify MCP server.
- Descriptions rewritten with the reason of each default: `searchQueries` (duplicates searched once), `maxResultsPerMarketplace`, `country` (US: the only country whose four sites were verified), `condition`; `currency` says that rows in another currency are left out of the cheapest offer; `priceMin` and `priceMax` give an example and `priceMax` must be at least `priceMin`.
- Bounds added to the form: `maximum` 1,000,000,000 on `priceMin` and `priceMax` (the code accepts any non-negative amount; no realistic value reaches the bound).
- `maxResultsPerMarketplace` is prefilled with 10 in the form (the default stays 20): the prefilled input Apify tests daily (30 products) is exactly our daily check.
- `inputs/daily-check.json`: `currency` removed (was `USD`), as in the prefilled form; the run is the same, since an empty currency means the currency of the country, USD for the United States. The live daily-check task follows when the pilot re-sends it. `store.json › dailyCheck.what` says it is the prefilled form.
- Store texts carry "price comparison": `actor.json` description opens with "Product price comparison across" (was "Compare product prices across"), `store.json` `seoDescription` with "Price comparison of" (was "Compare prices on"); new `searchSynonyms` (price comparison scraper, compare prices across marketplaces, cross-marketplace price comparison, cheapest offer), all within the first 3,000 README characters, the second one added to the opening paragraph. README input table: prefill of `maxResultsPerMarketplace`, `priceMax` at least `priceMin`.
- No code change: rows, logs and charges are identical.

#### Unreleased (repair: memory of the run)

- **512 MB by default** (brief `docs/cloud-tasks/2026-10-21_ecommerce-compare-fix-release.md`, 2026-10-21). The control run of the first release (`6idsXSoQrCtkEN3yg`, build 0.1.1, 256 MB, 100 products per marketplace on the four marketplaces) was stopped after 801 products, with no error line, no status message, no SUMMARY and no RUN_STATS record. Two marketplaces crawl at once, each with a crawler that its own Actor validated alone at 256 MB, and above 64 products eBay reads its 240-listing page (2.7 MB of HTML, about 110 MB to parse). The default memory is now 256 MB per marketplace crawled at once (`.actor/actor.json`, `store.json`, the 8 example tasks). `minMemoryMbytes` stays at 256 for small runs such as the daily check (10 products per marketplace; the 160-product validation run succeeded at 256 MB).
- **V8 heap capped** at 3/4 of the run's memory (`ACTOR_MEMORY_MBYTES`; Dockerfile `CMD`), so that garbage is collected before the process goes over the run's memory.
- **Memory watch** (`src/memory.ts`): the resident memory is sampled every second, with a warning at 80 % and at 95 % of the run's memory. The peak goes into `RUN_STATS.extra.memoryPeakMbytes` and into the final `Done:` log line.
- **Crashes outside the main flow**: an uncaught exception or an unhandled promise rejection is now logged as an `ERROR` line and fails the run with a status message. Before, Node ended the process with a bare stack trace.
- Rows, SUMMARY, charges and the other log lines are unchanged. Tests: the watch's thresholds and peak, the memory settings against the number of marketplaces crawled at once (the test fails on the 256 MB of 0.1.1), the heap cap evaluated by `sh`, and the control input kept at the scale that failed.

#### 0.1 — 2026-10-21

First version (brief `docs/cloud-tasks/2026-10-20_ecommerce-compare-build.md`, built offline on the committed fixtures, to validate at the first runs).

- One run searches every keyword of `searchQueries` on the chosen `marketplaces` (AliExpress, eBay, Walmart by default; Etsy on request), two marketplaces at a time, each with the crawler, transport and proxy of its own Actor (imported and bundled, never called Actor to Actor): AliExpress and Walmart on the Apify datacenter proxy, eBay on residential, Etsy on residential with a US exit.
- One row per product in the shared `ecommerce-product/1.1` schema plus `marketplace` and `rankInMarketplace`; `SUMMARY` record with the count, lowest, median and highest price and the cheapest offer per keyword and marketplace, the cheapest offer across marketplaces per keyword, and the status of every marketplace (`ok`, `empty`, `blocked`, `failed`, `not-served`, `skipped`).
- `country` (10 countries) picks each marketplace's site; `currency` defaults to the country's; a marketplace without a site for the country is skipped and reported. eBay searches fixed-price listings only (the current bid of an auction is not a price to compare).
- `condition` is sent to eBay's search filter and checked on the condition labels of eBay and Walmart rows; AliExpress and Etsy (no condition shown) are kept for `new` and skipped for `used`. `priceMin` / `priceMax` are sent to each marketplace's own filter where it has one and checked again on every row.
- A marketplace that fails, is blocked or finds nothing does not stop the others; the run fails only when no marketplace returned any product.
- The SUMMARY figures leave out AliExpress welcome-deal prices (first-time buyers only, `attributes.newShopperDeal`): the rows stay in the dataset, `newShopperDeals` counts them. A marketplace whose turn comes after the charge limit of the run is not searched (`skipped`).
- Pay-per-event: one `result` event per product row, whatever the marketplace.