Product price comparison across AliExpress, eBay, Walmart and Etsy in one run: one keyword, one table with the same columns everywhere (price, currency, shipping, rating, sales, marketplace, rank) and a summary of the lowest, median and highest price per marketplace. No personal data.
Unreleased (repair: retry of a search stopped on an error)
Brief docs/cloud-tasks/2026-10-21_ecommerce-compare-fix-release-2.md (2026-10-21). The control run of the release after PR #67 (rHzIudtDukJkomoJS, build 0.1.3, 512 MB) succeeded with 1,059 products but failed the release check no crawler errors (1 error). On Etsy, DataDome refused page 1 of "ceramic mug" (served after the session rotation), then page 2. The reused Etsy crawler rotates once per search, so it gave the search up at 59 of 100 listings.
A search that stops on an error before its share of products is retried once, from its first page, on new sessions. The products it already returned are skipped: never written nor charged twice. The retry stops as soon as the query has its maxResultsPerMarketplace products. The errors of a search are those of its last attempt: an error the retry recovered is not an error of the run, as for a page recovered by the crawler's own session rotation. Log: a warning when a search is retried, one line with its outcome, and the count in the marketplace's closing line. RUN_STATS.extra.searchesRetried and searchesRecovered are new.
Each search query now goes through its own crawl call of the marketplace's crawler, with the page and request caps of a one-keyword run (before, the keywords of a marketplace shared one request pool of the same total). The comparison now deduplicates the products across the queries of a marketplace; before, the crawler did, and it skipped the products it had seen before cutting a page to what the keyword still lacked. That order is kept (review of PR #69: cut first, a page lost its products after the cut, and a keyword sharing products with an earlier one could end at 19 of 20 with one more request). A crawl that may meet products already handed in (several keywords, or a retry) is uncut: the comparison skips the products an earlier crawl handed in (written, left out by the filters, or read beyond its keyword's share, as the crawler's seen set did), cuts the page to what the keyword lacks, before the filters, then ends the crawl at the keyword's share. A product found by two keywords still appears once, under the first.
When no search stops on an error. One keyword: same requests, rows and log lines as before. Several keywords: same rows, ranks and requests as one crawl of every keyword, with one exception. A keyword whose pages bring no new product besides those of earlier keywords is no longer stopped after two such pages (Etsy with a price range: three pages without a new listing in the range), since each crawl knows only its own products. It reads on to its page cap (the pages its share needs, plus 1, or plus 3 with a price range or a condition), nothing charged. There is no difference while a keyword reads at most two pages (the default 20 products). Log of a several-keyword run: the reused crawler ends each keyword with result limit reached after page N (AliExpress, eBay and Walmart then skip that page's own line), and Etsy's warning about a price range that left out most listings quotes of Infinity instead of the keyword's share. Columns, SUMMARY, prices and inputs are unchanged. A search refused on every page now costs two more blocked requests, which are not charged. README: one line under "Limits and behaviour worth knowing" and the FAQ on missing products.
Tests: the control scenario replayed on the committed Etsy fixtures (page 1 refused once, page 2 refused once). The old code gives 1 error and 18 of 26 listings; now 0 errors, with the products and ranks of an unblocked run and none twice. Fake marketplaces cover the retry rules: stop at the share, an error counted once when the retry fails again, no retry for a full search or a clean short one, none after the charge limit, deduplication across queries. The fake crawlers cut their pages as the real ones do. Real crawlers: two keywords sharing products, with and without a price range, give the rows, ranks and requests of one crawl of every keyword (the code before the repair). On AliExpress the second keyword meets 5 products the first wrote and 10 it read beyond its share; on eBay, Walmart and Etsy both keywords get the same pages. The code first reviewed in PR #69 fails both cases.
Unreleased (repair: eBay prices in the currency of the eBay site)
Symptom (brief docs/cloud-tasks/2026-10-21_ebay-currency-proxy-country.md, 2026-10-21): the release control run CpWgj7LRcws7BC7oZ (country US, currency USD, 3 searches × 4 marketplaces, 1,100 rows, no error) had 100 eBay rows in KRW, all from "wireless earbuds" on www.ebay.com : the eBay residential proxy had no country and one session left from South Korea; the "currency USD" check failed and the release was abandoned.
eBay exits in the country of its site: the eBay adapter adapts the proxy plan to the site searched (localizeProxy, the eBay Actor's withMarketplaceProxyCountry): residential exit in the site's country (US for www.ebay.com , GB for www.ebay.co.uk… ), for the default proxy and for a residential group of the input without a country. A country, proxy URLs, another group or no proxy set by the customer are kept as they are; the other marketplaces are unchanged. proxyPlanFor takes the site as an optional third argument; the "searching" log line shows the exit country.
The eBay crawler's currency guard (eBay Actor, same brief) runs in the comparison: a page in another currency is asked again on up to 3 new sessions, inside eBay's request cap; the adapter hands it the exit country of the plan.
Rows in another currency than their site's stay in the dataset with the currency they show and stay out of the price figures and of the cheapest offer (existing behaviour); they are now named in the marketplace's message ("eBay showed 20 products in KRW instead of USD on www.ebay.com : …"), in the message of their search query and in the log.
Tests: exit country of every eBay site, customer's group / country / URLs / other group / no proxy, Etsy and Walmart unchanged; the real eBay crawler on a derived KRW page (the US capture in KRW, see the eBay Actor's derivedKrwPage): one KRW page then dollars (USD rows, no remark), KRW pages that persist (KRW rows, remarks, cheapest offer from AliExpress).
README: the currency FAQ says that eBay's proxy exits in the country of its site and how a product in another currency is reported.
Unreleased (quality score: input documented for AI agents)
Brief docs/cloud-tasks/2026-10-20_studio-quality-score-insights.md (2026-10-21): src/store.test.ts now runs the shared checks of @scraper-studio/core/testing (runInputQualityChecks with this Actor's result limit, keywords × marketplaces × products per marketplace, and runSearchTextChecks on "Price Comparison"), the rules of Apify's quality score and of what an AI agent receives through the Apify MCP server.
Descriptions rewritten with the reason of each default: searchQueries (duplicates searched once), maxResultsPerMarketplace, country (US: the only country whose four sites were verified), condition; currency says that rows in another currency are left out of the cheapest offer; priceMin and priceMax give an example and priceMax must be at least priceMin.
Bounds added to the form: maximum 1,000,000,000 on priceMin and priceMax (the code accepts any non-negative amount; no realistic value reaches the bound).
maxResultsPerMarketplace is prefilled with 10 in the form (the default stays 20): the prefilled input Apify tests daily (30 products) is exactly our daily check.
inputs/daily-check.json: currency removed (was USD), as in the prefilled form; the run is the same, since an empty currency means the currency of the country, USD for the United States. The live daily-check task follows when the pilot re-sends it. store.json › dailyCheck.what says it is the prefilled form.
Store texts carry "price comparison": actor.json description opens with "Product price comparison across" (was "Compare product prices across"), store.jsonseoDescription with "Price comparison of" (was "Compare prices on"); new searchSynonyms (price comparison scraper, compare prices across marketplaces, cross-marketplace price comparison, cheapest offer), all within the first 3,000 README characters, the second one added to the opening paragraph. README input table: prefill of maxResultsPerMarketplace, priceMax at least priceMin.
No code change: rows, logs and charges are identical.
Unreleased (repair: memory of the run)
512 MB by default (brief docs/cloud-tasks/2026-10-21_ecommerce-compare-fix-release.md, 2026-10-21). The control run of the first release (6idsXSoQrCtkEN3yg, build 0.1.1, 256 MB, 100 products per marketplace on the four marketplaces) was stopped after 801 products, with no error line, no status message, no SUMMARY and no RUN_STATS record. Two marketplaces crawl at once, each with a crawler that its own Actor validated alone at 256 MB, and above 64 products eBay reads its 240-listing page (2.7 MB of HTML, about 110 MB to parse). The default memory is now 256 MB per marketplace crawled at once (.actor/actor.json, store.json, the 8 example tasks). minMemoryMbytes stays at 256 for small runs such as the daily check (10 products per marketplace; the 160-product validation run succeeded at 256 MB).
V8 heap capped at 3/4 of the run's memory (ACTOR_MEMORY_MBYTES; Dockerfile CMD), so that garbage is collected before the process goes over the run's memory.
Memory watch (src/memory.ts): the resident memory is sampled every second, with a warning at 80 % and at 95 % of the run's memory. The peak goes into RUN_STATS.extra.memoryPeakMbytes and into the final Done: log line.
Crashes outside the main flow: an uncaught exception or an unhandled promise rejection is now logged as an ERROR line and fails the run with a status message. Before, Node ended the process with a bare stack trace.
Rows, SUMMARY, charges and the other log lines are unchanged. Tests: the watch's thresholds and peak, the memory settings against the number of marketplaces crawled at once (the test fails on the 256 MB of 0.1.1), the heap cap evaluated by sh, and the control input kept at the scale that failed.
0.1 — 2026-10-21
First version (brief docs/cloud-tasks/2026-10-20_ecommerce-compare-build.md, built offline on the committed fixtures, to validate at the first runs).
One run searches every keyword of searchQueries on the chosen marketplaces (AliExpress, eBay, Walmart by default; Etsy on request), two marketplaces at a time, each with the crawler, transport and proxy of its own Actor (imported and bundled, never called Actor to Actor): AliExpress and Walmart on the Apify datacenter proxy, eBay on residential, Etsy on residential with a US exit.
One row per product in the shared ecommerce-product/1.1 schema plus marketplace and rankInMarketplace; SUMMARY record with the count, lowest, median and highest price and the cheapest offer per keyword and marketplace, the cheapest offer across marketplaces per keyword, and the status of every marketplace (ok, empty, blocked, failed, not-served, skipped).
country (10 countries) picks each marketplace's site; currency defaults to the country's; a marketplace without a site for the country is skipped and reported. eBay searches fixed-price listings only (the current bid of an auction is not a price to compare).
condition is sent to eBay's search filter and checked on the condition labels of eBay and Walmart rows; AliExpress and Etsy (no condition shown) are kept for new and skipped for used. priceMin / priceMax are sent to each marketplace's own filter where it has one and checked again on every row.
A marketplace that fails, is blocked or finds nothing does not stop the others; the run fails only when no marketplace returned any product.
The SUMMARY figures leave out AliExpress welcome-deal prices (first-time buyers only, attributes.newShopperDeal): the rows stay in the dataset, newShopperDeals counts them. A marketplace whose turn comes after the charge limit of the run is not searched (skipped).
Pay-per-event: one result event per product row, whatever the marketplace.
