# Changelog of Etsy Listings Scraper & Price Tracker (`myth84/etsy-search-scraper`) Actor - **URL**: https://apify.com/myth84/etsy-search-scraper/changelog.md - **Full Actor documentation**: https://apify.com/myth84/etsy-search-scraper.md ### Changelog History of `etsy-search-scraper` (folder `actors/etsy-search`). Versions are the Apify build numbers. #### Unreleased (quality score: input documented for AI agents) - Brief `docs/cloud-tasks/2026-10-20_studio-quality-score-insights.md` (2026-10-21): the input schema passes the shared checks of `@scraper-studio/core/testing` (`runInputQualityChecks`, `runSearchTextChecks`), the rules of Apify's quality score and of what an AI agent receives through the Apify MCP server. - Descriptions rewritten so that each one states its default and the reason: `searchQueries` (the default only serves runs started without any keyword or URL), `startUrls` (757 characters cut to 496, with an example; the details stay in the README section *Price range and search URLs*), `locale`, `maxResults` (also its range and the form prefill), `freeShippingOnly`, `includeSponsored`, `onlyChanges`, `monitorStoreName` (with an example), `maxRequests`, `delayMs`, `debugStoreHtml`; `priceMin` and `priceMax` gain an example, `priceMax` at least `priceMin`. - Bounds added to the form: `maximum` 1,000,000,000 on `priceMin` and `priceMax` (the code accepts any non-negative whole amount; no realistic value reaches the bound). - `maxResults` is prefilled with 20 in the form (the default stays 60): the prefilled input Apify tests daily is exactly `inputs/daily-check.json` (20 listings, ceramic mug, US / USD), which is unchanged, so the live daily-check task needs no update. - `store.json` gains `searchSynonyms` (Etsy scraper, Etsy price tracker, Etsy search results, product research), all already within the first 3,000 README characters; no change to the title, the descriptions or the README. - No code change: rows, logs and charges are identical. #### Unreleased (README visibility) - **README first screen and links** (brief `docs/cloud-tasks/2026-10-20_studio-readme-visibility.md`, 2026-10-20). The Store search indexes only the first ~3,000 README characters, so they now hold: an opening sentence with the phrase agents search and its synonyms, a "No login, no cookies, no personal data" line, the Free-plan price in figures in *How much* ($2.40 per 1,000 listings; Apify's $5 monthly credit covers about 2,083), computed from `pricing/2026-10-20_pay-per-event.autopilot.json` (Free tier of the primary event, one `apify-actor-start` event deducted), and a real output snippet (11 key columns of the README example row). Further down: *Use it from your AI assistant* (MCP URL `https://mcp.apify.com?tools=myth84/` and a ready-to-copy prompt with a minimal input), *Alternatives and switching* (official API, home-made scraper, input-name mapping table, no competitor named), *More scrapers by Flatrow Data Studio* (Store links to the other Actors of the studio) with an invitation to rate the Actor (no counterpart), and the non-affiliation line at the very bottom. The cost section and the MCP cost bullet quote the same figures; the line 1 heading is unchanged. Checked by `runStoreChecks` (`readmeVisibility`): keyword in the opening prose, snippet equal to the example row and within 3,000 characters, figures equal to the pricing file in force (a new grid fails the test until the README follows), MCP block with a valid input, links to existing Actors of the same studio only, no `fpr=`. No code change: rows, logs and charges are identical. #### Unreleased (version 0.2, to deploy first as a `beta` build) - **Price range and pasted URLs** (brief `docs/cloud-tasks/2026-10-20_etsy-search-extend-monitor-filters.md`, 2026-10-20; reworked after the review of PR #32, user decision of 2026-10-20, option b): Etsy's robots.txt disallows `order=`, `min=` and `max=` for crawlers (audit §1), so **no sort option is offered** (Etsy's default order only) and **no request ever carries `order`, `min` or `max`** (test on `buildMarketUrl`, pasted URLs included, and on every request of the crawler tests). `priceMin` / `priceMax` (whole amounts in the locale currency, bounds included) are a **local filter on the listings fetched**: a listing outside the range, without a price or shown in another currency than the locale's (the USD-rendered first cards of a `/fr/` page) is neither written nor charged nor counted against `maxResults`; pagination goes on past pages whose listings are all left out (they still count as new listings for the "two pages without new listings" stop) and stops at `maxResults` listings in the range, the last page, **3 consecutive pages without any listing in the range** (cost guard added after the second review of PR #32: such pages are paid in residential traffic, about 0.0008 $ each, and never charged; without it a range no listing falls in read up to 150 pages per default run), the 200-page cap or `maxRequests`; a search stopped by the last three is warned about at its end and in the final log (`RUN_STATS.extra.priceFiltered`, `priceFilterStopped`, `priceFilterCapped`); after the cost-guard stop the monitor record is written as after an exhausted search; a listing left out by one search can still be kept by another one of the run. `startUrls`: market pages `/{prefix/}market/` and search URLs `/{prefix/}search?q=…`, converted to the market route of the audit with the locale of the URL and its `free_shipping` parameter; `order` is removed and `min` / `max` become the bounds of the local filter when the input sets none (each bound on its own), with one warning per URL in the log before the first request; any other URL, host or locale prefix fails the run before any request with a message naming the URL. `explicit=1` accompanies the free-shipping filter only, as on the captured page. - **Price monitoring**, same rules as the AliExpress Actor and **same option names as the eBay Actor** (aligned after the second review of PR #32, as the brief asks): `monitorPrices` (off by default), `onlyChanges` and `monitorStoreName` (default `etsy-price-monitor`, lower-cased, Apify storage-name shape, 63 characters at most); previous prices kept in that named key-value store of the customer's account, one record per search (words or URL, locale, price range, free shipping); with `onlyChanges` only new listings and price changes are returned and charged, otherwise every compared listing with its `changeType`; first run = reference (`RUN_STATS.monitorBaseline`); `RUN_STATS.monitor` summary and `monitorCompared` / `monitorNew` / `monitorPriceChanged` / `monitorUnchanged` counters; records written only after a search that ended cleanly (fail-closed: not after a block, a failed API call, a charge cut or a run that ran out of budget); a monitoring run without any change succeeds with an empty dataset; `maxResults` = listings compared per search; same currency only (the USD-rendered first cards of a locale-prefixed page compare as `unchanged` until the currencies match). - Each crawl task now carries its own search parameters (`SearchTask.params`: words, locale, filters) so that keyword and URL tasks of different locales run in one run; `Accept-Language` follows the task. Without the new options the requests, rows and log lines are those of 0.1 (regression test on the fixtures). - Dataset: new view *Changes*; the four monitoring columns are documented as filled in monitoring runs. Input form: *Results and filters* section extended, new *Price monitoring* section. README: sections *Price range and search URLs* and *Price monitoring*, FAQ reworded (10 entries; the price filter applies to the results fetched, not to Etsy's search). One more example task (`etsy-monitor-ceramic-mug-prices`, 9 in total). Tests: URL conversion, removed parameters and refusals, input defaults with URLs only, local price filter (bounds, missing price, other currency, `maxResults`, pagination, caps, the 3-page cost guard and its reset, across searches), monitoring options (eBay names, store-name shape, old draft names ignored), monitor record and comparison, crawler locale per task, monitoring baseline, `onlyChanges` on and off, caps, identities, fail-closed, unknown record format, unreadable store. - To check at the first run (Mac): a run with a price range reads more pages for the same `maxResults` (cost per listing kept, `RUN_STATS.extra.priceFiltered`, `priceFilterStopped`); a `/search?q=` URL pasted by a customer converts to a market page that exists. - **Store texts** (brief `docs/cloud-tasks/2026-10-20_studio-promote-store-texts.md`, 2026-10-20): README opens with what you get / for whom / how much, shows the real input of the validation run, a pricing table that points to the Pricing tab (no figure in the text), a *For AI agents (MCP)* section with a minimal input, the key output fields, the cost per 1,000 results and the typical time; FAQ kept to 8–10 questions; no proxy or anti-bot detail left in the texts visible on the Store. Tests on the Store lengths (`seoTitle` ≤ 60, `seoDescription` ≤ 155, `description` ≤ 300), the README structure and the forbidden mentions. No code change. #### 0.1 — 2026-10-20 - First version, written offline on the committed fixtures of the audit (`docs/audits/etsy.md`, brief `docs/cloud-tasks/2026-10-19_etsy-build.md`, 11 captures by the probe on 2026-10-19: `/market/ceramic_mug` pages 1 and 2, `/fr/` and `/uk/` variants, the free-shipping filter, a 404 term, a category hub, two DataDome pages and three `market_async_listings` answers). Each result page is read in two requests of the same sticky session: `GET /{prefix/}market/?ref=pagination&page=N` (8 server-rendered cards, `Etsy.Context.data` read by key search and brace balancing: `market_request_param`, the 36 organic and 16 sponsored ids the page defers, their logging keys, `csrf_nonce`, `detected_locale`) then `POST /api/v3/ajax/bespoke/member/neu/specs/market_async_listings` (the page's objects sent back unchanged, `x-csrf-token`, `x-detected-locale`, same cookie jar: 46 to 52 cards of the same markup). Pagination until `maxResults`, a page without next link, two pages without a new listing or `maxRequests`; global deduplication by listing id (sponsored cards repeat); `searchRank` = (page − 1) × 60 + position; pay-per-event `result` charging through the shared runner, `RUN_STATS`, explicit failure on empty output (the keywords without market page are listed in the message). - **Transport**: got-scraping only, on **Apify residential proxies with a US exit by default** (`proxyConfiguration` default `RESIDENTIAL` + `US`): the only combination DataDome let through in the audit (32/34 pages, API 10/10); `impit` was refused 7/7 even from residential IPs and is never used. DataDome pages (`captcha-delivery.com`, `var dd={`, "Please enable JS and disable any ad blocker") and the API's captcha redirect JSON are block markers; an unusable page or API answer rotates the session once (new proxy session, cookie jar and header fingerprint), then the search gives up (page) or keeps the 8 rendered cards (API). A 404 means Etsy has no market page for the term: the term is skipped with a warning, the run goes on. A term redirected to a category hub (`/c/…`, `page_type` `category_page_hub`) is read as a single page of 30 cards without API call. - **Locales**: path prefix of the input `locale`: `''` (etsy.com, USD), `uk` (GBP) and `fr` (EUR) validated; `de`, `es`, `it`, `nl`, `ie`, `ca`, `au` follow Etsy's URL convention and are marked "to confirm" in the form and the README. `x-detected-locale` and `Accept-Language` follow the page's own `detected_locale` and the prefix. **Observed on the `/fr/` capture**: the 4 cards rendered on the page carry a `$` symbol and a USD `ld+json` offer while the deferred cards are in `€`: the displayed symbol decides `currency` (a bare `$` is the locale's dollar currency on `/ca/` and `/au/`, USD elsewhere). To check at the first run on pages 2+ and on `/uk/` pages. - Output schema `ecommerce-product/1.1`: `price` / `originalPrice` from `currency-value` texts ("16.33", "42,89", "1.234,56"), `discountPercent` from "(33% off)" / "(58 % de réduction)" else computed, `rating` from `input[name="rating"]`, `reviewCount` from "(15.4k)" / "(159)" / "(4,2 K)" / "(5,871)", `shippingFree` from the badges ("FREE shipping", "FREE UK delivery", "Livraison GRATUITE" → true; "Plus postage" → false), `attributes.starSeller` from the Star Seller badge, `isSponsored` from the `ad-listing-title-` heading id or the "Ad by Etsy seller" label (compared, never stored), `inStock` from the `ld+json` `ItemList` offers (first cards only; `brand` never read), `imageUrl` upgraded to `il_600x600`, `url` reduced to `https://www.etsy.com/listing//`; `storeType` `unknown` so that every store column is cleared; `data-shop-id` and the shop-name container are never read; `stripPersonalData` on every row. - **Decisions**: no `maxPages` input (`maxResults` and `maxRequests` bound a search; a safety cap of 200 pages exists in the crawler); `maxRequests` counts pages and API calls (default 300, two per page); `viewCountryCode` is the proxy country of the input (US by default, null with own proxies) since Etsy prices follow the locale prefix, not the IP; sorting and price bounds (`order=`, `min=`, `max=`) are not offered (disallowed by Etsy's robots.txt); `is_star_seller`, `instant_download`, `is_personalizable` filters left for a later version (untested). - Known limits (audit §3, §9): a keyword is served only when Etsy has a market page for it (8 of 12 common terms in the audit; 404 otherwise), pagination depth unknown (links to page 5 seen on page 1, `maxResults` capped at 1,000), ratings and review counts are the shop's, not the listing's, no stock, shipping cost nor ships-from country on market cards. - Store readiness: README, input schema (`apify validate-schema` passed), dataset schema with Overview / Prices / Ranking views, output schema, 8 example tasks (residential US proxy, one page each), consistency tests (README ⇔ schemas ⇔ code, Ajv replay of the 48 fixture rows), crawler tests on a fake HTTP client (session, headers, nonce and locale on the API call, pagination, deduplication, caps, 404, category hub, DataDome and API rotations, sponsored filter), shop-name leak tests.