All notable changes to this Actor are documented here.
- Collection URLs are now scraped as collections: an input like
https://shop.com/collections/sale scrapes only that collection (via Shopify's per-collection products.json), instead of being silently widened to the whole store. Store-root inputs behave exactly as before.
- New-product mode no longer loses products: keys of new products now enter the "last seen" snapshot only after their row is actually pushed. Previously, products cut off by
maxItems or the pay-per-event budget were silently marked as seen and never delivered in later onlyNewProducts runs.
- Entity decoding in plain-text descriptions no longer double-decodes escaped sequences (
&lt; now correctly becomes <, not <).
- An empty product sub-sitemap no longer triggers the "not a Shopify store" error meant for the entry sitemap.
- Redirect SSRF guard is now actually enforced: every redirect hop is re-validated (scheme, hostname, DNS resolution) via a got
beforeRedirect hook, on both crawler requests and the cart.json currency probe.
- Sitemap same-origin filtering now requires an exact
origin/ prefix, closing a lookalike-host bypass (https://shop.com.evil.com no longer passes the check for https://shop.com).
- Sitemap fallback now starts from
/sitemap.xml and follows product sub-sitemaps (standard sitemap_products_N.xml and headless /sitemap/products/N.xml layouts), covering large stores that split products across many sitemaps. Follows are capped at 20 sub-sitemaps per store so a pathological sitemap index cannot burn the whole run.
- Concurrent pages of the same store now share a single
cart.json currency lookup instead of firing duplicates.
- Global request rate capped at 300/min to stay under storefront rate limits.
- Pure data-shaping helpers extracted to
src/product.js with a unit-test suite (npm test).
- New "new product" mode: every row gets an
isNew field, and the onlyNewProducts input filters output to just products/variants not seen in a previous run. History is tracked per Apify account in a named key-value store with a 90-day rolling retention, independent of maxItems/variant-expansion settings.
Initial release.
- Full extraction with a flat, CSV/Excel-friendly dataset schema and a Store output table view.
maxItems cap enforced globally (safe under concurrency), plus input validation with clear, fail-fast errors.- Pay-per-event pricing:
actor-start + per-result-item charging (charged only after an item is produced).
- Apify Proxy support, automatic retries with backoff, and graceful per-item degradation (one bad page never crashes the run).
- Resumable & migration-safe via
Actor.useState (no double-charge on platform migration).
- Input/redirect SSRF guards (block localhost, private/reserved IPs, cloud-metadata,
file://) with DNS-rebinding resolution checks where user URLs are fetched.
- ReDoS-safe extraction patterns and response-body size caps.
- Push-failure rollback so a failed push never silently loses items or mischarges.
