# Changelog of Shopify Store Scraper - Full Product Catalog Export (`halvenlabs/shopify-store-scraper`) Actor

- **URL**: https://apify.com/halvenlabs/shopify-store-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/halvenlabs/shopify-store-scraper.md

## Changelog

All notable changes to this Actor are documented here.

### 1.2.0

#### Features

- **Collection URLs are now scraped as collections**: an input like `https://shop.com/collections/sale` scrapes only that collection (via Shopify's per-collection `products.json`), instead of being silently widened to the whole store. Store-root inputs behave exactly as before.

#### Fixed

- **New-product mode no longer loses products**: keys of new products now enter the "last seen" snapshot only after their row is actually pushed. Previously, products cut off by `maxItems` or the pay-per-event budget were silently marked as seen and never delivered in later `onlyNewProducts` runs.
- Entity decoding in plain-text descriptions no longer double-decodes escaped sequences (`&amp;lt;` now correctly becomes `&lt;`, not `<`).
- An empty product sub-sitemap no longer triggers the "not a Shopify store" error meant for the entry sitemap.

#### Security

- Redirect SSRF guard is now actually enforced: every redirect hop is re-validated (scheme, hostname, DNS resolution) via a got `beforeRedirect` hook, on both crawler requests and the `cart.json` currency probe.
- Sitemap same-origin filtering now requires an exact `origin/` prefix, closing a lookalike-host bypass (`https://shop.com.evil.com` no longer passes the check for `https://shop.com`).

#### Improved

- Sitemap fallback now starts from `/sitemap.xml` and follows product sub-sitemaps (standard `sitemap_products_N.xml` and headless `/sitemap/products/N.xml` layouts), covering large stores that split products across many sitemaps. Follows are capped at 20 sub-sitemaps per store so a pathological sitemap index cannot burn the whole run.
- Concurrent pages of the same store now share a single `cart.json` currency lookup instead of firing duplicates.
- Global request rate capped at 300/min to stay under storefront rate limits.
- Pure data-shaping helpers extracted to `src/product.js` with a unit-test suite (`npm test`).

### 1.1.0

#### Features

- New **"new product" mode**: every row gets an `isNew` field, and the `onlyNewProducts` input filters output to just products/variants not seen in a previous run. History is tracked per Apify account in a named key-value store with a 90-day rolling retention, independent of `maxItems`/variant-expansion settings.

### 1.0.0

Initial release.

#### Features

- Full extraction with a flat, CSV/Excel-friendly dataset schema and a Store output table view.
- `maxItems` cap enforced globally (safe under concurrency), plus input validation with clear, fail-fast errors.
- Pay-per-event pricing: `actor-start` + per-result-item charging (charged only after an item is produced).
- Apify Proxy support, automatic retries with backoff, and graceful per-item degradation (one bad page never crashes the run).
- Resumable & migration-safe via `Actor.useState` (no double-charge on platform migration).

#### Security & robustness

- Input/redirect SSRF guards (block localhost, private/reserved IPs, cloud-metadata, `file://`) with DNS-rebinding resolution checks where user URLs are fetched.
- ReDoS-safe extraction patterns and response-body size caps.
- Push-failure rollback so a failed push never silently loses items or mischarges.