# Changelog of Shopify Scraper (`autofacts/shopify`) Actor - **URL**: https://apify.com/autofacts/shopify/changelog.md - **Full Actor documentation**: https://apify.com/autofacts/shopify.md ## Changelog #### 23/09/2026 (morning) - **Fixed**: A store that refused a run's request, and refused every retry of it too, is now read on that run. The URL used to fail. - **Changed**: Each URL's report to run monitoring also counts how often a store's bot check refused it, apart from its retries. - **New**: 15 tests. #### 23/09/2026 (later) - **Changed**: A store or collection URL on a site that is not, or is no longer, a Shopify store fails at once with a message saying so, including a site that answers with a web page instead of data. It used to be retried three times. - **Fixed**: A store whose catalog briefly failed on one of its two paths is no longer written off as not being a Shopify store. - **Changed**: A store that requires a login, or that its owner has frozen, fails at once instead of being retried three times. - **New**: 13 tests. #### 23/09/2026 - **Changed**: Each URL's report to run monitoring now covers every attempt at it, with how often it was retried and how long it waited out rate limits. - **Fixed**: A URL that failed is no longer reported to run monitoring as having taken no time, or, when the store answered with a page instead of data, as a success. - **New**: 16 tests. #### 22/09/2026 (late night) - **Fixed**: A renamed product is found even when the store refuses the first request for its page. That used to fail the product with a 404 on some runs and not on others. - **Fixed**: A renamed product that is read from its new address no longer logs an error first. - **Changed**: A 404 on a product whose page could not be checked for a rename now says so, instead of reading as a product that is gone. - **Changed**: A request that fails in transit is named, with its store, in the log and in the error message. When a store's own address cannot be read and its details are read another way, the log says why. - **New**: 19 tests. #### 22/09/2026 (night) - **Fixed**: A whole-store or collection crawl reads to the end of the catalogue. It could stop after the first page and return a fraction of the store, while the run still succeeded. - **Fixed**: The `/collections` listing reads to the end the same way. - **Fixed**: A collection URL with a tag keeps its tag when it is retried. A retry used to crawl, and bill, the whole collection. - **Fixed**: A run given both a collection and a tag of it no longer skips the collection's first pages after the tagged crawl fails part-way. - **New**: 13 tests. #### 22/09/2026 (evening) - **Fixed**: A product URL whose product the store has since renamed returns the product from its new address instead of failing. - **New**: A product reached that way carries `source.redirectedFrom`, the URL you gave. `canonicalUrl` is the page the store now serves, which can be a replacement product rather than a renamed one. - **Changed**: A product URL whose product is no longer on the store fails at once instead of being retried three times. - **Fixed**: A response that stops arriving part-way is retried, and reported as a timeout instead of as a bot challenge. - **Fixed**: In search mode, a store whose search fails is skipped and the remaining stores are still searched. It used to end the run. - **Changed**: In search mode, a product that cannot be read is skipped instead of failing the run. A run that stores nothing still fails, naming what it gave up on. - **Fixed**: A catalogue or product request that fails in transit no longer names the address it was sent to. - **New**: 32 tests. #### 22/09/2026 (later) - **Fixed**: Prices are no longer returned in another country's currency. A store that prices per country could be crawled in a currency an unrelated run had seen, which changed the prices themselves and not only their label. - **Changed**: A run uses the Apify proxy unless it says otherwise, and the log names the proxy it used — or warns that it used none. - **Fixed**: Custom proxy URLs are used. They were accepted in the input and then ignored. - **Changed**: Run logs and error messages name only the store address you supplied. #### 22/09/2026 - **Fixed**: Hello Molly can now be crawled. `hellomolly.com` and `hellomolly.com.au` both return products, each in its own currency. - **New**: A store that cannot be identified from its own pages is now identified automatically, instead of having to be added by hand. - **Fixed**: A store that could not be reached on one run is no longer skipped on every run after it. - **Fixed**: A store held under the wrong details is corrected on its next run instead of staying wrong. - **Fixed**: Product pages are read on stores that used to refuse them. - **Fixed**: A bot challenge is no longer treated as a rate limit. It used to cost a minute of waiting per product and could never succeed. - **Changed**: A store whose product pages keep refusing is left alone after three tries, instead of being retried for the rest of the run. - **Changed**: The three per-run fetch caps — panels, inventory and storefront detail — are unlimited and no longer appear in the input form. A cap of 200 used to leave the rest of a larger crawl without the data asked for. They can still be set in the JSON input. - **New**: 38 tests. #### 20/09/2026 (later) - **Fixed**: "Grab product information panels" works on its own on a collection crawl. It used to return nothing unless "Grab storefront detail" was on as well. - **Fixed**: The "Grab product information panels" toggle appears in **Extra data per product** beside the other two enrichment switches. It used to render inside the collapsed **Proxy** block, where it was invisible. - **Changed**: `grabRealInventory` no longer spends a request per product on stores that publish no live stock counts. - **New**: Products carry `contentSections` — the labelled panels a storefront shows outside the description (tabs, accordions and similar), as rendered. Labels are verbatim; filtering boilerplate such as shipping or returns is left to the consumer. - **Changed**: `description` is unchanged. A Specifications table shown in its own tab appears in `contentSections` as well, without being removed from `description`. - **New**: A store that renders no panels is left alone after three products come back empty, and that verdict survives into later runs. - **New**: `grabContentSections` (default **off**) turns panel extraction on, and a fifth charge event `content-sections` bills one per product that actually carries panels. An existing run is untouched until the flag is set. - **Changed**: With the flag on, both a product URL and a collection crawl extract panels. `grabContentSections` is independent of every other switch. - **Changed**: Panel fetches are paced like the other enrichment fetches. - **New**: 30 tests. #### 20/09/2026 - **Fixed**: A run that stores nothing now exits FAILED, naming the start URLs it gave up on. A run that stored anything still succeeds. - **Fixed**: A migrated run no longer re-stores what the first process already wrote. - **New**: A store root whose catalogue fails to list is tried a second way before being given up on. - **New**: A domain that cannot list a catalogue at all fails permanently instead of being retried. - **Fixed**: A rate-limited request backs off and retries instead of failing, honouring the store's own `Retry-After` when it sends one. - **New**: A rate limit is held against the whole store, so every request waits it out. - **Fixed**: An interrupted paginated walk resumes at the page that failed instead of page 1. - **Fixed**: A product already stored is skipped before its extra requests, not after. - **Changed**: Product start URLs are crawled before store and collection URLs, so a run given both keeps the fuller record. - **Changed**: Catalogue pages are paced apart, and further apart on a store that has rate-limited the run. #### 14/09/2026 - **Changed**: A product URL reads stock availability more reliably, and pays for the product page only when nothing else answers. - **Changed**: Availability degrades in three steps. A partial answer does not trigger the extra fetch. - **New**: First tests in this actor (`npm test`). - **New**: Added `grabStorefrontDetail`, which publishes selling plans and video media. Off by default on collection crawls. - **Changed**: `grabStorefrontDetail` governs the collection path only. A product URL publishes that detail regardless. - **New**: Products carry a derived price roll-up — `price`, `priceRange` (`min`/`max`/`varies`) and `fullyOutOfStock`. - **New**: Variants carry `requiresShipping`, `taxable` and `weightGrams` on every crawl. - **New**: On a product URL, variants also carry `barcode`, `weight`/`weightUnit`, `taxCode`, `fulfillmentService`, `quantityRule` and `quantityPriceBreaks`; products carry `publishedScope` and `templateSuffix`. - **Changed**: A video's poster frame is typed `Video` and points at the video file, instead of being emitted as a second, image-typed entry. **Diffing `medias` counts against an older version will show such products losing one entry.** - **Changed**: Video media merges into the existing image list by source URL, never by id. - **Changed**: Two cross-checks run when the extra detail is fetched. Neither changes output. - **Fixed**: Product data is no longer rejected over stray control characters. - **Fixed**: Image alt text is no longer lost on collection crawls. - **Fixed**: The product-level price roll-up is recomputed after real inventory is applied. - **Fixed**: In search mode a product handle no longer keeps a search-attribution query string. - **Fixed**: A Shopify Markets locale prefix no longer breaks collection URLs. - **Fixed**: A product handle carrying a percent escape is read again. Collection handles were widened the same way. - **Changed**: The unsupported-url error names the URL shapes the actor reads. - **Changed**: An unsupported url is no longer retried. - **Changed**: The dataset schema is restructured so the Apify Store renders a field table. - **Changed**: Every field in the dataset schema carries a description — 114 fields across 15 definitions, plus the three enums. - **Changed**: Dropped four fields never assigned by any code path: `Source.crawlUrl`, `Product.stats`, `Product.extraInfo` and `Product.subProducts`. - **Changed**: `product` price lowered from $0.0025 to **$0.0018**, with BRONZE $0.0016, SILVER $0.0014 and GOLD/PLATINUM/DIAMOND $0.0012. `collection`, `recommends` and `real-inventory` are unchanged. - **Changed**: README publishes the per-event prices and the volume-tier table, and its product output example is a freshly captured record. - **Fixed**: The troubleshooting entry for `?currency=XYZ` now scopes it to stores not using Shopify Markets, and points Markets stores at the locale URL. - **Decision**: Scraping SEO and template fields off the product page was planned and is not being built. - **Known limitation**: Shopify's `model` (3D/AR) media is skipped. #### 25/08/2026 - **Changed**: Actor moved from rental to **pay-per-event**. Four events are charged: `product`, `collection`, `recommends` (one per nested recommendation) and `real-inventory`. - **Changed**: Every request now carries a timeout. - **Fixed**: Duplicate results are no longer stored or billed, across an interrupted request, several start URLs and overlapping collections. - **Fixed**: A run that reaches its **Max total charge** limit stops cleanly and finishes SUCCEEDED. - **Fixed**: Transient failures are retried again, with backoff, and a 4xx never is. - **Fixed**: A failed availability lookup degrades instead of aborting the product. - **Fixed**: `maxRequestsPerCrawl` is exact in search mode; it no longer stores one record over the cap per extra start URL. - **Changed**: Crawl errors carry the upstream HTTP status. A 200 carrying an anti-bot challenge page is named as such. - **Changed**: Dependencies updated. - **Fixed**: The published image builds and runs correctly. #### 09/07/2026 - **New**: Added `grabRealInventory` — reads per-variant `quantityAvailable`, `inventoryTracked` and `inventoryPolicy`. Free on product URLs; one extra request per product on collection crawls. A count of 0 sets `OutOfStock`, `<= 5` sets `LowInStock`. - **Changed**: Bumped the `apify` SDK. #### 05/06/2026 - **Changed**: Actor is now compatible with limited permissions, and run monitoring no longer uses ad-hoc webhooks. #### 29/12/2025 - **New**: Added `maxRecommendationsPerProduct` to fetch product recommendations. - **New**: Keyword-based product discovery via `query`. #### 10/12/2025 - **New**: Improved currency detection. - **Fix**: Refreshed the request headers. - **Fix**: Corrected a few store lookups. - **Fix**: Resolved a build error. #### 02/03/2025 - **New**: Stores behind stricter gateways can be crawled. - **New**: Passthrough currency support. - **Improved**: Stock status detection.