# Changelog of 🧩 Shopify Apps Spy + Product Scraper (`kazkn/shopify-scraper-apps-spy`) Actor

- **URL**: https://apify.com/kazkn/shopify-scraper-apps-spy/changelog.md
- **Full Actor documentation**: https://apify.com/kazkn/shopify-scraper-apps-spy.md

## Changelog

All notable changes to **Shopify Apps Spy + Product Scraper** are documented here.
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

The Apify build numbers track each deployment to the Apify Store; the version
numbers below reflect functional changes visible to users.

***

### \[1.2.4] — 2026-06-24

#### Fixed

- Split actor output into separate run-scoped datasets for Products, Store summary, Apps detected, and Reviews so Apify Console sections no longer display unrelated rows.
- Flattened product rows for cleaner CSV exports: tags and image URLs are now string fields, while nested app/store metadata and raw HTML were removed from the Products dataset.
- Added focused dataset schemas for each output tab and updated the README output examples.

***

### \[1.2.3] — 2026-06-24

#### Fixed

- Improved app detection for Shopify App Extension script URLs, including Shopify Inbox, Smile.io, and Free Gifts widgets.
- Added a dedicated **Apps detected** dataset view in Apify Console.
- Added review diagnostics to `store_summary` so Full mode explains whether reviews were found, no supported review app was detected, or providers were tried with no results.

***

### \[1.2.2] — 2026-06-24

#### Fixed

- Fixed Apify proxy session IDs for hyphenated Shopify domains such as `kbeauty-cosmetics.com`, which previously stopped the run before scraping began.
- Automatically infers the collection filter when users paste a Shopify `/collections/...` URL and leave the advanced collection filter empty.

***

### \[1.2.1] — 2026-06-24

#### Changed

- Moved optional input controls into a collapsible **Advanced options** section in Apify Console, starting at collection filtering and including variants, review limits, concurrency, proxy, and debug settings.

***

### \[1.2.0] — 2026-06-24

#### Fixed

- Hardened store URL validation to reject non-public hosts, private IPs, localhost, ports, and unsupported protocols before any request is made.
- Preserved extended app categories (`sms_marketing`, `pricing_ab_testing`, `cookie_consent`, `geo_region`) inside product-level `apps_detected` rows.
- Fixed `store_summary.review_count` so multi-store runs report the current store's reviews instead of the global run counter.
- Restored `npm test` with focused Node regression tests and refreshed vulnerable transitive dependencies.
- Removed the unsupported placeholder tier from the public input and output contract. Active modes are now `basic`, `standard`, and `full`.
- Updated output schema, dataset views, and README wording to match the default-dataset `record_type` output.

***

### \[1.1.0] — 2026-05-26

#### Fixed (critical — actor was returning only the store domain instead of products)

- **`Insufficient permissions for the dataset` hung the run when the actor was called via the Apify API or by any user other than the actor owner.** Apify runs every public actor under a `LIMITED_PERMISSIONS` token that can write to the run's default dataset but cannot open globally-named datasets like `apps` or `reviews`. The previous code called `Actor.openDataset('apps')` which hung indefinitely under that token, freezing the run after the first store and leaving the dataset effectively empty (just an error/domain record). The actor now writes every row to the default dataset only, tagged with a `record_type` field so the output stays filterable.
- Anti-bot block detection (Cloudflare Bot Fight Mode, Akamai, PerimeterX, Datadome) — datacenter IPs sometimes get a 403 from anti-bot, which the previous code misreported as `not_a_shopify_store`. The actor now distinguishes "blocked by anti-bot" from "not a Shopify store" and auto-retries with a residential proxy on the first block. If the block persists, the diagnostic record now includes an actionable `next_steps` array (whitelist Apify IPs in Cloudflare, disable Bot Fight Mode for `/products.json`, etc.).

#### Added

- `record_type` field on every dataset record (`product` / `apps` / `reviews`). Filter the default dataset by this field instead of opening separate named datasets.
- `proxy_tier_used` field in diagnostic records (`datacenter` / `residential_fallback`) so users can see exactly which network path succeeded.
- Anti-bot signal codes in the `signals` array (`anti_bot_cloudflare`, `anti_bot_akamai`, etc.) for downstream filtering.
- 4-second timeout on every `Actor.charge()` call so a hung billing endpoint can never freeze the run.

#### Changed

- **Output structure.** All records now flow through the default dataset. Existing scripts that read from the `apps` and `reviews` named datasets need to switch to `dataset.filter(r => r.record_type === 'apps')` etc. on the default dataset.
- **`include_variants: true` now emits one row per variant** (matching the README contract) instead of a nested `variants[]` array on a single product row. Each variant row carries the full product fields plus per-variant `variant_id`, `variant_title`, `sku`, `option1/2/3`, variant-level `price`, `compare_at_price`, `available`, and `main_image` (variant featured image when available). Verified on `aplusnutri.com`: the product "Équilibre et bien-être au quotidien" now correctly produces 5 rows (Limonade framboise / Pastèque fraîche / Cerise gourmande / Orange douce / Piña colada).
- **Revenue placeholder row** (emitted at `extract_level: 'pro'`) moved from the named `revenue` dataset to the default dataset with `record_type: 'revenue'`. Same LIMITED\_PERMISSIONS rationale.

#### Added (apps detection coverage — major expansion)

- **30+ new detectors** across 4 new categories:
  - **Analytics**: Microsoft Clarity, Segment, Mixpanel
  - **Pricing / AB testing** (new category): Intelligems, Visually.io, Dexter
  - **SMS marketing** (new category): Attentive, Postscript, Emotive
  - **Bundles / Upsell**: Kaching Bundles, Bundler, FastBundle
  - **Geo / Region** (new category): Boostymark Region Block, Shopify Geolocation
  - **Cookie consent** (new category): Axeptio, Iubenda, Cookiebot, OneTrust, Termly, Shopify Privacy Banner
  - **Live chat extended**: Shopify Inbox, Re:amaze, Crisp, Freshchat
  - **Currency / i18n extended**: Multi Currency Converter, Bagging
- Verified on `aplusnutri.com`: detection went from 1 app (klaviyo only) to 6 apps across 6 categories — Klaviyo, Microsoft Clarity, Intelligems, Kaching Bundles, Boostymark Region Block, Shopify Privacy Banner.

#### Added (Apify Console UI)

- **4 dedicated dataset views** for clean per-type rendering in Apify Console:
  - **All records** — universal columns with `record_type` first for sort/filter
  - **📦 Products** — image, title, variant\_title, sku, price, currency, availability, vendor, link
  - **🧩 Apps detected** — one row per store, columns per category (email/SMS/reviews/subs/popups/loyalty/chat/upsell/search/analytics/pricing\_ab/consent/geo/page\_builder/i18n)
  - **⭐ Reviews** — store, product, provider, rating, body, author, date
  - **💰 Revenue** — placeholder until the model lands

***

### \[1.0.2] — 2026-05-05

#### Added

- 4-language video tutorials (EN / FR / DE / ES) embedded in README and on Apify Store
- `examples/` folder with 5 runnable code samples covering ICP qualification, app market-share research, cold-outreach personalization, cURL quickstart, and Zapier/Make.com integration
- README badges (Apify Run, MIT License, YouTube Tutorial)
- "Video Tutorial" section in README + Apify Store, mirroring the format used on the Vinted Turbo Scraper actor

#### Changed

- Renamed actor from "Shopify Product Scraper - Apps Spy + Reviews" to **"Shopify Apps Spy + Product Scraper"** to lead with the differentiated apps-detection feature while keeping the product-scraper SEO term
- Aligned `.actor/actor.json` title and `.actor/README.md` with the new name

***

### \[1.0.0] — 2026-04-30

#### Added

- Apps detection across 150+ Shopify apps (email/SMS, reviews, subscriptions, popups, search, loyalty, ad pixels)
- Product catalog extraction via the official `/products.json` endpoint
- Reviews extraction at `extract_level: full` for Yotpo, Judge.me, Stamped, Okendo, Loox, Junip, Reviews.io, Rebuy
- Pay-per-event pricing: $0.003/store, $0.0005/product, $0.001/apps\_detected, $0.0003/review
- 4 extract levels: `basic`, `standard`, `full`, `pro`
- JSON / CSV / Excel / RSS export
- Free $5 monthly credit covers ≈1,500 stores at standard tier

#### Performance

- Polite concurrency of 5 simultaneous requests scans 1,000 stores in ~25 minutes
- No proxy required for the vast majority of stores
- No login, no API key, no headless browser

***

### Roadmap

- **MCP server mode** — query the actor directly from Claude desktop / Cursor (in line with the existing [Vinted MCP Server](https://apify.com/kazkn/vinted-mcp-server) and [GPT Crawler MCP](https://apify.com/kazkn/gpt-crawler-mcp))
- **Theme detection** — useful for agency outbound (Shopify 2.0 vs vintage themes vs custom)
- **More detectors** — every "this is missing" issue gets a 15-minute add. Open an issue to request one.