# Changelog of Sephora Product Scraper — Prices, Stock, Shades & Ratings (`autofacts/sephora`) Actor

- **URL**: https://apify.com/autofacts/sephora/changelog.md
- **Full Actor documentation**: https://apify.com/autofacts/sephora.md

## ChangeLog

#### 28/09/2026 — v2.7.2 Store listing copy (docs)

- Store title, SEO title, SEO description and description rewritten in buyer terms: what comes back, from which 29 countries, how it is billed. The description no longer describes the fetch mechanism.
- One count everywhere: 29 countries (US, Canada, 9 EU, 6 Middle East, UK, India, 10 Asia-Pacific). README, `actor.json`, input schema and dataset schema each carried a different number before.
- Review wording corrected: the actor returns rating and review count (plus US sentiment summaries), not review texts.
- `categoryIds` help text in the input form now matches the README: read by EU, Middle East, UK and India. LATAM dropped from customer-facing text until that market is verified.
- Dataset schema descriptions no longer name Sephora's upstream fields, and the overview view no longer claims integer minor-unit prices.
- README field table covers every storefront family; the "Production-tested" bullet naming retired actors is gone. One actor name across Console, `actor.json`, input form and README.

#### 22/09/2026 — v2.7.1 Scraper Insight telemetry restored (bug fix)

- `httpx` is now declared in `requirements.txt`. Builds from 16/09/2026 reported nothing to Scraper Insight — neither request logs nor the run summary.
- Telemetry that disables itself now logs at WARNING, not INFO.
- Run logs no longer name the telemetry endpoint or the app id.

#### 15/09/2026 — v2.7.0 Revenue ledger line per run (feature)

The actor reported every upstream request to Scraper Insight but never reported what a run **earned**, so it was missing from the fleet earnings page entirely — despite being the second-highest earner.

**Added: one ledger line per run**

- `report_run_summary()` in `src/insight.py` posts the run's charged-event counts and the per-event prices that applied to **that caller's subscription tier** to `/api/v1/run-summary`. Apify publishes no creator-earnings API — `/v2/users/me/{insights,monetization,earnings,payouts}` all return 404 — so this is the only record of what a run charged.
- Neither billable event is charged by actor code: `apify-actor-start` is charged by the platform at run start and seeded into the SDK's `ChargingManager` when it initializes, and `apify-default-dataset-item` is mirrored into the charging state on every dataset write. Reading the manager therefore needed no change at any call site.
- The run's business shape (`catalog` / `product` / `mixed`) is tallied from the labels of the requests actually enqueued, and its item count comes from the same per-market counters the final log summary reports. Raw request counts do not track revenue; catalog-type work does.
- Sent as a single awaited request with its own 10-second budget, not through `InsightReporter`'s queue — that queue is drained and closed by the time the run ends, and its 3-second budget is sized for fire-and-forget request logs.

**Changed: `main()` split into a ledger-writing wrapper and `_run()`**

- The crawl had five exit paths and the ledger is only trustworthy if it has every run, so the summary is now written once in a `finally`, with the status derived from whether the crawl raised. The bare `raise` is kept so reporting cannot mask a crawl failure.
- The two early exits that called `await Actor.exit()` now simply `return`. Exiting the Actor also shuts the `ChargingManager` down, which would leave the summary with nothing to read; the enclosing `async with Actor:` exits cleanly anyway.

**Tests**

- `tests/test_insight.py` pins the wire format against the Go `db.FleetRun` decoder: field names, `Decimal` → `float` conversion (stdlib `json` has no `Decimal` encoder, so a stray one would raise inside the POST and lose the run's money silently), an uncapped `maxTotalChargeUsd` omitted rather than sent as a null the dashboard would read as a zero ceiling, a run still reported when the charging state is unreadable, and no report at all from a local run.

#### 23/08/2026 — v2.6.1 MENA product-search wire corrected (bug fix)

Two defects on `/app/backend/v1/shopper/product-search`, both of which failed **silently** — the gateway ignores unrecognised query params rather than rejecting them, so neither ever surfaced as an error.

**Fix: category crawls returned the entire catalogue, mislabelled as one category**

- The actor sent `categoryId`, which this gateway ignores: a valid `C342` returned all **15,827** products where the storefront shows **919**.
- The category needs **two** params sent together — `refine=cgid=<id>` **and** `targetCategoryId=<id>`. Alone, `targetCategoryId` is ignored and `refine` answers `400 CatalogValidationError`, which is why neither showed up in isolation. Both now sent; verified live: `C342` → **918** products, and the top-level ids match the gateway's own advertised counts (C302 2,615 / C303 2,434 / C301 2,013 / C304 758 / C307 1,107).

**Fix: pagination never advanced — every page re-fetched the first**

- The actor sent `start`/`count`; the gateway uses `offset`/`limit`, so every request came back at `offset: 0` with the default page size. Proven live: `offset=100&limit=5` returns `offset: 100` with different products, `start=100&count=5` returns `offset: 0` with 25.

**Provenance**

- Param names and the `key=value` refine encoding come from the AE app's Retrofit interface `rne#b` and its `nid` request model (`com.sephora.sephorame`, classes3.dex): `b(@Query("refine") List<String>, @Query("q"), @Query("sort"), @Query("offset") long, @Query("limit") int, @Query("locale"), @Query("targetCategoryId"))`.
- The `cgid` refinement in the search response advertises its own valid value set (top-level `C301`/`C302`/`C303`/`C304`/`C307` plus `new`, `bestsellers`, `gifts-me`, …), which is how the pairing was found — sub-category ids like `C342` also work once both params are present.
- Tests: `tests/test_mena_category.py` pins the paired category params, the refine encoding and the `offset`/`limit` wire. **Verified live** 2026-08-23 (AE residential proxy): `category C342: 918 products`, 60 unique records across multiple pages, 0 failed.

**Docs**

- `categoryIds` was documented as EU-only. It is read by **EU, MENA, LATAM, UK and India** — each in that market's own id format (SFCC `C…` ids for EU/MENA/LATAM, numeric ids for UK, category slugs for India). US and SEA ignore it; use category URLs there.
- The `onlyNewProducts` notes gave the wrong reason for MENA's exclusion. A usable mechanism does now exist (the search catalog's `new` category, ~281 products); it simply is not wired up.

**Follow-up available:** that `new` category means MENA can be added to `onlyNewProducts` — not done here to keep this a pure bug fix. In the meantime it can be crawled directly via `categoryIds`.

#### 19/08/2026 — v2.6 Incremental refresh: only fetch recently-added products

**Feat: `onlyNewProducts` — skip the catalogue, fetch what's new (19 of 28 markets)**

Re-scraping a whole storefront to catch a handful of changes is slow and expensive. Setting `onlyNewProducts` restricts every category listing to recently-added products: on the US catalogue that is 1,201 products in **~21 listing requests**, against 10,784 products / ~180 requests for a full sweep (measured 2026-08-19).

What "new" means depends on what each storefront's API can actually do, and the run logs which of the two applies:

| Mechanism | Markets | Result |
|---|---|---|
| **Filter** | US/CA + EU (FR, IT, DE, ES, PL, RO, PT) | Exactly the products Sephora flags as new |
| **Sort** | SEA (all 10) + IN | The most recently published, capped by `newProductsPageLimit` |

- **US/CA** send `ref=filters[isNew]=true` on the v2 catalog endpoint (skincare 2,971 → 383).
- **EU** sends `refine_5=c_tagsEnabled=<value>` on SFCC `product_search`. All seven storefronts mark new products with the same internal tag (`tag6`, established by correlating `c_productFlags[].name` against each market's localised label), but they do not accept the same refinement value: **DE accepts only the raw id `tag6enable`** (its labels return 0), **ES/IT/PL/PT only the localised label** (the raw id returns 0), FR/RO either. The value is therefore a per-market constant (`EU_NEW_TAG`) resolved locally, with no discovery round-trip. Verified live on C342 (baseline → filtered): fr 886→113, it 778→101, de 776→91, es 704→107, pl 763→99, pt 704→107, ro 844→120.
- **SEA and IN cannot filter, only sort**, so they use their storefront's own newest-first ordering — `?sort=published_at` on SEA's category listing and `sort_on=latest` on IN's Fynd endpoint. Because a sort returns the whole category merely reordered, both are bounded by the new **`newProductsPageLimit`** input (default 3 ≈ 70-110 products per category).
- **MENA and UK are refused, not silently widened.** `NEW_PRODUCTS_SUPPORT` in `src/markets/__init__.py` is the single source of truth; markets absent from it are **skipped with an ERROR** rather than returning their full catalogue at the customer's cost. UK exposes no newness anywhere — filters are `price`/`brand`/`badge`/`discount_level` (badge = Bundle/Free Gifts/Savings), sorts are rating/price/discount (checked on a 319-result query). MENA advertises a `c_tag6enable` "New in" refinement (927 of 15,731) and a `new-arrival` sort id, but `refine=`, `refine_1=`, bare params and `sort=` all fail to apply it.
- **The filter is carried onto every page** of a paginated crawl, and sort markets carry their page cap the same way — dropping either after page 1 would silently widen the run back to the whole category.

**Feat: `advancedFilter` — raw refinement escape hatch (US/CA)**

- Sends any US refinement expression as `ref`, e.g. `filters[shoppingPreferences]=koreanBeauty`. Valid expressions are discoverable in the `refinements` block of any category response.
- A `ref=` already present on a start URL is honoured too and wins over both inputs; `advancedFilter` wins over `onlyNewProducts`. Whenever a lower-precedence source is overridden it is logged, so a filter never vanishes silently.

**Both failure modes are silent, and in opposite directions**

- US with the wrong spelling returns the **full** set: the expression must be sent as `ref=filters[isNew]=true`; a bare `filters[isNew]=true` is accepted and then ignored, with no error. The builder and its tests pin the working form.
- EU with a wrong tag value returns **`total=0`**, also with no error. The EU search handler therefore warns when a filtered page comes back empty, so a stale tag cannot masquerade as "no new products".
- Support was decided per market by applying each mechanism live and watching the count move, never from metadata — a category that does not advertise `c_tagsEnabled` in its `refinements` can still be filtered by it (RO C342: unadvertised, still 844→120).

**Unchanged wire for existing runs**

- With no filter set, the US category URL is byte-identical to v2.5 — the frozen v1.x wire is preserved, and a regression test pins it.
- Tests: `tests/test_only_new_products.py` (15 cases) covers the support matrix, the page cap, precedence, request building, the exact upstream URL per market, pagination carry-through, and the frozen unfiltered wire. Sample input at `test-new-products.json`.
- **Verified live** (2026-08-19, residential proxy): a six-category US incremental run produced 46 records / 46 unique / 0 failed / no URL leaks, and filtered vs unfiltered `/shop/skincare` return different product sets. EU on C342: `eu-fr` 8 records and `eu-de` 13 records (EUR, no URL leaks), exercising both the localised-label and raw-id forms. SEA and IN with `newProductsPageLimit=1`: `sea-nz` 21 records (NZD) and `in` 9 records (INR), both logging the page cap and stopping. MENA and UK URLs produce **0 records** and a clear per-market error.

**Known issue found during this work, not fixed here**

- **MENA category listing is broken, independently of this feature.** The `categoryId` parameter is ignored entirely — a valid `C342` returns 15,731 products (the whole catalogue) where the storefront shows 919 — and `_parse_url` rejects both live MENA category URL shapes (`/shop/{cat}/{sub}/{ID}` and the `/c/` form). Every MENA category crawl therefore pulls the full catalogue. The correct parameter is still unknown; the AE app's Retrofit interface is the place to look.

#### 18/08/2026 — v2.5.3 US `/beauty/` editorial pages supported (bug fix)

**Fix: `www.sephora.com/beauty/{slug}` URLs were rejected as "Invalid / Unsupported url"**

- **Root cause:** the US market recognised exactly two start-URL shapes — `/shop/{slug}` (category) and `/product/…-P{id}` (detail). `/beauty/{slug}` matched neither, so the reported URL `https://www.sephora.com/beauty/new-skin-care-products?icid2=action_link_new_skincare` was dropped before enqueue and the run exited with nothing to scrape.
- **Why it isn't just a wider regex:** `/beauty/` pages are **not** catalog categories. The storefront's own soft links tag them `"type": "PageContent"` (vs `"PageCategory"` for `/shop/`), and the v2 categories endpoint answers their slugs with `404 There is no matching category for :/categories/new-skin-care-products/seo`. They are Contentful-backed curated grids served by a **separate v1 content endpoint**, recovered from the Retrofit interface `wu0.b#getBeautyContentPage` in `com.sephora` 26.6 (`@GET("content/beauty/{slug}")` + `@Query("ch")` + `@Query("loc")`, no `countryCode`) and confirmed live.
- **New `content` request kind:** `src/masker.py` gains a `content` kind, `src/auth/us_urls.py` builds the real URL with the app's exact param names and order, and a new `us-content` handler reads `data.layout.skuList` and enqueues one detail request per product. The other four layout types (buying guide, customer services, directory, open) hold editorial copy, not products — they are skipped with a market-tagged log line rather than failing the run.
- **Known gap — this emits the page's editorial list, which is probably not what a visitor sees.** `layout.skuList` (9 items for `new-skin-care-products`) is an SSR/editorial list. The page's visible grid is a separate Constructor.io-ranked feed at `https://www.sephora.com/gway/v1/dotcom-sys/browse/{layout.ctorPodId}` — same collection (`display_name` "New Skincare", pod id supplied by the page itself, and the only product call the page makes) but a substantially different set: **only 1 of the 9** pinned ids appears in the feed's first 100, and the feed is broader (lip balms, value sets, a hand sanitizer). It is not "the rest of the page" — the two lists barely intersect. **Verified 2026-08-19 by rendering the page**: the visible grid is the feed (first tile `P480529` Touchland Power Mist Hand Sanitizer, then Supergoop! PLAY, Vacation Mini Classic Whip, rhode Glazing Milk, rhode Highlight Milk — the feed's order, none of them in `skuList`), so what this actor emits for a `/beauty/` URL is 9 products a visitor does not see there.
- **Why it is not wired up yet (investigated 2026-08-18).** It is reachable — earlier notes here blamed Kasada, then Akamai; both were wrong, and the TCP drops were self-inflicted by probing with the app User-Agent over a Chrome JA3. The feed returns HTTP 200 to plain `curl_cffi` with the market's ordinary `Seph-Access-Token`. What stops it being a drop-in win: it caps at 200 results per sweep at any page size and reorders between requests (~160-180 unique of 383; 216 across two sweeps), and it returns nothing for the bestsellers pods on these parameters. So switching to it is a product decision — a different, larger, non-deterministic set — not a bug fix. For a guaranteed-complete assortment, use the matching `/shop/…` category URL, which paginates fully.
- **Canada:** `sephora.com/ca/en/beauty/…` and `/ca/fr/…` resolve through the same path and produce CAD records with `/ca/en` canonical URLs.
- **Output hygiene:** `source.crawlUrl` on every record stays the public `/beauty/` page the customer passed (tracking params intact); the v1 content endpoint never reaches the dataset or the logs.
- **Scope note:** only `/beauty/{slug}` is claimed. A multi-segment `/beauty/{a}/{b}` (e.g. `/beauty/guides/how-to-apply-blush`) is still reported unsupported — the app declares the slug as an `@Path`, which percent-encodes slashes, so that is not a shape the upstream API can serve.
- **Tests:** URL classification (incl. the reported URL verbatim, both CA locales, trailing slash, multi-segment rejection), `/shop/` and PDP regression cases, synthetic-request routing, the exact upstream URL, and handler behaviour across grid / non-grid / empty / 404 pages are locked in (`tests/test_us_content_pages.py`, 15 cases). The e2e manifest schema gained a `content` URL shape, wired so the operator runner picks it up automatically.
- **Verified live** (2026-08-18, US residential proxy): the reported URL produced **9 records, 0 failed requests**, every `crawlUrl` the public `/beauty/` page and every `canonicalUrl` a public `/product/` page. CA (`/ca/en/beauty/…`, CA proxy): **3 records, CAD, 0 failed**.

#### 11/08/2026 — v2.5.2 UK slug PDP URLs resolved (bug fix)

**Fix: `www.sephora.co.uk/p/{slug}` product URLs failed with G-2 "could not be found" (HTTP 400)**

- **Resolution hop:** a `/p/{slug}` URL is now detected by token shape (`P\d{4,}` = id, else slug) and routed through a new `resolve` request that queries `/catalog/v1/products?query={slug-as-text}` (the only search param the gateway honors — `q`/`term`/`keyword` return a static suggestion set) and matches the slug against each result's normalized `{brand} {name}` (exact / prefix / ≥0.6 token overlap). The winning `P…` id is enqueued as a normal detail request. No confident match → recorded not-found, never a wrong product.
- **Output fix (`source.canonicalUrl`):** the record now carries the real public `/p/{slug}` page the request started from, not a `/p/{productId}` deeplink or the internal fetch endpoint.
- **Proxy:** UK reads only clear Akamai from a GB residential IP (datacenter and reputation-flagged residential exits get 403). A market may now declare `preferred_proxy_country`; when every active market agrees and the customer left the country unset, the run defaults to it (UK → GB). Mixed runs and an explicit `proxy.apifyProxyCountry` are untouched.
- **Latency:** the Akamai-blocked `/login/v1/login` burst is no longer re-run on every catalog request — after one failed attempt it backs off for 5 min (reads don't need the session), saving ~6s + three dead round-trips per call.
- **Tests:** slug-vs-id parsing, slug→id matching (incl. the reported URL), request routing, `canonicalUrl` threading, and the proxy-country pin are locked in (`tests/test_uk_resolution.py`, 21 cases).
- **Verified live** (2026-08-11, residential proxy, country auto-pinned to GB): the reported URL `https://www.sephora.co.uk/p/rhode-pocket-blush-buildable-hydrating-cream-blush` resolved to `P1000209341` and produced a valid record (title, 12 variants, 5 images, GBP, canonical = the public slug URL) — 2 requests, 0 failed.

#### 07/08/2026 — v2.5.1 MENA storefront URLs accepted (bug fix)

**Fix: `www.sephora.me` product URLs were rejected as "Invalid / Unsupported url"**

- **Root cause:** the six MENA markets were registered under per-country hostnames (`www.sephora.ae`, `www.sephora.sa`, …) inferred during Phase 1 recon and never verified live. The real consumer storefront is the single host `www.sephora.me` with a `/{cc}-{lang}/` path prefix — the AE app's own declared production web base. Unregistered hostnames fall back to the `us` market, which then rejected every MENA URL and tagged the error `[us]`.
- **Routing:** `detect_market` now resolves `www.sephora.me` (and the apex) by path prefix — `/ae-en/`, `/sa-en/`, … → `mena-ae`, `mena-sa`, …. Bare (`/ae/`) and Arabic (`/ae-ar/`) prefixes resolve to the same market. An absent or unknown prefix defaults to AE rather than falling through to `us`. The legacy per-country hostnames stay registered.
- **URL parsing:** the live grammar joins the product id with a slash (`/p/{slug}/P{id}`); the parser only accepted the hyphen form (`{slug}-P{id}`). Both now parse. `?productVariantId=` is ignored — every variant is already emitted.
- **Output fix (`source.canonicalUrl`):** MENA records carried a wrong host and a doubled slash (`https://www.sephora.ae/ae//mega-mix-palette/P10061352`). They now emit the real page URL, e.g. `https://www.sephora.me/ae-en/p/mega-mix-palette/P10061352`. **This changes existing MENA output** — consumers keying on the old value must update.
- **Diagnosability:** an unrecognized hostname now logs a warning naming the host and the fallback market, once per host, instead of failing silently under the wrong market's tag.
- **All families:** apex hostnames now resolve both directions. Previously only `www.x` → `x` was tried, so e.g. `https://sephora.fr/...` fell through to `us`; it now resolves to `eu-fr`. Same bug class as the MENA fault above.
- **Tests:** the five reported URLs are locked in as routing + parse regressions; all six MENA path prefixes are covered; the `canonicalUrl` assertion is now exact rather than a prefix check (the prefix check is what let the malformed URL ship). README market table and the AE e2e manifest updated to the real grammar.
- **Verified live** (2026-08-07, residential proxy, country-pinned): all 6 MENA markets PASS — 28 records, 0 failed requests, every `canonicalUrl` on the real host with the correct locale prefix. Run: `tests/e2e/runs/mena-20260807T141315Z.jsonl`.

**Fix: live e2e gate was red for every market since v2.5 (unrelated pre-existing bug)**

- `tests/e2e/scripts/assert_dataset.py` kept its own hardcoded copy of the record's top-level key list. v2.5 added `size` to `dataset_schema.json`, all seven market transforms, and `test_dataset_schema_compat.py` — but not to this copy, so the script rejected every record with `unexpected keys: ['size']`. Any Layer-3 run since 30/06/2026 would have failed on all markets, not just MENA.
- The script now derives the key set from `.actor/dataset_schema.json` instead of duplicating it, so the schema stays the single source of truth and this drift cannot recur.

#### 30/06/2026 — v2.5 Per-shade undertone/finish + size (US)

**Feat: shade undertone/finish + per-SKU size**

- **NEW** `options[].values[].description` — per-shade undertone/finish text, mapped from the US API's `variationDesc` (e.g. "light, neutral peach", "with neutral undertones", "medium warm 4"). Colour shades only; blank for size/scent variations.
- **NEW** top-level `size` — headline size/volume of the default SKU (US: `currentSku.size`, e.g. "1 oz / 30 mL"). Non-US markets emit `""` (their SFCC/GraphQL APIs don't expose a clean product-level size).
- **NEW** `variants[].size` — per-SKU size/volume (US-only; ancillary minis differ from the headline size).
- **Scope:** undertone/finish + size are US/CA only — the international storefronts (SFCC EU/LATAM/MENA, GraphQL UK/SEA/IN) carry no per-shade description, so `description` is absent and `size` is `""` there.
- Schema docs updated: `dataset_schema.json`, `test_dataset_schema_compat.py`, and `API_DOCUMENTATION.md`.

#### 09/05/2026 — v2.4 Phase 4 IN ship + Phase 3 UK reconciliation

**Feat: Phase 4 — IN (Sephora India on Fynd Platform)**

- Added `in` family — single market on `api.sephora.cn`'s Reliance-owned cousin `api.fynd.com`. Static Bearer auth (Base64 of compile-time `applicationId:applicationToken` recovered from `com.jio.sephora` 1.0.34). Path grammar `/service/application/catalog/v1.0/products/{slug}/`. Slug-based product addressing, `page_no`/`page_size` pagination. Module name `in_market` (avoids Python `in` keyword); family slug `"in"` in `_FAMILY` map.
- `www.sephora.in`, `sephora.in` registered. Currency INR, locale `en-IN`.
- `api.fynd.com` added to no-url-leak banlist.

**Fix: Phase 3 — UK auth + DTO reconciliation**

- Reconciled UK code with deep static-analysis findings from `co.uk.sephora` 19.7.1's 38 MB Hermes JS bundle. 8 substantive corrections: token POST path was `/login/v1/login` (not `/token`); auth model was cookie-session, not Bearer; pagination uses `offset`/`limit` (not `start`/`count`); search filter format `category:(c{id})`; detail query needs `idType=default`; image shape is `{normal, thumbnail}`; response shape is `{product: {...}}` envelope; ProductDetail field names tightened (name, brand, description, ingredients, tips, price, formerPrice, status, stockLevel, variants\[], etc.).

#### 08/05/2026 — v2.3 Phase 2 LATAM + Phase 3 UK initial ship

**Feat: Phase 2 — LATAM (Mexico + Brazil)**

- Added `latam-mx` (es-MX, MXN) and `latam-br` (pt-BR, BRL) on `apps.sephora.eu` with `/latam/` URL prefix and OCAPI v23\_2 (newer than EU's v18\_3). Single SODA key + DW client ID across both countries; per-country UA via `LATAM_VERSIONS_BY_COUNTRY`. Recovered from `latam.sephora.sephoramx` 26.4.3 + `latam.sephora.sephorabr` 26.4.2.

**Feat: Phase 3 — UK (initial scaffold)**

- Added `uk` family — single market on `apps.sephora.eu` with a flatter REST surface (`/catalog/v1/`, `/login/v1/`, `/content/v1/`) than EU's OCAPI commerceCloud paths. Recovered from `co.uk.sephora` 19.7.1 (React Native + Hermes + Akamai BMP). `x-consumer-id: AOC-ANDROID-UK` (vs EU's `AOC-ANDROID-EME`).

#### 08/05/2026 — v2.2 Phase 1 MENA ship

**Feat: Phase 1 — MENA (6 countries on SCAPI/Apigee)**

- Added `mena-ae`, `mena-sa`, `mena-bh`, `mena-om`, `mena-kw`, `mena-qa` on `apps.sephora.me` (Apigee gateway + SCAPI). Per-country routing via lowercase `apikey` header carrying per-country Apigee key (AE + SA have their own; BH/OM/KW/QA fall back to AE's, mirroring how the App Store routes them). Six countries served by one backend. Recovered from `ae.sephora.sephorauae` 3.45.0-5386 + `sa.sephora.sephorasa` 3.45.0-5385.

#### 18/04/2026 — v2.1 Crawlee migration + URL masking

- Unified Crawlee `HttpCrawler` across US + EU + SEA with synthetic-URL masking, global circuit breaker, per-market pre-flight, declarative retry policy, serialized EU guest-token refresh, EU product-ID case normalization, `maxProducts` cap honored in EU pagination; CZ, GR, and PPE instrumentation removed, PT now served via the Spain backend with `pt-PT` locale.

#### 17/04/2026 — v2.0 Global Merge

**Breaking / additive schema changes:**

- **NEW** `market` top-level field stamped onto every dataset item (values: `us`, `eu-fr`, `eu-it`, `eu-de`, `eu-es`, `eu-pl`, `eu-cz`, `eu-gr`, `eu-ro`, `eu-pt`, `sea-nz`, `sea-au`, `sea-sg`, `sea-my`, `sea-th`, `sea-id`, `sea-ph`, `sea-hk`, `sea-tw`, `sea-bn`).
- **NEW** `variants[].wishlisted` boolean field (SEA-only; analog of US `lovesCount`).

**Feat:**

- Multi-market support — US + Canada + 9 EU countries + 10 APAC countries in one actor.
- Auto-detect market from `startUrls` hostname; no input changes required for existing US users.
- Optional `market` and `locale` override inputs for edge cases (bare IDs, region mismatches).
- Per-market observability counters persisted to the default KV store as `run-summary`.

**Architecture:**

- Refactored `src/main.py` into a lightweight dispatcher. Per-region logic lives in `src/markets/{us,eu,sea}.py`.
- EU and SEA ports use `curl_cffi` for upstream compatibility. US keeps Crawlee's `HttpCrawler` with the existing session-pool auth flow unchanged.
- Strict market-module isolation — per-region auth state cannot cross-contaminate.

**Compatibility:**

- Existing US run configs work unchanged. Pre-v2.0 output fields are preserved; only the additive `market` key is new.
- Standalone `autofacts/sephora-eu-scraper` and `autofacts/sephora-nz-scraper` listings remain live but are pinned to their current version. New bug fixes and features land in this repo only.

**Deps:**

- Added: `curl_cffi[async] >= 0.7`, `lxml >= 5.0`.

#### 10/03/2026

- Feat: add `sentiments` field to product output, merging sentiment summary and sentiment items

#### 04/12/2025

- Chore: update readme file

#### 29/11/2025

- Fix: improved session management

#### 28/11/2025

- Fix: enhanced stability with firewall blocking
- Refactor: refactor with python

#### 25/10/2025

- Migrate from apify to crawlee

#### 17/11/2024

- Fix bug of product image url

#### 28/05/2023

- Fix bug of setup proxy
- Fix bug of product list pagination