# Changelog of Zid Store Scraper — Products, Prices & Stock (`al_mansouri/zid-store-scraper`) Actor - **URL**: https://apify.com/al\_mansouri/zid-store-scraper/changelog.md - **Full Actor documentation**: https://apify.com/al\_mansouri/zid-store-scraper.md ## Changelog All notable changes to the Zid Store Scraper are recorded here. The record shape, the error codes, and the limits quoted in the README are a published contract. Adding, renaming, or restatusing any of them is a breaking change and must reach the README, the dataset schema, and this file together. ### \[Unreleased] #### Changed - **`stoppedBy` and `discoveryMethod` now publish the Salla Actor's spellings.** `max_items` → `maxItems`, `timeout` → `timeLimit`, `category_crawl` → `category-crawl`, `direct_urls` → `direct-url`. The field *names* already matched, so the two datasets merged cleanly and the divergence was invisible — until you filtered the merged sheet. `stoppedBy = maxItems` returned the Salla rows and silently dropped every Zid one, which looks like a clean result and is not. A test now pins the literal strings rather than only checking them against this Actor's own schema, because checking a schema against the code it was generated from cannot catch this. Breaking for anyone reading those two values; this Actor is not yet published. #### Added - **Listing assets.** `assets/output-preview.png` shows real rows from four live Zid stores — Arabic names, list price beside sale price, live stock, SKUs, option counts, and the one product page that timed out, shown as the `failed` row the dataset actually carries rather than edited out. It is a browser screenshot, because Arabic is cursive and needs real shaping; drawing the glyphs directly would produce exactly the mojibake this Actor claims to avoid. `scripts/capture_output_rows.py` and `scripts/render_output_preview.py` rebuild it from a live run, and `scripts/publish_readme_assets.py` uploads it to a public key-value store — Apify publishes a README's text and not the files beside it, so a repository path renders broken on the listing. - **Store icon**, drawn from code by `scripts/prepare_brand_assets.py` so a redraw is a reviewable diff. Same geometry as the Salla Actor's mark and a violet price tick against its emerald: one product with two front doors should read as siblings in a Store grid, and colour is the only difference that survives 32px. - **`language` does something.** It was accepted and then ignored entirely — an input that changed nothing. This Actor reads whatever the storefront serves, so it now does what the Salla Actor does: reads the languages the shop declares about itself (`` and `hreflang` alternates) and puts a `LANGUAGE_UNAVAILABLE` warning on the store row when the requested one is not among them. It never translates and never switches the shop. A store that declares no language is left alone — silence is not a denial, and warning on it would fire everywhere and mean nothing. - `maxItems` now applies **per store**, matching the Salla Actor. It was a shared run budget, and the first platform run showed what that cost: given four stores and `maxItems: 4`, the first store took all four and the other three published nothing while reporting `failed`. Nothing had failed — the budget was gone. A store truncated by the limit now says so with an `ITEM_LIMIT_REACHED` warning, the same code and wording Salla uses. - Products already published by an earlier container of a resurrected run count against that store's limit. Counting only new rows would let a run that died after three products publish a full allowance again, billing the customer for more than they asked for. - First implementation. Reads public Zid storefronts and emits two record types: `product-scraped` (one row per product) and `store-analyzed` (one row per store per run). - **Custom-domain store detection.** About 59% of Saudi Zid stores run on the merchant's own `.com` domain, so detection reads page content — several independent platform markers — rather than the hostname. A page that merely mentions Zid is refused with `NOT_A_ZID_STORE`. - **List price and sale price are reported separately.** The storefront's embedded product object carries both; its JSON-LD carries only the effective price, so JSON-LD is the fallback and never the source of `price`. - **Availability is reconciled, never guessed.** The storefront's own state and its structured data are read independently. When they disagree, the row says `unknown` and carries a warning. - Sitemap-first discovery with a category-crawl fallback, chosen on the response *body*: one measured store answers `sitemap_products.xml` with HTTP 400 and an HTML error page. - Arabic throughout — product names, categories, and variant options — with sitemap IRIs percent-encoded before fetching, and a CSV export that opens in Excel with a UTF-8 BOM. - `scripts/probe_live_store.py`, the measuring instrument for the published field-accuracy scorecard, and `scripts/scorecard_stores.txt`, the store list it was run against. - **Published field-accuracy scorecard**, measured on 38 live Zid stores and 152 products: name, price, currency, availability, SKU, images and variants at 100%, categories at 96.7%. 28 of the 38 stores run on custom domains and every one was detected. Numbers and method in the README; full account in `docs/ACTOR_6_MARKET_EVIDENCE.md` §8. #### Notes - **Free.** `product-scraped` and `store-analyzed` are defined but not priced, and only Apify's platform usage applies. They exist so a price later would be a price change rather than a redesign; none is planned. - Field names are shared with the Salla Store Scraper on purpose, so the two datasets concatenate into one sheet. `platform` records which store each row came from. - The rationale, the plan override, and the pre-build survey are in `docs/ACTOR_6_MARKET_EVIDENCE.md`.