salePrice no longer misreports a third of rows. Salla's catalogue uses 0 as its
no-discount sentinel rather than null, and its price field is the list price on some rows
and what the shopper actually pays on others — price=290 sale_price=145 regular_price=290 and
price=749 sale_price=749 regular_price=1099 are both real rows from live stores. Passing those
numbers through published salePrice: 0.0 on products that cost money, and salePrice equal to
price on products that were not discounted at all. Measured across 480 products on 8 live
stores: 67 rows (14%) advertised as free, a further 91 (19%) carrying a discount to their own
price, and every one of them status: "verified" with no warning.
price is now read from regular_price and a sale is claimed only when sale_price is a real
number below it, which is what the dataset schema has always labelled the two fields. The same
480 products now produce no zero sale prices and no self-discounts, and 117 genuine reductions
are visible where 26 were before — the regular_price the old mapping discarded was the one
that made a markdown legible.
The store cross-check could not have caught this: it accepts a page price matching either
published number, and the number it matched was the one that was right.
robots.txt is now honoured for every address, not only the one submitted. The start URL
was checked and everything discovered afterwards was not, so a store that welcomed crawlers to
its front page and closed its product pages had those pages read anyway — by sitemap discovery
and by the price cross-check alike. Both now consult the policy before fetching. Skipped
product addresses raise ROBOTS_LIMITED_DISCOVERY, spelled as the Zid Actor spells it, so the
catalogue is never quietly short. A cross-check skipped this way is silent: not checking is not
a claim, and pricesCrossChecked already reports the smaller number.