- A store that refused us is no longer reported as a store with no catalog. Any non-200 on
/products.json produced the same unavailable marker, so a rate-limit on a big storefront
read as "this store does not publish a catalog" — a confident wrong answer. The marker now
carries fetch_error: null means the store genuinely publishes no catalog, HTTP 429 (or
similar) means we were refused and the run is worth repeating.
- Blocked stores are retried through the residential pool automatically, the way
shopify-store-intel already did it. No proxy toggle to find; only refused domains use
residential bandwidth.
- A refusal costs seconds instead of minutes. Neither pool now sits through a long backoff on
refusals that were never going to clear — a comparable run spent 262s to return nothing.
- A connection error degrades instead of failing the run, so it no longer counts against the
Store maintenance badge.
- Runs no longer time out delivering their own results (#132). Delivery was one
push_data
plus one charge_result round-trip per record — ~2.6s/item, which capped a run at roughly 90
rows and timed out 1 of 3 customer runs at the inherited 300s platform timeout. Rows now go
out in chunks of 50 with one charge per chunk. The actor's default run timeout was also raised
from 300s to 3600s.
- A truncated run says so. When the PPE budget cap ends a run early, a free
truncated marker
is emitted with the number of undelivered changes. Previously the snapshot heartbeat announced
every change the diff found (e.g. 248) while the dataset held only what fit (e.g. 82), and
nothing told the customer the difference.
Push still precedes charge, so a customer is never billed for a row they did not receive. The
trade-off of batching is the mirror image: if the budget cap lands mid-chunk, up to 49 rows were
delivered free. That error is bounded and falls in the customer's favour.
- Monitor Shopify stores for catalog changes via the public
/products.json endpoint.
- Diff each store against the previous run's snapshot (persisted in a named KV store):
price increases/decreases, back-in-stock / out-of-stock flips, products added/removed.
- First run per domain captures a free baseline; subsequent runs emit a free per-store
heartbeat plus one billable record per change.
minPriceChangePercent noise filter; maxProductsPerStore cost cap; optional proxy.- PPE: charge per detected change (
dataset-item); never charge for runs with no changes.
