Shopify Store Assortment Comparison

Compares 2-3 public Shopify stores using their public /products.json feeds and returns vendor and category assortment, price bands, overlapping SKUs, and change-friendly JSON/CSV. Built by PAL (Practical AI Lab).

No Amazon or Google access, no proxies, no logins, no private data. Only the public product feed and robots.txt of the stores you list are requested.

Safety and politeness

• robots.txt is honored. If it disallows /products.json, that store is skipped. If robots.txt cannot be read (anything other than 200, 404 or 410), the Actor fails closed for that store. A Crawl-delay (capped at 10 s) is respected.
• SSRF protection. Only https on the default port, no credentials, no IP-literal or internal-looking host names. Every host is resolved first, and the request is refused if any answer is a non-public address. The connection is then pinned to the validated IP.
• Redirects are followed manually (max 3), re-validated at every hop, and limited to the same site (a www. toggle is allowed). Use a store's final domain if it redirects elsewhere.
• Bounded: at most 2,500 products per store, 40 requests per store, 6 MB per response, uncompressed responses only, a configurable delay (default 1 s) and timeout. Stores are read one after another.
• Only fields needed for comparison are kept (no descriptions, images or customer data).

Output

Dataset rows by recordType:

recordType	Content
summary	Stores, totals, overlapping SKUs, SKUs in every store, shared vendors, currency note.
store_summary	Products, variants, vendors, categories, min/median/max price, on-sale and unavailable variants.
price_band	Products per price band per store.
vendor_assortment, category_assortment	Products, variants and median price per store for each vendor or product type.
sku_overlap	SKUs found in 2+ stores (normalized: trimmed, case-insensitive), cheapest store and price delta %.
product_variant	One row per variant with a stable rowKey (store:variantId) and a rowHash.
change	Only when previousSnapshot is given: added, removed, price-changed and availability-changed variants.
store_error	A store that could not be read (never charged).

Key-value store: products.csv, sku_overlap.csv, vendor_assortment.csv, category_assortment.csv, SNAPSHOT (compact rowKey -> [hash, price, available]) and OUTPUT.

To track changes, pass the previous run's SNAPSHOT as previousSnapshot in the next run.

Input

storeUrls (2-3), maxProductsPerStore (default 500), priceBands (default [25, 50, 100, 200, 500]), requestDelaySeconds, timeoutSeconds, includeProductRows, previousSnapshot.

Limits

• products.json does not state a currency. Prices stay in each store's own currency, so compare price bands across stores only when the currencies match.
• Some stores disable or restrict the feed. These are reported as store_error. If fewer than 2 stores can be read, the run fails without a charge.
• SKU overlap needs stores to fill in SKUs.

Pricing event

Pricing uses pay-per-event. The billing event name is shopify-assortment-compare-report, charged once per successful comparison. Failed runs and store_error rows are not charged.

Local tests

python3 -m unittest discover -s tests

All tests are offline (fake transport and resolver).