jdmuscle-scraper

Scrapes products (JDM/aftermarket auto parts, from own-brand shift knobs to third-party wheels, tires, and body kits) and a real brand directory from JDMuscleUSA, a multi-brand JDM performance parts retailer — via the site's own real Shopify product data. No login required.

Contents

• Key features
• Output
• Input
• Input examples
• Incremental (delta) mode
• FAQ

🔑 Key features

• Two entity types, one Actor. Set entityType to products or brands.
• Scope products to specific brands via the searchable brands multi-select — run entityType=brands once to see what's available, then rerun entityType=products with just the brands you want, paginated to each brand's real full catalog (not capped at 250).
• The full catalog: 48,938+ products across 373 real brands (RAYS, WeatherTech, TOM'S, Mickey Thompson, and more, alongside JDMuscle's own house brand) — one of the largest catalogs scraped by this developer.
• Real vehicle fitment, parsed from the site's own tags where it's unambiguous (e.g. "2015-2017 Subaru WRX STI") — terser internal shorthand tags are kept as raw reference data rather than guessed at (see FAQ).
• A real, live brand directory with live product counts — entityType=brands is a fresh crawl every run, not an estimate.
• Delta mode built in for products — every run classifies each item as new, changed, unchanged, or delisted against a persisted baseline.
• maxItems defaults to 50 — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. Clear it (null) for a full crawl.

📋 Output

One dataset item per product or brand, depending on entityType — see .actor/dataset_schema.json for the full field list.

Example product record:

{
  "source": "jdmuscle",
  "entity_type": "products",
  "external_id": "3730310529077",
  "url": "https://jdmuscleusa.com/products/tanso-carbon-fiber-trunk-lid-covers-2015-wrx-sti",
  "name": "JDMuscle [15-21 WRX/STI] Pre-preg Carbon Fiber Trunk Lid Cover w/ Gloss Finish",
  "brand": "JDMuscle",
  "category": "Exterior Garnishes",
  "tags": ["2015-2017 Subaru WRX", "2018-2021 Subaru WRX STI", "subaru", "WRX 15+", "..."],
  "fitment": [
    { "year_start": 2015, "year_end": 2017, "vehicle": "Subaru WRX" },
    { "year_start": 2018, "year_end": 2021, "vehicle": "Subaru WRX STI" }
  ],
  "option_names": [],
  "image_urls": ["https://cdn.shopify.com/s/files/..."],
  "price_min": 189.0,
  "price_max": 189.0,
  "currency": "USD",
  "variant_count": 1,
  "variants": [
    {
      "sku": "JDM-WRX15-LG#CF",
      "title": "Default Title",
      "option1": "Default Title", "option2": null, "option3": null,
      "price": 189.0,
      "currency": "USD",
      "available": true
    }
  ],
  "change_type": "new"
}

Example brand record:

{
  "source": "jdmuscle",
  "entity_type": "brands",
  "external_id": "rays",
  "url": "https://jdmuscleusa.com/collections/rays",
  "name": "Rays",
  "tracked_product_count": 187
}

🔧 Input

Field	Type	Default	Description
entityType	string	products	products or brands.
startUrls	array	—	Specific product URLs to scrape directly instead of the full catalog. products only.
brands	array	—	Restrict to specific brands (searchable multi-select) instead of the full catalog. products only; mutually exclusive with startUrls.
maxItems	integer	50	Stop after pushing this many items. Set to null for a full crawl.
mode	string	auto	auto / full / incremental — see Incremental mode.
impersonate	string	chrome	curl_cffi TLS-impersonation target. products only.

📥 Input examples

Default preview (50 products):

{ "entityType": "products" }

Full product catalog:

{ "entityType": "products", "maxItems": null }

Full brand directory:

{ "entityType": "brands", "maxItems": null }

Only specific brands' full catalogs:

{ "entityType": "products", "brands": ["rays", "weathertech"], "maxItems": null }

🔁 Incremental (delta) mode

auto mode does a full scan the first time it runs for entityType=products, then only pushes new/changed items on later runs (price_min/price_max/variant_count are the watched fields). Only an uncapped, unscoped run (no maxItems limit reached, no startUrls/brands filter, no item errors) can detect delistings or update the baseline. entityType=brands has no delta mode — it's a fresh live directory crawl every run.

❓ FAQ

Why does fitment only cover some of the vehicle-specific tags? A real, deliberate choice based on live evidence: this catalog's own tags mix two shapes for the same concept — a fully-qualified string like "2015-2017 Subaru WRX STI", and terser internal shorthand like "STI 15+" or "WRX 08-14" that names only a trim/chassis code, not the full make. Guessing a make from a bare code would be an unverified inference, so only the fully-qualified shape is parsed into fitment; every tag (including the shorthand) is always available in the raw tags field.

Why is tracked_product_count capped around 250 for a brand? It's read from Shopify's own /collections/{brand}/products.json?limit=250 endpoint, which caps at 250 products per page. A brand with more than 250 listed products would be undercounted — plausible given the catalog's real scale, not exhaustively verified for every one of the 373 brands.

Does entityType=brands cover every brand on the site? 373 of them — the ones shown as a real tile on the site's own /pages/brands directory. A further 117 brands are only linked via a search-style URL (/collections/vendors?q=...) rather than a dedicated collection page, and aren't covered by this entity type.

Why does the brands list not include every brand on the site, and could it go stale? It's built from the same 373 brands entityType=brands finds (see the question above) — the same 117-brand gap applies. It was captured from a real full run; if the site adds a new brand later, it won't appear in the dropdown until the list is regenerated (there's no automated sync for this yet, unlike a couple of sibling actors' category dropdowns).

Why does brand-scoped products use different pagination than the full catalog? Both /products.json (full catalog) and /collections/{brand}/products.json (brand-scoped) support a since_id pagination parameter — but real testing found the site currently stopped honoring it on both, silently repeating the same page forever instead of erroring. For the full catalog, the fix was switching to the site's own sitemap. For a single brand, page-based pagination (?page=N) is a simpler fix that works safely at that smaller scale — Shopify's own page-based pagination hard-caps at 25,000 items, which no single brand's catalog comes anywhere close to, even though it would silently truncate the full 48,938-item catalog.

Does this need a proxy? No anti-bot layer was found on the main product catalog — a real concurrency burst against /products.json came back 100% clean. The brand-collection endpoint used by entityType=brands does have a real, hard rate limit at full scale, fixed with sequential fetching and a fixed delay between requests (same fix already proven on a sibling actor's identical endpoint shape) — no proxy involved either way.