# JDMuscleUSA Product & Brand Scraper (`artsiom_k/jdmuscle-scraper`) Actor Scrape JDMuscleUSA.com's full JDM/aftermarket parts catalog (48,938+ products across 373 brands), with vehicle fitment, pricing, and variant detail. - **URL**: https://apify.com/artsiom\_k/jdmuscle-scraper.md - **Developed by:** [Artsiom Kunitsyn](https://apify.com/artsiom_k) (community) - **Categories:** E-commerce, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $5.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## jdmuscle-scraper Scrapes **products** (JDM/aftermarket auto parts, from own-brand shift knobs to third-party wheels, tires, and body kits) and a real **brand directory** from [JDMuscleUSA](https://jdmuscleusa.com), a multi-brand JDM performance parts retailer — via the site's own real Shopify product data. No login required. ### Contents - [Key features](#key-features) - [Output](#output) - [Input](#input) - [Input examples](#input-examples) - [Incremental (delta) mode](#incremental-delta-mode) - [FAQ](#faq) ### 🔑 Key features - **Two entity types, one Actor.** Set `entityType` to `products` or `brands`. - **Scope `products` to specific brands** via the searchable `brands` multi-select — run `entityType=brands` once to see what's available, then rerun `entityType=products` with just the brands you want, paginated to each brand's real full catalog (not capped at 250). - **The full catalog: 48,938+ products across 373 real brands** (RAYS, WeatherTech, TOM'S, Mickey Thompson, and more, alongside JDMuscle's own house brand) — one of the largest catalogs scraped by this developer. - **Real vehicle fitment**, parsed from the site's own tags where it's unambiguous (e.g. "2015-2017 Subaru WRX STI") — terser internal shorthand tags are kept as raw reference data rather than guessed at (see FAQ). - **A real, live brand directory with live product counts** — `entityType=brands` is a fresh crawl every run, not an estimate. - **Delta mode built in** for `products` — every run classifies each item as `new`, `changed`, `unchanged`, or `delisted` against a persisted baseline. - **`maxItems` defaults to 50** — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. Clear it (`null`) for a full crawl. ### 📋 Output One dataset item per product or brand, depending on `entityType` — see [`.actor/dataset_schema.json`](.actor/dataset_schema.json) for the full field list. **Example product record:** ```json { "source": "jdmuscle", "entity_type": "products", "external_id": "3730310529077", "url": "https://jdmuscleusa.com/products/tanso-carbon-fiber-trunk-lid-covers-2015-wrx-sti", "name": "JDMuscle [15-21 WRX/STI] Pre-preg Carbon Fiber Trunk Lid Cover w/ Gloss Finish", "brand": "JDMuscle", "category": "Exterior Garnishes", "tags": ["2015-2017 Subaru WRX", "2018-2021 Subaru WRX STI", "subaru", "WRX 15+", "..."], "fitment": [ { "year_start": 2015, "year_end": 2017, "vehicle": "Subaru WRX" }, { "year_start": 2018, "year_end": 2021, "vehicle": "Subaru WRX STI" } ], "option_names": [], "image_urls": ["https://cdn.shopify.com/s/files/..."], "price_min": 189.0, "price_max": 189.0, "currency": "USD", "variant_count": 1, "variants": [ { "sku": "JDM-WRX15-LG#CF", "title": "Default Title", "option1": "Default Title", "option2": null, "option3": null, "price": 189.0, "currency": "USD", "available": true } ], "change_type": "new" } ``` **Example brand record:** ```json { "source": "jdmuscle", "entity_type": "brands", "external_id": "rays", "url": "https://jdmuscleusa.com/collections/rays", "name": "Rays", "tracked_product_count": 187 } ``` ### 🔧 Input | Field | Type | Default | Description | |---|---|---|---| | `entityType` | string | `products` | `products` or `brands`. | | `startUrls` | array | — | Specific product URLs to scrape directly instead of the full catalog. `products` only. | | `brands` | array | — | Restrict to specific brands (searchable multi-select) instead of the full catalog. `products` only; mutually exclusive with `startUrls`. | | `maxItems` | integer | `50` | Stop after pushing this many items. Set to `null` for a full crawl. | | `mode` | string | `auto` | `auto` / `full` / `incremental` — see [Incremental mode](#incremental-delta-mode). | | `impersonate` | string | `chrome` | curl\_cffi TLS-impersonation target. `products` only. | ### 📥 Input examples **Default preview (50 products):** ```json { "entityType": "products" } ``` **Full product catalog:** ```json { "entityType": "products", "maxItems": null } ``` **Full brand directory:** ```json { "entityType": "brands", "maxItems": null } ``` **Only specific brands' full catalogs:** ```json { "entityType": "products", "brands": ["rays", "weathertech"], "maxItems": null } ``` ### 🔁 Incremental (delta) mode `auto` mode does a full scan the first time it runs for `entityType=products`, then only pushes new/changed items on later runs (`price_min`/`price_max`/`variant_count` are the watched fields). Only an uncapped, unscoped run (no `maxItems` limit reached, no `startUrls`/`brands` filter, no item errors) can detect delistings or update the baseline. `entityType=brands` has no delta mode — it's a fresh live directory crawl every run. ### ❓ FAQ **Why does `fitment` only cover some of the vehicle-specific tags?** A real, deliberate choice based on live evidence: this catalog's own tags mix two shapes for the same concept — a fully-qualified string like `"2015-2017 Subaru WRX STI"`, and terser internal shorthand like `"STI 15+"` or `"WRX 08-14"` that names only a trim/chassis code, not the full make. Guessing a make from a bare code would be an unverified inference, so only the fully-qualified shape is parsed into `fitment`; every tag (including the shorthand) is always available in the raw `tags` field. **Why is `tracked_product_count` capped around 250 for a brand?** It's read from Shopify's own `/collections/{brand}/products.json?limit=250` endpoint, which caps at 250 products per page. A brand with more than 250 listed products would be undercounted — plausible given the catalog's real scale, not exhaustively verified for every one of the 373 brands. **Does `entityType=brands` cover every brand on the site?** 373 of them — the ones shown as a real tile on the site's own `/pages/brands` directory. A further 117 brands are only linked via a search-style URL (`/collections/vendors?q=...`) rather than a dedicated collection page, and aren't covered by this entity type. **Why does the `brands` list not include every brand on the site, and could it go stale?** It's built from the same 373 brands `entityType=brands` finds (see the question above) — the same 117-brand gap applies. It was captured from a real full run; if the site adds a new brand later, it won't appear in the dropdown until the list is regenerated (there's no automated sync for this yet, unlike a couple of sibling actors' category dropdowns). **Why does brand-scoped `products` use different pagination than the full catalog?** Both `/products.json` (full catalog) and `/collections/{brand}/products.json` (brand-scoped) support a `since_id` pagination parameter — but real testing found the site currently stopped honoring it on *both*, silently repeating the same page forever instead of erroring. For the full catalog, the fix was switching to the site's own sitemap. For a single brand, page-based pagination (`?page=N`) is a simpler fix that works safely at that smaller scale — Shopify's own page-based pagination hard-caps at 25,000 items, which no single brand's catalog comes anywhere close to, even though it would silently truncate the full 48,938-item catalog. **Does this need a proxy?** No anti-bot layer was found on the main product catalog — a real concurrency burst against `/products.json` came back 100% clean. The brand-collection endpoint used by `entityType=brands` does have a real, hard rate limit at full scale, fixed with sequential fetching and a fixed delay between requests (same fix already proven on a sibling actor's identical endpoint shape) — no proxy involved either way. # Actor input Schema ## `entityType` (type: `string`): Scrape products (JDM/aftermarket auto parts, with price, brand, category, vehicle fitment, and every real purchasable variant) or brands (a real directory crawl with live per-brand product counts — see dataset\_schema.json for the field set of each). ## `startUrls` (type: `array`): Optional list of specific JDMuscleUSA product URLs (https://jdmuscleusa.com/products/{handle}) to scrape directly, instead of discovering via the full catalog. Only applies to entityType=products. A startUrls-scoped run cannot detect delistings or update the incremental baseline. ## `brands` (type: `array`): Optional: restrict entityType=products to specific brands instead of the full 48,938-product catalog. Leave empty to crawl every brand. Each selected brand is paginated to its real completion (not capped at 250, unlike entityType=brands's own tracked\_product\_count) via /collections/{slug}/products.json?page=N — confirmed live to paginate correctly (unlike the site's own since\_id mechanism, which is currently stalled — see README). This list was captured from a real full brands run; if the site adds a new brand later it won't appear here until the list is regenerated. ## `maxItems` (type: `integer`): Stop after pushing this many dataset items. Defaults to 50 — a fast, cheap preview, and what keeps an unconfigured run within Apify's automated 5-minute QA check. The full site has 48,938+ products and 373 brands. Raise this or clear it (set to null) for a full run. ## `mode` (type: `string`): "auto" (recommended): full scan on the first run for entityType=products, incremental (new/changed only) afterwards. "full": always push every item and refresh the baseline — schedule this periodically to catch delistings and price changes. "incremental": always push only new/changed items. Only a plain, unscoped run (no startUrls, no item errors) can detect delistings or update the baseline. Not used by entityType=brands (a fresh live directory crawl every run, no delta mode). ## `concurrency` (type: `integer`): How many product detail pages to fetch in parallel for entityType=products. Defaults to 25 (this portfolio's standard default). Not used by entityType=brands, which always fetches sequentially regardless of this value — the brand-collection endpoint has a real, hard rate limit at full scale (see README). ## `impersonate` (type: `string`): curl\_cffi browser TLS-impersonation target. Defaults to "chrome" internally. Only applies to entityType=products (entityType=brands always uses "chrome", not exposed as an input — see README for why swapping impersonation targets needs real cloud-infra verification, not just a local test). ## Actor input object example ```json { "entityType": "products", "maxItems": 50, "mode": "auto", "concurrency": 25 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("artsiom_k/jdmuscle-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("artsiom_k/jdmuscle-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call artsiom_k/jdmuscle-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,artsiom_k/jdmuscle-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/LFNVlXlHZVpZFfAiY/builds/qeLTVMfadquGBICLS/openapi.json