Dealer Website Inventory Scraper

What does this Actor do?

This Actor reads bounded public dealer inventory pages and extracts vehicle identity, VIN or stock number, pricing, mileage, colors, fuel, descriptions, and public listing/image URLs when those fields are exposed. It supports common DealerOn, Dealer.com, Dealer Inspire, and CDK-style public pages without authentication, private endpoints, proxy use, CAPTCHA solving, or access-control bypass.

Why use it?

Use it to build a point-in-time snapshot of public dealer inventory for local inventory research, catalog comparison, and parser validation. The source is caller-supplied and may be any public dealer HTTPS page; an access boundary or page without trustworthy vehicle evidence becomes a typed diagnostic rather than a placeholder vehicle.

How it works

The Actor requests only public HTTPS dealer pages and permits bounded redirects within the original dealer host or its subdomains. It follows only clearly marked approved next-page links, processes pages sequentially within maxPages, deduplicates records by VIN, stock number, or public listing identity, and never follows vehicle detail links automatically. Response size, timeout, retries, URLs, pages, and emitted vehicles are bounded. A repository-relative fixturePath is for deterministic local checks only.

Input

{
  "dealerUrls": [{"url": "https://www.danvilletoyota.com/searchnew.aspx"}],
  "maxPages": 1,
  "maxVehicles": 50,
  "maxDetailPages": 0,
  "timeoutSecs": 20,
  "maxRetries": 1,
  "maxResponseBytes": 5000000,
  "userAgent": "",
  "acceptLanguage": "en-US,en;q=0.8",
  "fixturePath": ""
}

inventoryUrls and startUrls are aliases for dealerUrls; url is an alias for one URL; maxProducts is an alias for maxVehicles. proxyConfiguration is accepted for compatibility, but the implementation always uses direct public requests and reports proxyConfigured: false.

Field	Type	Default	Description
dealerUrls	array of URL objects	Danville Toyota example	Public dealer inventory/search/detail pages; up to 25 URLs.
inventoryUrls	array of URL objects	—	Compatibility alias for dealerUrls.
startUrls	array of URL objects	—	Compatibility alias for dealerUrls.
url	string	—	Compatibility alias for one public dealer page.
maxPages	integer	3	Maximum public pages requested sequentially, including starts and approved next links; 1–25.
maxVehicles	integer	50	Maximum normalized vehicle rows for the complete run; 1–500.
maxProducts	integer	—	Compatibility alias for maxVehicles.
maxDetailPages	integer	0	Retained compatibility field; unrequested vehicle detail pages are not automatically called; 0–25.
timeoutSecs	integer	20	Per-request timeout in seconds; 5–120.
maxRetries	integer	1	Retries for transient failures; 0–3. Access challenges are never retried as a bypass.
maxResponseBytes	integer	5000000	Maximum streamed response size per page; 100,000–10,000,000 bytes.
userAgent	string	default	Optional transparent user-agent string, capped at 300 characters.
acceptLanguage	string	en-US,en;q=0.8	Optional language preference, capped at 120 characters.
proxyConfiguration	object	—	Compatibility input only; direct public transport remains in use.
fixturePath	string	empty	Optional repository-relative .html, .htm, or .json fixture for deterministic local checks.

Output

Normal dataset rows contain only public inventory fields, for example:

{
  "recordType": "data",
  "vin": "1NXBR32E73Z000001",
  "stockNumber": "FIX-001",
  "name": "2023 Toyota Camry SE",
  "year": 2023,
  "make": "Toyota",
  "model": "Camry",
  "trim": "SE",
  "mileage": 1200,
  "price": 28995,
  "currency": "USD",
  "dealerHost": "www.danvilletoyota.com",
  "productUrl": "https://www.danvilletoyota.com/vehicle-1",
  "sourceUrl": "https://www.danvilletoyota.com/searchnew.aspx",
  "sourceDomain": "www.danvilletoyota.com",
  "searchUrl": "https://www.danvilletoyota.com/searchnew.aspx",
  "pageNum": 1,
  "scrapedAt": "2026-09-16T00:00:00.000Z"
}

Access challenges, upstream failures, invalid inputs, empty pages, and pages without usable vehicle data are emitted as minimal typed diagnostic rows with recordType: "diagnostic", errorCode, url, sourceUrl, and scrapedAt. The run summary is stored in OUTPUT_SUMMARY; operational counters and remoteGates: "NOT_RUN_LOCAL_ONLY" are stored in RUN_HEALTH.

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data fields

Field group	Fields
Identity and dealer	vin, stockNumber, dealerHost, dealerId, name
Vehicle	year, make, model, trim, vehicleType, bodyStyle, salesClass, transmission, location
Commercial and operation	mileage, price, advertisedPrice, msrp, currency, fuelType, engine
Appearance and text	exteriorColor, interiorColor, description
Dealer and enrichment	dealerName, dealerPhone, dealerEmail, dealerUrl, features, options
Links and provenance	productUrl, imageUrls, sourceUrl, sourceDomain, searchUrl, pageNum, scrapedAt

Fields are omitted when the public page does not expose a trustworthy value. A row is a point-in-time observation, not a guarantee that the vehicle remains in stock or that its price, VIN, or dealer details remain current.

Cost and limits

The Actor uses ordinary public HTTPS requests and does not call a paid vehicle-data API. Apify compute, storage, and any platform charges depend on the plan running the Actor. Input limits bound pages, retries, response bytes, source URLs, and normal vehicle rows.

Tips

• Start with one dealer inventory page and a small maxVehicles value.
• Supply the dealer’s public inventory/search page, not an authenticated dashboard.
• Use maxPages only when the page publishes an explicit next link.
• Keep maxRetries low and respect each dealer site’s terms and request rate.
• Use fixturePath while changing parsers or schemas so local checks stay deterministic.
• Treat diagnostic rows as source-health evidence, not vehicle inventory.

FAQ

Does the Actor log in or access dealer systems?

No. It reads only public dealer pages and public structured data in ordinary HTTPS responses.

Why did I receive a diagnostic instead of vehicles?

The source may have returned human verification, an upstream error, unsupported content, invalid input, an empty page, or no trustworthy vehicle record. Inspect errorCode, OUTPUT_SUMMARY, and RUN_HEALTH.

Does maxDetailPages make the Actor visit every vehicle page?

No. It is retained for input compatibility; this direct-only implementation follows explicit pagination but does not automatically call unrequested vehicle detail pages.

Can I use a proxy configuration?

The field is retained for compatibility, but the Actor intentionally uses direct public requests and does not bypass source access controls.

Responsible use and support

Respect each dealer site’s terms, robots guidance, applicable law, and reasonable request rates. Do not use this Actor to bypass access controls, collect restricted personal data, or represent a diagnostic as a vehicle row. For support, include the input shape, OUTPUT_SUMMARY, RUN_HEALTH, and relevant diagnostic code after removing credentials or sensitive data.

Local validation

npm run check
node --test test/public-contract.test.js
apify validate-schema
apify run --purge --input-file test/inputs/fixture.json
node validate-datasets.js

The fixture input is local-only and uses a repository-relative HTML fixture. Normal runs use direct public dealer-page requests; local validation does not claim Cloud qualification.