FMCSA DOT Crawler

Overview

The FMCSA DOT Crawler collects structured information about U.S. motor carriers directly from the FMCSA SAFER "Company Snapshot" pages. It supports keyword‑style filtering (DOT range, state, fleet size, etc.) and an optional Premium mode that adds crash statistics, safety ratings and public email addresses.

Typical use–cases include:

• Lead generation for logistics or insurance companies
• Competitor or market‑size analysis
• Compliance monitoring and risk assessment
• Building datasets of motor carriers for analysis
• Monitoring FMCSA updates for compliance or reporting
• Integrating carrier data with logistics systems

---

Key Features

• Comprehensive Data Extraction: Scrapes legal names, DBA names, phone numbers, addresses, operational stats, and cargo types.
• Flexible filters – pull only the carriers you need by DOT number window, state, fleet size, driver count or add‑date.
• Premium mode – enriches each record with crash history, FMCSA safety rating and public e‑mail (if listed).
• Resilient crawling – automatic retries, exponential back‑off and proxy rotation keep the run moving even when SAFER throttles.
• Straight‑forward output – each company snapshot is returned as a single JSON object that is easy to ingest downstream.

---

Input Schema (v2)

The scraper reads the parameters below at run‑time. Required fields are marked bold. Values not relevant to the chosen filter_mode can be omitted.

Field	Type	Description
filter_mode	string (enum)	Primary filter. One of: • DOT_RANGE (default) • STATE • POWER_UNITS • DRIVERS • ADDED_AFTER
dot_start	integer	First DOT number (inclusive) when filter_mode = DOT_RANGE.
max_requests_per_crawl	integer	How many sequential DOT numbers to query (0 → no limit).
state	string (2 char US state)	Two‑letter state code when filter_mode = STATE.
power_units_min	integer	Minimum fleet size when filter_mode = POWER_UNITS.
power_units_max	integer	Maximum fleet size when filter_mode = POWER_UNITS.
drivers_min	integer	Minimum driver count when filter_mode = DRIVERS.
drivers_max	integer	Maximum driver count when filter_mode = DRIVERS.
added_after	string (YYYY‑MM‑DD)	Include carriers whose ADD_DATE ≥ this date when filter_mode = ADDED_AFTER.
is_premium_mode	boolean	If true, scrape crash data, safety rating & email (≈ +3 requests / record). Default false.
sp_intended_usage	string	Short sentence describing how you plan to use the data.
sp_improvement_suggestions	string	Your feedback for future upgrades.

Example – state filter

{
  "filter_mode": "STATE",
  "state": "TX",
  "is_premium_mode": false,
  "sp_intended_usage": "Outbound prospecting list for TX carriers",
  "sp_improvement_suggestions": "Option to filter by cargo type"
}

---

Output Fields

Each successful company snapshot is returned as a flat JSON object. The columns differ slightly between Standard and Premium runs:

Field	Standard	Premium
DOT_num	✓	✓
entity_type, legal_name, dba_name	✓	✓
mc_mx_ff_numbers	✓	✓
phone, physical_address, mailing_address	✓	✓
power_units, drivers	✓	✓
cargo_carried, carrier_operation, operation_classification	✓	✓
DUNS_num	✓	✓
inspections_us (vehicle / driver / hazmat / iep)	✓	✓
inspections_ca (vehicle / driver)	✓	✓
email		✓
crashes_us (fatal / injury / tow / total)		✓
crashes_ca (fatal / injury / tow / total)		✓
safety_rating (rating_date / review_date / rating / type)		✓

Sample (Standard)

{
  "DOT_num": "2802023",  
  "entity_type": "Carrier",
  "legal_name": "Example Logistics LLC",
  "dba_name": "Example Trucks",
  "mc_mx_ff_numbers": "MC‑123456",
  "phone": "555‑123‑4567",
  "physical_address": "123 Main St, Springfield, IL",
  "mailing_address": "PO Box 456, Springfield, IL",
  "power_units": "50",
  "drivers": "75",
  "cargo_carried": ["General Freight", "Household Goods"],
  "carrier_operation": ["Interstate"],
  "operation_classification": ["For Hire"],
  "DUNS_num": "012345678",
  "inspections_us": { "vehicle": "10", "driver": "2", "hazmat": "0", "iep": "0" },
  "inspections_ca": { "vehicle": "–", "driver": "–" }
}

---

Notes & Best Practices

• Stick to a single primary filter per run – combining filters is currently not supported.
• Large DOT ranges (≥ 1 M numbers) are best run in chunks to avoid 24 h timeouts.
• If you hit repeated 403 Forbidden responses, lower concurrency or enable an IP‑rotation proxy plan.

---

Need More Features?

If you'd like to add new data fields to this scraper or need a custom scraper for another purpose, feel free to file an issue or get in touch! We are open to customizing the scraper to suit your needs.

Why Choose This Scraper?

• Efficient: Processes a high volume of requests with intelligent retries and session management.
• Customizable: Tailor input options to suit your scraping needs.
• Reliable: Leverages robust error-handling and proxy configurations for uninterrupted scraping.

Proxies and Anti-blocking

This scraper uses the Apify Proxy or your custom proxy settings to reduce the risk of being blocked. It supports automatic session management for smooth operation.

Resources

• Apify SDK Documentation
• CheerioCrawler Documentation

---

Enhance your data collection capabilities with the FMCSA DOT Crawler. Get started today!