SAFER FMCSA DOT Crawler
Pricing
Pay per event
SAFER FMCSA DOT Crawler
Crawl the SAFER DOT.GOV database for publicly registered vehicles. Supports Address, phone, email, DUNS and other registration details. Perfect for Lead Generation!
5.0 (1)
Pricing
Pay per event
6
Total users
111
Monthly users
6
Runs succeeded
>99%
Issues response
4.3 hours
Last modified
a month ago
Actor experienced OOM many hours in, have to resurrect with more memory, but unclear (from Apify msg) how
Closed
Unattended run, failed 17h30m in after 112250 results
BowTiedRacoon (jungle_synthesizer)
Thanks for the report. Its the longest run I have seen to date! Probably some memory issues, will take a look.
donrb
It's running again. Suspect we'll hit a failure condition again around the sound time, will take multiple days to complete. I need to learn how to run more actors concurrently and send to the same dataset or merge.
BowTiedRacoon (jungle_synthesizer)
Just deployed a few changes that should help address the memory issue. Can you restart your run?
Also, had to disable the "premium" mode for now since Apify forces a 14 day wait period for such changes.
For this specific actor, I doubt that concurrent runs will be any faster. We already use over 20 threads to hit the server. The main bottleneck is the DOT website. If you run crawlers for different DOT number ranges (say 500k per run), you can then export the datasets and just copy-paste them together into a single master.
