Google Maps Scraper
Pricing
Pay per event
Google Maps Scraper
Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, opening hours, location, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.
4.2 (90)
Pricing
Pay per event
1556
Total users
124K
Monthly users
17K
Runs succeeded
97%
Issues response
3.2 days
Last modified
a day ago
time to first result, any way to speed up?
Closed
hi there, I'm developing an app that feeds restaurants to people based on their location. it works great with google API, which is of course quite expensive. it is also fast, with results returning within 300 milliseconds. I'd like to switch to your actor but the time to first result seems to be at least 20-30 seconds. any way to improve? I've only run with the free version but even with the 8192 MB / 2 vCPU, the time is 25s to first result. I'd be looking for something < 10 seconds (with the google API next page token wait period, it ends up being ~6s total for 3 pages = 60 places).
Ondrej Klinovský (ondrejklinovsky)
Hey,
thank you for considering this actor.
There are several steps between launching the actor and seeing first results:
- downloading and starting Docker container
- initializing the actor (parsing input, geolocation, initializing crawler, etc)
- launching browser (this is probably the slowest part)
You can try our other actor - Google Maps Extractor - it doesn't use browser so you should see the first results sooner, after about 12-15 seconds.
We will discuss it with team whether we can optimize it some more.
nasty_wombat
Yes I understand spinning up the container in a Kubernetes cluster isn't instant and can be in fact quite slow. My day job is backend dev (python/django/C#)/DevOps for a couple EKS clusters I created. If launching the browser is the slowest part, not a lot can be done there except optimize the actual hardware it's running on (guessing that prioritizing singled threaded CPU performance could 2x the launch speed compared to a traditional server CPU like modern EPYC/Xeon). Does each request to google for one of the map segments ("INFO Splitted the map into 148 map segments") use a different proxy and would therefore require a new browser instance to be launched?
I suppose I should start thinking about potentially doing the place collection asynchronously (e.g. 1x per week) and have places where I launch my app "pre-scraped".
Ondrej Klinovský (ondrejklinovsky)
Does each request to google for one of the map segments ("INFO Splitted the map into 148 map segments") use a different proxy and would therefore require a new browser instance to be launched?
I don't know the exact details but the browser gets restarted every 20 requests or so (e.g. to clear cache). Each request has assigned a proxy from proxy pool. After request completion, the proxy goes back to the pool and gets reused couple of times before it's discarded.
chanmathew
Hi Ondrej, now that there are Actor standby modes, is it possible to enable it on this actor which may help speed it up much more?
