7 days trial then $20.00/month - No credit card required now
Google Trends Scraper
7 days trial then $20.00/month - No credit card required now
Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.
DATA ERROR in log result
Closed
Hi Team, I've been trying to set up a job for comparing different ingredients and it's failing quite often (never really returning all the results). The pageBlocks I understand, but I am also seeing some DATA ERROR messages in the log that I can not quite interpret, e.g.: "WARN PuppeteerCrawler: Reclaiming failed request back to the list or queue. DATA ERROR at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/dist/main.js:153:19)"
Is that a bug in the code or something wrong in my config? Any idea is welcome to make this run possible/stable since I want to schedule it for the upcoming months as well :)
peaksandpies
I tried to narrow it down even more and e.g. this URL does not want to work: https://trends.google.com/trends/explore?geo=DE&date=today%205-y&cat=44&q=%2Fm%2F07zqy,%2Fm%2F07zjbq,%2Fm%2F03prgw,%2Fm%2F059h5_,%2Fm%2F0b4fjv I see nothing wrong with it (apart from the excessive use of non-search terms, e.g. rather using Topics and Compounds etc.).
peaksandpies
I also tried using the beta build, but the problem persists - looking forward to any ideas :)
Hi! Thanks for feedback, looks like actor unable to handle correctly case when there is 5 topics used at once as in your referenced URL. Bug confirmed, will keep you informed about the progress
peaksandpies
Thanks Alexey :)
devout_tuna
Some news ?
Ondrej Klinovský (ondrejklinovsky)
Hi,
we're still looking into this. Google trends are probably blocking users that are not signed in (429 error responses). We'll keep you posted.
devout_tuna
I am not signed in (Google) and it's work for me if i go on Google Trends.
peaksandpies
Just a quick ping on this again :) Was wondering if you guys could provide a rough estimate on when the fix is available. I will build some workarounds in the meantime, but just so that we can tell something to our clients?
Ondrej Klinovský (ondrejklinovsky)
Hi,
I guess 1-2 weeks. Still trying to minimize 429 rate, although it seems there's not that many of them in last days.
Ondrej Klinovský (ondrejklinovsky)
Hi, we've just released a new version of the scraper with improved handling of blocked requests.
I'm gonna close the issue now but feel free to reopen it if you encounter any more issues.
Emiliano Mastragostino- 115 monthly users
- 27 stars
- 99.2% runs succeeded
- 4.8 days response time
- Created in Jan 2020
- Modified 25 days ago
