No credit card required
Instagram Hashtag Scraper
No credit card required
Scrape Instagram hashtags data. Just add one or more hashtags and extract posts, images, URLs, comments, likes, users, locations, timestamps, and more. Export scraped datasets, run the scraper via API, schedule and monitor runs or integrate with other tools.
ERROR Request batch insert too large
Closed
I am trying to scrape hashtag information from some Japanese hashtags (#プラントベースフード for example). Over the summer I scraped about 6000 posts using this scraper no problem, but now I am trying to use it to get more data, and every time I run, it gives roughly 15 results regardless of how high the limit is set or how many gb of ram I use. The log notes repeated errors that the batch input fails due to the size being too large. I'm not sure if this is the principal reason for limiting the number of results returned or if there is something else about the scraper that is new. Would appreciate any help you can offer. Thanks!
EDIT: After waiting and playing around with different max post parameters, I was able to get more results, but the log continues to give repeated errors that the request batch insert is too large. During the times when the errors are registered, it looks like no data is being collected, so even if this actor were allowed to run to completion, I suspect it would leave out a significant portion of the data. Perhaps there would be a way to limit the amount of data that is being collected? For example, I don't need the first comment and that sort of data might be making the file too big. Unsure of how to proceed.
Hi! Thanks a lot for letting us know. Fixed in latest build: https://console.apify.com/view/runs/63aguzZ0S61Fjor7z Payload issue was the reason why pagination not continued, amount of results should be as usual for all tags in updated version.
I´m going to close the issue now, but if there would be anything else we could help with, please let us know.
Apify- 1.5k monthly users
- 99.6% runs succeeded
- 1.1 days response time
- Created in Nov 2021
- Modified 2 days ago
