Reddit Scraper
Pricing
$45.00/month + usage
Reddit Scraper
Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.
3.9 (2)
Pricing
$45.00/month + usage
115
Total users
6.9K
Monthly users
441
Runs succeeded
>99%
Issues response
2.2 days
Last modified
an hour ago
Issue Scraping more than 1000 Results
Closed
Hello, I have rented the actor and have money on my usage limit however. I am unable to get the Reddit Scraper to scrape more than 1000 results. How can I fix this? Thank you!
That's a hard limit on Reddit pagination. They store up to 1000 posts for any subreddit. Checking the logs for your run, the scrapper reached the last page until no more results were returned.
redditscrapermit
Thank you. So if I understand correctly there would not be a away to scrape more than 1000 posts in a subreddit? If so would there be a way to scrape 1000 posts at a time to multiple times to get more posts form a subreddit? Thank you
As newer posts are created, older ones become unavailable, so you must repeat the process to keep storing the newer ones.
redditscrapermit
Thank you again for the response. Just to confirm, for example I would not be able to get results going back years?
In a very active subreddit, no.
