1 day trial then $45.00/month - No credit card required now
Reddit Scraper
1 day trial then $45.00/month - No credit card required now
Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.
Crawling data are not enough.
Closed
I only collected 3000 posts and comments even though I didn't set the limit. I wonder if I could collect more data than 3000.
can you share the run ID so I can look at what is happening?
junior_fist
https://console.apify.com/view/runs/IDYivXgybpMP9K7OD
This is the latest one that I collected. Thank you for your help.
junior_fist
I also wonder why I only collected data for a few days even though I set the time for all time.
Reddit only provides the last 1000 posts and your run went to that last page available on the website. According to the logs, this was the last page: https://www.reddit.com/r/doordash_drivers/?after=t3_18d6lbg. You are also limiting to 3 comments per each post which results in the ~3000 results you are getting.
junior_fist
Is there any way for me to get more than the last 1000 posts?
Not that I know of. It is a hard limit imposed by Reddit for many years now.
Gustavo Rudiger- 286 monthly users
- 99.2% runs succeeded
- 0.5 days response time
- Created in Feb 2022
- Modified 3 days ago
