Reddit Scraper avatar
Reddit Scraper
Try for free

1 day trial then $45.00/month - No credit card required now

View all Actors
Reddit Scraper

Reddit Scraper

trudax/reddit-scraper
Try for free

1 day trial then $45.00/month - No credit card required now

Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.

User avatar

Crawling data are not enough.

Closed

junior_fist opened this issue
5 months ago

I only collected 3000 posts and comments even though I didn't set the limit. I wonder if I could collect more data than 3000.

User avatar

can you share the run ID so I can look at what is happening?

User avatar

junior_fist

5 months ago

https://console.apify.com/view/runs/IDYivXgybpMP9K7OD

This is the latest one that I collected. Thank you for your help.

User avatar

junior_fist

5 months ago

I also wonder why I only collected data for a few days even though I set the time for all time.

User avatar

Reddit only provides the last 1000 posts and your run went to that last page available on the website. According to the logs, this was the last page: https://www.reddit.com/r/doordash_drivers/?after=t3_18d6lbg. You are also limiting to 3 comments per each post which results in the ~3000 results you are getting.

User avatar

junior_fist

5 months ago

Is there any way for me to get more than the last 1000 posts?

User avatar

Not that I know of. It is a hard limit imposed by Reddit for many years now.

Developer
Maintained by Community
Actor metrics
  • 281 monthly users
  • 100.0% runs succeeded
  • 12 hours response time
  • Created in Feb 2022
  • Modified 7 days ago
Categories