1 day trial then $45.00/month - No credit card required now
Reddit Scraper
1 day trial then $45.00/month - No credit card required now
Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.
Irrelevant data despite using keyword search
Closed
Run Id: "g0U8gX5voivV1y39v"
I wanted to scrape reddit data from the sub-reddit "r/birthcontrol" using 58 keywords. However, when I went through the outputs, I observed the following:
- The sub-reddit has more than 250K post + comments in the past 2.5 years but I am getting only 5.8K
- Even in the 5.8K post + comments, many of it did not contain the keywords that I had provided
Could you provide a solution for the same.
Also, had few additional questions
- Is there a possibility to scrape data for a particular time period i.e. last 3 years data
- What do I have to do if I want to scrape all the data available for a particular sub-reddit
- Is there a data limit per keyword?
- Is there a data limit per sub-reddit?
When you use a startUrl the search parameters are ignored. I like what you are trying to do, I will update the actor to allow you to search using a startUrl as base. The navigation on Reddit does not allow you to filter by date so the only option is to scrape all that you can and filter it out after. Reddit also limits the number of posts you can get so it is difficult to scrape all data for a subreddit because even if you go page by page Reddit will not show you all of it.
Gustavo Rudiger- 366 monthly users
- 37 stars
- 99.9% runs succeeded
- 1.2 days response time
- Created in Feb 2022
- Modified 9 days ago
