Reddit Scraper
1 day trial then $45.00/month - No credit card required now
Reddit Scraper
1 day trial then $45.00/month - No credit card required now
Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.
Hi Gustavo, awesome tool you have developed! I am trying to scrape a specific subreddit for some specific keywords.
Specifically, I am using a URL like this: https://www.reddit.com/r/askscience/search/?q=radioactive&cId=4d6df564-c307-413f-a048-d9d7cdf25365&iId=044b33fa-000b-48c2-b532-363a94b24888
However, the results I'm getting back only include posts that mention the keyword in their body and the comments of these post. But on Reddit itself, if i do a similar search I can switch to Comments at the top and it will show me the comments that contain the keyword regardless if the keyword is mentioned in the post body.
Am I doing something wrong? How can I make sure I am getting the comments as well, not just the posts?
Let me know if there's additional information you need from me, thank you so much!
PS(I'm including the actual actor run so you can see what I mean)
Looks like a bug on the actor. I am working on a fix
You might be reaching the limit with only posts. You can try to use one run to get only posts and another to get only comments.
Hi Gustavo, thanks so much for looking into this. I tried running a scrape just for comments, but it doesnt seem to work properly or maybe I'm not setting it up properly. From the main screen I activate Skip user posts and Skip community. But when I check what the run actually did I find these parameters in the json input: "searchComments": true, "searchCommunities": false, "searchPosts": true, "searchUsers": false,
As you can see, search Posts is still set to true. And when I check the results I still have results that do not include the keyword. I understand that the URL i'm giving it it's a bit more special perhaps since it includes both a subreddit and a search query parameter, since what i want is to search for that keyword in that specific subreddit. Is it not possible to search in this way and get only comments?
For example, if you want to search on r/news the term "Shakira" the comments you see on this link they all contain the keyword: https://www.reddit.com/r/news/search/?q=shakira&type=comments&cId=704ffb7c-0ffe-4a09-9c00-63fdf5f3f6a1&iId=2816285f-f7b6-4a14-91ed-203a5259e0e2 Is there a way for the scraper to achieve this with the scraper?
Thank you so much for your time!
Hi Andreea,
Sorry for the delayed response. "Skip user posts" and "Skip community" only prevent the actor from getting those while doing a broad search. What you need to check is the Search settings. You will find the toggle to disable search for posts, users, and communities there.
Actor Metrics
317 monthly users
-
60 stars
>99% runs succeeded
12 hours response time
Created in Feb 2022
Modified 21 days ago