(New) Reddit Scraper Pro - Fast, Affordable, Supported
Pricing
$20.00/month + usage
(New) Reddit Scraper Pro - Fast, Affordable, Supported
Reddit Scraper Pro is a powerful, user-friendly tool for extracting data from Reddit without API limitations. Offers scraping of posts, users, comments, and communities, advanced search capabilities, and multiple export options. Perfect for brand monitoring, trend tracking, and competitor research.
5.0 (1)
Pricing
$20.00/month + usage
18
Total users
290
Monthly users
83
Runs succeeded
98%
Issues response
5.3 days
Last modified
8 days ago
How do you search keywords within a community if I provide community URL
Closed
For example, I want to search posts with a keyword: "gaming" inside a r/technology community. I already have the url for r/technology, but I need to find posts within this community.
alex.af.junior
For example, if I search 'gaming chair' inside 'r/gaming', it will find posts, but it will find posts not just in this community, but in other communities too. https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/6WxGYu77w2LT0nMqF#log
harshmaur-owner
Hi,
You can use this url To scrape. https://www.reddit.com/r/technology/search/?q=gaming
The idea is to first go to the subreddit then in the search bar add gaming. Then copy rhe url and put it in the scraper.
Let me know if you need more help.
alex.af.junior
This is the run: https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/KC4hTqWQ7IVwxa0T3#log
Error: ERROR [Status message]: Invalid search url, Please provide a search url for posts, communities, or comments
alex.af.junior
It works if you scrape only the community without specifying keywords: https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/pzffsPSg0p29hpLvi#input
But as I mentioned earlier, if you specify a keyword: https://www.reddit.com/r/technology/search/?q=gaming
It doesn't work. Run: https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/7lyU2Lgl8gRfIcedS#log
harshmaur-owner
I'll fix it. Give me a couple of days if possible. I've been travelling.
alex.af.junior
No problem, thank you. I appreciate it.
Hi,
I have updated the code that now allows you to provide that URL in start URL and it will work and only give you results for that subreddit.
Additionally I have also added functionality in search to search within a community, so you can instead of providing the search URL, you can provide search options.
Also, I saw I had an issue fetching all comments, and I fixed that too
alex.af.junior
Can you check this run: https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/JgNKO31xzuXHcQC6b#input I specified the URL of the community and two keywords. But it gave me the results for different subreddits.
Regarding "withinCommunity": "" What values does it take? Does it take URLs or names? My goal is to have many subreddits and many keywords.
alex.af.junior
My other issue was not fetching comments, it was fetching communities based on keywords. example run: https://console.apify.com/actors/3XedXIRBcjfKrnsDJ/runs/0FyboaHe9V7FDbz2g#log
Hi,
You are trying to use search and start URLs together, and the in the crawler you either provide the start URL or provide the search keywords and within community.
So in your case, you have two options
Option 1 - Provide 2 start URLs https://www.reddit.com/r/technology/search/?q=gaming+headset https://www.reddit.com/r/technology/search/?q=best+headset+for+gaming
and make sure you don't enter any search keywords
Option 2 - Provide Search Keywords
For this to work, make sure you keep the start URLs empty, Then go to search, and provide keywords like you did, and then in withinCommunity, mention technology.
In your second comment you mentioned that it wasn't fetching comments, that's because you have crawlCommentsPerPost": false
Make sure you toggle Crawl Comments Per Post to On (Its below the start URLs and before search keywords section.
alex.af.junior
Okay, it's working. Thank you!
