Reddit Scraper
Pricing
$45.00/month + usage
Reddit Scraper
Unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.
3.9 (2)
Pricing
$45.00/month + usage
96
Monthly users
334
Runs succeeded
>99%
Response time
5.8 days
Last modified
15 days ago
returning 0 comments, tested a few different subreddits
Closed
output from my code:
Scraping r/devops...
Debug: Found 20 total items in dataset Debug: Processing item: 'Getting into DevOps' with 0 comments Debug: Processing item: No title with 0 comments Debug: Processing item: No title with 0 comments Debug: Processing item: How should this sub respond to reddit's api changes, part 2 with 0 comments Debug: Processing item: No title with 0 comments Successfully scraped 0 comments from r/devops
First few rows of the scraped data: Empty DataFrame Columns: [] Index: []
Can you share the run id?
harshhellohello
AcOrjJHODyGFbkinv
I've shared access to all runs from my account
Thank you!
harshhellohello
or can you give me sample code that I can try out to fetch comments for posts in a specific subreddit?
You have to increase the maxItems value in your input. Right now, it is the same as the number of posts, so the run will stop once 20 items are stored.
harshhellohello
I'm still having trouble, can you give me sample code that I can try out to fetch comments for posts in a specific subreddit?
I don't know what you mean by sample code. You mean the input for the actor?
harshhellohello
yes, something that I can build on.
1{ 2 "startUrls": [ 3 { 4 "url": "https://www.reddit.com/r/devops/hot/" 5 } 6 ], 7 "sort": "top", 8 "maxItems": 20000, 9 "maxPostCount": 20, 10 "maxComments": 10, 11 "maxCommunitiesCount": 1, 12 "maxUserCount": 100, 13 "scrollTimeout": 60, 14 "proxy": { 15 "useApifyProxy": true, 16 "apifyProxyGroups": [ 17 "RESIDENTIAL" 18 ] 19 }, 20 "searchFilter": "top", 21 "skipDailyThreads": false, 22 "skipComments": false, 23 "skipUserPosts": false, 24 "skipCommunity": false, 25 "searchPosts": true, 26 "searchComments": false, 27 "searchCommunities": false, 28 "searchUsers": false, 29 "includeNSFW": true, 30 "debugMode": false 31}
harshhellohello
Great thanks, I've got it working.
Just one more thing, even with the settings below it outputs exactly the same content as with the settings you shared above (only outputs comments from two posts). Thanks!
How come I'm not able to output let's say all comments from the last 750 posts?
run_input = { "startUrls": [ { "url": "https://www.reddit.com/r/devops/hot/" } ], "sort": "top", "maxItems": 20000, "maxPostCount": 750, "maxComments": 1000, "maxCommunitiesCount": 1, "maxUserCount": 1000, "scrollTimeout": 120, "proxy": { "useApifyProxy": True, "apifyProxyGroups": [ "RESIDENTIAL" ] }, "searchFilter": "top", "skipDailyThreads": False, "skipComments": False, "skipUserPosts": False, "skipCommunity": False, "searchPosts": True, "searchComments": False, "searchCommunities": False, "searchUsers": False, "includeNSFW": True, "debugMode": True }
Pricing
Pricing model
RentalTo use this Actor, you have to pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage.
Free trial
1 day
Price
$45.00
