Reddit Scraper Pro

A production-ready Apify Actor that scrapes Reddit posts, comments, subreddit info, and user profiles without login — using Reddit's public .json endpoints.

Features

• 📝 Post scraping — title, text, author, score, media, timestamps
• 💬 Comment scraping — recursive comment tree parsing with depth tracking
• 📊 Subreddit info — members, description, category, icons
• 👤 User profiles — karma, account age, recent activity
• 🔍 Keyword search — search Reddit with sort options
• 📄 Pagination — automatic pagination until limits are reached
• 🔄 Rate limiting — respects Reddit rate limits with delays and concurrency caps
• 🛡️ Proxy support — Apify Proxy or custom proxy configuration
• ⚡ Fast — uses CheerioCrawler (no browser overhead)

Input

Field	Type	Default	Description
startUrls	Array	Required	Reddit URLs (subreddit, post, user)
searchKeywords	Array	[]	Keywords to search on Reddit
maxPosts	Number	100	Max posts to scrape
maxComments	Number	200	Max comments to scrape
sortBy	String	"hot"	Sort: hot, new, top, relevance
scrapePosts	Boolean	true	Enable post scraping
scrapeComments	Boolean	true	Enable comment scraping
scrapeUsers	Boolean	false	Enable user profile scraping
proxyConfiguration	Object	—	Apify proxy settings

Example Input

{
  "startUrls": [
    { "url": "https://www.reddit.com/r/programming/" },
    { "url": "https://www.reddit.com/r/webdev/comments/abc123/some_post/" }
  ],
  "searchKeywords": ["AI tools", "web scraping"],
  "maxPosts": 100,
  "maxComments": 200,
  "sortBy": "hot",
  "scrapePosts": true,
  "scrapeComments": true,
  "scrapeUsers": false
}

Output

Each item in the dataset has a type and data field:

Post

{
  "type": "post",
  "data": {
    "postId": "abc123",
    "title": "Example Post Title",
    "postText": "Post body text...",
    "username": "author_name",
    "upvotes": 1234,
    "score": 1234,
    "numberOfComments": 56,
    "postUrl": "https://www.reddit.com/r/sub/comments/abc123/...",
    "mediaUrls": ["https://i.redd.it/image.jpg"],
    "subreddit": "r/programming",
    "flair": "Discussion",
    "timestamp": "2024-01-15T10:30:00.000Z"
  }
}

Comment

{
  "type": "comment",
  "data": {
    "commentId": "xyz789",
    "commentText": "This is a comment...",
    "username": "commenter",
    "upvotes": 42,
    "timestamp": "2024-01-15T11:00:00.000Z",
    "commentUrl": "https://www.reddit.com/r/sub/comments/abc123/.../xyz789/",
    "parentPostId": "abc123",
    "depth": 0
  }
}

Subreddit

{
  "type": "subreddit",
  "data": {
    "name": "programming",
    "fullName": "r/programming",
    "members": 5000000,
    "activeUsers": 12000,
    "category": "Technology",
    "communityUrl": "https://www.reddit.com/r/programming/",
    "description": "..."
  }
}

User

{
  "type": "user",
  "data": {
    "username": "spez",
    "totalKarma": 250000,
    "postKarma": 100000,
    "commentKarma": 150000,
    "accountAge": "6205 days",
    "cakeDay": "2005-06-06T...",
    "recentPosts": [],
    "recentComments": []
  }
}

URL Types Supported

URL Pattern	Type	Example
/r/<subreddit>/	Subreddit	https://www.reddit.com/r/programming/
/r/<sub>/comments/<id>/...	Post	https://www.reddit.com/r/programming/comments/abc123/title/
/user/<name> or /u/<name>	User	https://www.reddit.com/user/spez
Search keywords	Search	Input: searchKeywords: ["AI tools"]

Rate Limiting

The Actor respects Reddit's rate limits:

• Max 3 concurrent requests
• Max 30 requests per minute
• Random delays (1–3s) between requests
• Automatic retry with backoff on 429 responses
• Session pool for sticky proxy rotation