Reddit Public Post & Comment Scraper

Point it at any public subreddit and get structured posts and comments back - no Reddit API key, no login, no rate-limit headaches.

What it does

• Fetches posts from any public subreddit via Reddit's .json endpoints (no OAuth, no API key)
• Optionally extracts top-level and nested comments for each post
• Deduplicates posts by Reddit post ID across multiple runs of the same subreddit (persisted in the Actor's Key-Value Store)
• Skips private, banned, and quarantined subreddits with a clear log message - never crashes
• Respects Reddit's public rate limits with built-in throttling (single concurrent request + random delay) and automatic exponential backoff on 429 responses
• Supports two output shapes: nested (one row per post with a comments array) or flat (one row per post plus one row per comment - easier for CSV export)

Input

Field	Type	Default	Description
startUrls	array<string>	(required)	Subreddit names (javascript, r/javascript) or full Reddit post URLs. Mixed input is allowed.
sortMode	enum	hot	hot | new | top | rising. Ignored for direct post URLs.
maxPosts	integer	100	Max posts per subreddit (1-1000).
includeComments	boolean	true	Fetch comments for each post.
maxCommentsPerPost	integer	50	Max comments extracted per post (0-500, depth-first).
timeRange	enum	all	Used only with sortMode: top (hour/day/week/month/year/all).
outputMode	enum	nested	nested (one row per post with comments array) or flat (one row per post + one row per comment).
dedupeAcrossRuns	boolean	true	Skip posts already seen in previous runs.
minRequestDelayMs	integer	800	Minimum delay between requests (with random jitter). Increase if you see 429s.
useApifyProxy	boolean	true	Route requests through Apify's residential proxy pool. Required - Reddit blocks data-center IPs from .json endpoints.
proxyGroups	array<string>	[]	Optional Apify proxy groups (e.g. ["RESIDENTIAL"]).
proxyCountryCode	string	""	Optional ISO country code (e.g. US, GB).

Output

Each dataset row contains:

Post fields: id, title, author, score, upvoteRatio, numComments, createdUtc, subreddit, flair, permalink, url, selftext, postHint, gilded, totalAwards, isStickied, isOver18, isLocked, and more.

Comment fields (in comments array for nested mode, or as separate rows in flat mode): id, author, body, score, createdUtc, depth, parentId, isStickied, isOp, permalink, gilded.

Example

Input:

{
  "startUrls": ["javascript", "https://www.reddit.com/r/programming"],
  "sortMode": "top",
  "timeRange": "week",
  "maxPosts": 50,
  "includeComments": true,
  "maxCommentsPerPost": 20,
  "outputMode": "nested"
}

Output (nested mode, one row):

{
  "type": "post",
  "id": "1abc23",
  "title": "Example post title",
  "author": "some_user",
  "score": 1234,
  "upvoteRatio": 0.95,
  "numComments": 87,
  "createdUtc": "2025-01-15T12:34:56.000Z",
  "subreddit": "javascript",
  "permalink": "https://www.reddit.com/r/javascript/comments/1abc23/...",
  "url": "https://example.com/article",
  "selftext": "",
  "comments": [
    {
      "id": "def45",
      "author": "another_user",
      "body": "Great post!",
      "score": 42,
      "depth": 0,
      "parentId": null
    }
  ]
}

Scheduling

This Actor is safe to run on a recurring schedule (daily, weekly). With dedupeAcrossRuns: true, each run only emits posts not seen in previous runs - ideal for monitoring subreddits over time. Reset dedup by toggling dedupeAcrossRuns: false or by clearing the Actor's reddit-dedup Key-Value Store.

Privacy & compliance

• Only publicly available Reddit data is accessed, via Reddit's unauthenticated .json endpoints.
• No Reddit OAuth, no API key, no login - no PII beyond what Reddit already displays publicly (usernames, public post/comment text).
• Private, banned, and quarantined subreddits are skipped, not bypassed.
• This Actor does not bypass Reddit's rate limits or ToS. It uses built-in throttling and backoff to stay within Reddit's public rate limits.

Proxy requirements (important)

Reddit actively blocks datacenter IP ranges (including Apify's default proxy pool) from .json endpoints. To run reliably, this Actor requires Apify's residential proxy group (RESIDENTIAL), available on Apify's Starter plan or higher. The Free plan does not include residential proxy access.

If you see repeated HTTP 403 errors in the logs:

1. Ensure your Apify account has the residential proxy group enabled.
2. Pass proxyGroups: "RESIDENTIAL" in the input.
3. If you have your own external proxy that Reddit doesn't block, set useApifyProxy: false and configure your proxy via the proxyGroups field.

The Actor's logic (parsing, deduplication, comment extraction, rate limiting) is fully functional - the only failure mode is Reddit blocking the proxy IP range, which is outside the Actor's control.