Reddit Scraper

Scrape posts, comments, and discover subreddits from Reddit — no API credentials required.

What Can This Actor Do?

Mode	What You Get	Example Use Case
📥 Scrape	Posts & comments from subreddits	Monitor r/startups for pain points
🔍 Discover	Find subreddits by keyword	Find where your audience hangs out
🔎 Search	Posts matching keywords	Track brand mentions
🌐 Domain	Posts linking to websites	Monitor your content sharing

---

⚡ Quick Start

Scrape Subreddits

{
  "mode": "scrape",
  "listings": [
    { "subreddit": "entrepreneur", "maxPosts": 25 },
    { "subreddit": "startups", "maxPosts": 25 }
  ]
}

Search Reddit

{
  "mode": "search",
  "searchTargets": [
    { "query": "best CRM software", "maxResults": 25 }
  ]
}

Discover Subreddits

{
  "mode": "discover",
  "searchTerms": ["saas", "startup tools"],
  "maxResults": 25
}

Track Domain Mentions

{
  "mode": "domain",
  "domainTargets": [
    { "domain": "mycompany.com", "maxResults": 25 }
  ]
}

---

📊 Output

Results are available in multiple dataset views:

• All Results — Everything scraped
• Posts Only — Just posts (no comments)
• Comments Only — Just comments
• Discovered Subreddits — Subreddits from Discover mode

Post fields include: title, text, author, url, domain, created_utc_iso, permalink, listing_rank, score, num_comments

Comment fields include: text, author, created_utc_iso, permalink, score, depth, parent_id

Subreddit fields include: display_name, title, public_description, estimated_posts_per_day, estimated_posts_per_week

---

✨ Key Features

• Parallel execution — All targets run simultaneously
• Unauthenticated scraping — Works without Reddit API credentials
• Optional comments — Fetch comments when needed
• Smart proxies — Automatic rotation to avoid blocks
• No API keys — Uses public Reddit data

---

🤖 MCP Server (AI Agent Integration)

Connect this Actor to Claude, Cursor, VS Code, or Windsurf as an MCP server. One command for Claude Code:

claude mcp add reddit-scraper \
  -e APIFY_TOKEN=<YOUR_APIFY_TOKEN> \
  -- npx -y @apify/actors-mcp-server@latest --actors spry_wholemeal/reddit-scraper

Full setup guide (all clients) → | Copy-paste agent prompts →

---

🧩 n8n Workflow Templates

Import-ready n8n workflows that use this Actor as the data source:

• Lead Finder — AI buying-intent scanner → Slack + Google Sheets
• Subreddit Discovery — Niche audience map
• Content Machine — Reddit threads → blog post drafts
• Weekly Digest — Email summary + AI trends

Browse templates & setup guides →

---

📋 Input Reference

Required Fields by Mode

Mode	Required Field	Format
scrape	listings	[{ "subreddit": "python" }]
discover	searchTerms	["saas", "startup", ...]
search	searchTargets	[{ "query": "best CRM" }]
domain	domainTargets	[{ "domain": "github.com" }]

Sorting & Limits

Field	Type	Default	Used In	Description
sort	string	hot	Scrape, Domain	hot, new, top, rising, controversial
timeframe	string	week	Scrape, Search, Domain	hour, day, week, month, year, all
maxPosts	number	100	Scrape	Default max posts per listing target
maxResults	number	25	Discover, Search	Max results per search/discovery target
searchSort	string	relevance	Search	relevance, hot, new, top, comments
maxResultsPerDomain	number	500	Domain	Default max posts per domain target

Reddit may return fewer items than requested for some sorts, timeframes, or communities.

Comments

Field	Type	Default	Description
includeCommentsMode	string	none	none or all
maxTopLevelComments	number	50	Max comments to fetch
maxRepliesDepth	number	3	Reply depth setting

Comment modes:

• none — Skip comments (fastest)
• all — Fetch for every post

Search Filters

Field	Type	Description
restrictToSubreddit	string	Limit search to one subreddit
authorFilter	string	Only posts by this username
flairFilter	string	Only posts with this flair
selfPostsOnly	boolean	Only text posts (no links)

Discover Options

Field	Type	Default	Description
estimateActivity	boolean	true	Calculate posts per day/week
includeNsfw	boolean	false	Include adult subreddits

Proxy Configuration

Field	Type	Default	Description
proxyConfiguration	object	Residential	Highly recommended. Reddit blocks without proxies.

Default proxy config (works well):

{
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Other Settings

Field	Type	Default	Description
tag	string	—	Label to identify this run's data

Advanced (rarely needed)

Field	Type	Default	Description
proxyCountry	string	US	ISO country code (US, GB, DE, etc.)
requestDelayMs	number	100	Milliseconds between requests
includeRaw	boolean	false	Include raw Reddit payloads (debugging)

---

🔧 Per-Target Overrides

Each mode supports easy arrays + optional advanced overrides. Example (scrape):

{
  "mode": "scrape",
  "sort": "hot",
  "maxPosts": 25,
  "includeCommentsMode": "none",
  "listings": [
    { "subreddit": "python" },
    {
      "subreddit": "AskReddit",
      "sort": "top",
      "timeframe": "day",
      "maxPosts": 25,
      "includeCommentsMode": "all"
    }
  ]
}

Here, r/python uses defaults, while r/AskReddit overrides sort/timeframe/comments.

---

📈 Run Metadata

Each run stores statistics in the OUTPUT key-value record:

{
  "mode": "scrape",
  "posts_scraped": 150,
  "comments_scraped": 2340,
  "http_stats": {
    "requests_made": 48,
    "requests_succeeded": 48,
    "rate_limits_hit": 0
  }
}

---

❓ Troubleshooting

"403 Forbidden" or "429 Too Many Requests"

Cause: Reddit is blocking your requests.

Fix: Enable Apify Proxy (residential works best):

{
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Empty results

Check:

• Subreddit name spelling (no r/ prefix needed)
• includeNsfw: true if searching NSFW content
• sort + timeframe combination (e.g., top with hour may have few results)
• Subreddit may be private or quarantined

Comments seem incomplete

Why: Reddit may return only a subset of very large comment sections.

What you get: Comment text, author, timestamps, scores where available, and permalinks.

Workaround: Increase maxTopLevelComments if you need more comments.

Slow performance

Tips:

• Set includeCommentsMode: "none" if you don't need comments
• Use includeCommentsMode: "all" only when comments are required
• Reduce scrape.maxPostsPerSubreddit / search.maxPostsPerQuery for faster testing
• All targets run in parallel automatically

Rate limits despite using proxies

Try:

• Increase requestDelayMs to 200-500
• Change proxyCountry to a different region (GB, DE, CA)
• The scraper automatically retries with exponential backoff

---

License

Apache-2.0