🔍 Reddit Search Scraper

Scrape Reddit search results and subreddit listings at scale — paste any Reddit URL (search, subreddit, or subreddit search) and the actor paginates Reddit's public JSON API, returns clean structured records, and live-saves each post to the dataset.

💡 Built for marketers, researchers, AI/LLM data pipelines, and competitive-intelligence teams who need clean, structured Reddit data without scraping headaches.

---

✨ Why choose this Actor?

• 🚀 Fast — pure async HTTP, no headless browser overhead.
• 🛡️ Smart proxy ladder — starts direct, auto-falls-back to datacenter → residential on 403/429/blocked responses, and stays on residential once it kicks in.
• 🔁 Resilient — per-request retries with jittered backoff, and 3 retries on the residential tier before giving up.
• 💾 Live saving — every post is pushed to the dataset as it's scraped, so a mid-run crash never loses work.
• 🧱 Bulk URLs — feed it any number of Reddit URLs in one run.
• 📊 Pre-built dataset views — Overview, Post, Subreddit, Author, Content, and Full Record tabs in the Apify Console.

---

🎯 Key features

• 🌐 Bulk URL input (search URLs, subreddit URLs, subreddit search URLs)
• 🔎 Optional keyword fallback when no URLs are supplied
• 📊 Sort by Relevance / Hot / Top / New / Most Comments
• 🔞 Safe-search toggle
• 📦 Hard cap on total items via maxItems
• 🛡️ Default no-proxy, auto-escalating fallback ladder
• 📝 Detailed real-time logs so you can watch progress live

---

📥 Input

{
  "urls": [
    { "url": "https://www.reddit.com/search/?q=ai&sort=new" },
    { "url": "https://www.reddit.com/r/python/" }
  ],
  "query": "artificial intelligence",
  "sort": "relevance",
  "safeSearch": "off",
  "maxItems": 300,
  "maxRetries": 3,
  "proxyConfiguration": { "useApifyProxy": false }
}

Field	Type	Description
urls	array	Reddit URLs to scrape (search, subreddit, or subreddit search).
query	string	Keyword fallback used only when urls is empty.
sort	enum	relevance / hot / top / new / comments.
safeSearch	enum	off (include NSFW) or on (hide NSFW).
maxItems	integer	Hard cap on total posts across all URLs.
maxRetries	integer	Per-request retries before escalating proxy tier.
proxyConfiguration	object	Standard Apify proxy input. Defaults to no proxy.

---

📤 Output

Each dataset record matches the original reference shape exactly, plus a few top-level mirror fields so the table views work without nested-path lookups:

{
  "post": {
    "title": "The more young people use AI, the more they hate it",
    "url": "https://www.reddit.com/r/technology/comments/1szusu6/the_more_young_people_use_ai_the_more_they_hate_it/",
    "score": 22036,
    "comment_count": 1612
  },
  "subreddit": { "name": "technology" },
  "author":    { "name": "spherocytes" },
  "contentText": "",
  "content_type": "link",
  "created_timestamp": "2026-04-30T12:34:21.000000+0000",

  "title": "The more young people use AI, the more they hate it",
  "subreddit_name": "technology",
  "author_name": "spherocytes",
  "score": 22036,
  "comment_count": 1612,
  "url": "https://www.reddit.com/r/technology/comments/1szusu6/the_more_young_people_use_ai_the_more_they_hate_it/"
}

---

🚀 How to use the Actor (via Apify Console)

1. 🔐 Log in at console.apify.com → Actors.
2. 🔎 Find Reddit Search Scraper and open it.
3. 📝 Paste one or more Reddit URLs (or type a keyword in the query field).
4. ⚙️ Pick a sort (Relevance / Hot / Top / New / Most Comments) and set maxItems.
5. 🛡️ Leave Proxy on default (no proxy) — the scraper auto-escalates if Reddit pushes back.
6. ▶️ Click Start.
7. 📊 Watch logs in real time; open the Output tab as records stream in.
8. 📁 Export to JSON / CSV / Excel.

---

🛡️ Proxy strategy

The scraper uses a three-tier ladder:

Tier	When it's used
🌐 Direct	Default — Reddit's public JSON API rarely needs a proxy.
🏢 Datacenter	Auto-engaged if direct requests get 403 / 429 / blocked.
🏠 Residential	Auto-engaged if datacenter still fails. Retries up to 3× then sticks for the rest of the run.

You can also start higher up the ladder by selecting a proxy group in the input.

---

💼 Best use cases

• 🤖 Building AI / LLM training datasets from Reddit discussion
• 📊 Brand monitoring & sentiment analysis
• 🧠 Market research and competitive intelligence
• 📝 Content trend discovery
• 🔬 Academic research on online communities

---

❓ Frequently asked questions

Q: Does it scrape comments? A: This actor returns post-level metadata (title, score, comment count, body text). For per-post comment threads, use an additional actor or extend this one to fetch <permalink>.json.

Q: Does it support private subreddits? A: No — only publicly accessible subreddits and search results.

Q: What happens if Reddit blocks me? A: The scraper auto-escalates the proxy tier and retries. If even residential fails after 3 retries, the run ends with a clear status message.

---

📨 Support and feedback

For issues, custom features, or feedback: dev.scraperengine@gmail.com

---

⚠️ Legal & ethical use

• Only collect data from publicly accessible Reddit pages.
• Respect Reddit's terms of service and applicable privacy laws (GDPR / CCPA).
• The end user is responsible for downstream use of the data.