Reddit Scraper

Reddit Scraper

Scrape Reddit posts from subreddit feeds and search results — titles, authors, scores, comment counts, URLs, flair, and full post bodies — without an API key. The Actor runs on Apify, giving you API access, scheduling, monitoring, and proxy rotation out of the box. Just set a subreddit or a search query and get a clean, structured dataset of posts.

This Actor uses the server-rendered HTML of old.reddit.com, so it works even where Reddit's JSON API endpoints are blocked for your IP.

Why use this Actor?

• No API keys or OAuth setup — Reddit's public JSON endpoints are frequently blocked, but this Actor reads the HTML directly.
• Feed or search — scrape hot, new, top, rising from any subreddit, or run keyword searches with time filters.
• Full post data — score, comment count, author, permalink, external URL, domain, flair, NSFW/spoiler flags, and optional full selftext bodies.
• Built to be cheap — plain HTTP requests, no browser needed; paginates efficiently and stops when it hits your maxItems limit.
• Apify platform advantages — schedule it to run on a cron, call it from the API, and download results as JSON, CSV, or Excel.

How to use this Actor

1. Open the Actor in Apify Console, select Input in the left sidebar.
2. Choose a Mode: Subreddit feed (posts from a subreddit) or Search (keyword search).
3. Enter a Subreddit (e.g. python, no r/ prefix), a Search query, and optionally adjust Sort order, Time filter, and Maximum items.
4. Click Start. The run will show a live log; results land in the Dataset tab.
5. Download the dataset in JSON, HTML, CSV, or Excel, or fetch it via the Apify API.

Reddit OAuth credentials (recommended)

The Actor uses Reddit's official JSON API whenever credentials are available — faster, more structured, and immune to the login-gating Reddit now applies to old.reddit.com. Without credentials it falls back to scraping the legacy HTML UI, which can be login-blocked.

Credentials are read as REDDIT_CLIENT_ID_n / REDDIT_KEY_n pairs (n = 1, 2, ...). Sources, in order:

1. Environment variables (REDDIT_CLIENT_ID_1, REDDIT_KEY_1, ...) — set them in Apify Console under the Actor's Environment variables, or locally in the shell.
2. Secrets file (local dev only) — by default ../secrets/reddit_keys.env next to the repo, overridable with the REDDIT_KEYS_FILE variable. Never commit this file.

Pairs are tried in order with a cascade of auth constructions (client_credentials both ways, refresh_token); a pair that returns 401 is skipped and the next is tried, so an expired key in the pool does not break the run. Cloud deployments may use apify secrets add to store a single pair and reference it via environment variables.

Input

Field	Type	Description
mode	enum	subreddit (feed) or search (keyword search)
subreddit	string	Subreddit name without the r/ prefix. Empty in search mode = search all of Reddit
query	string	Search term (search mode only)
sort	enum	Feed: hot, new, top, rising. Search: relevance, hot, new, top, comments
timeFilter	enum	hour, day, week, month, year, all (used by top and search)
maxItems	integer	Stop after scraping this many posts (default 1000)
includeSelfText	boolean	Include post bodies. Search results always include them; feed mode fetches one extra page per self-post
proxyConfiguration	object	Optional Apify proxy routing for IPs blocked by Reddit

Output

One dataset record per post:

{
  "type": "post",
  "id": "1ncuv8k",
  "title": "Best web scraping tools I've tried",
  "author": "DenOmania",
  "subreddit": "automation",
  "permalink": "https://old.reddit.com/r/automation/comments/1ncuv8k/...",
  "url": "https://example.com/article",
  "domain": "example.com",
  "score": 155,
  "numComments": 188,
  "createdUtc": "1785859525000",
  "createdAt": "2025-09-09T20:55:19+00:00",
  "flair": "Discussion",
  "spoiler": false,
  "nsfw": false,
  "stickied": false,
  "selftext": "I've gone through quite a few tools..."
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data table

Field	Type	Description
id	string	Post ID (t3_ suffix)
title	string	Post title
author	string	Reddit username of the author
subreddit	string	Subreddit the post was submitted to
permalink	string	URL of the post on Reddit
url	string	External URL (link posts)
domain	string	Domain of the external URL
score	number	Upvote score at scrape time
numComments	number	Comment count at scrape time
createdAt	string	ISO 8601 creation timestamp
flair	string	Link flair text
spoiler / nsfw / stickied	boolean	Content and listing flags
selftext	string	Full post body (when enabled / available)

Pricing / Cost estimation

How much does it cost to scrape Reddit with this Actor? Runs are cheap: one listing page returns ~25 posts, so 1,000 posts costs roughly 40–45 page requests plus a few KB of compute. On the Apify free tier you get 5 USD of monthly credits — a 1,000-post scrape typically costs well under a cent of compute. Use maxItems to keep large runs predictable. Reddit rate-limits aggressive requests, so the Actor spaces requests out automatically (exponential backoff on 429/403); for heavy or blocked runs, enable the Proxy configuration input.

Tips / Advanced options

• Search all of Reddit: leave subreddit empty in search mode.
• Watch for rate limits: if runs hit repeated 429, enable the Apify proxy in the Input tab — it rotates IPs.
• Limit cost: set a maxItems cap; the Actor stops paginating as soon as it's reached.
• Scheduling: use Apify Scheduler to re-scrape a subreddit daily and diff new posts.
• Selftext in feeds: body fetching adds one request per self-post; disable includeSelfText for large feed crawls.

FAQ / Disclaimers

• Is scraping Reddit allowed? Respect Reddit's Terms of Service and Reddit's rules. The Actor reads public data at a polite rate and honors rate-limit signals; use it responsibly and don't republish personal data without permission.
• Why old.reddit.com? Reddit blocks the .json API endpoints from many IPs. The old layout serves the same public data as server-rendered HTML, which is far more reliably accessible.
• Known limitations: live-streaming (continuously following /new) and user-profile scraping are not supported yet; comment scraping is planned.
• Issues or feature requests? Open one on the Issues tab. For custom scrapers and automation, contact us via the Apify account.