Reddit Subreddit Posts Scraper

Collect normalized public posts from one Reddit community with bounded pagination and optional post/comment enrichment. Choose the listing order and time window, cap saved rows, and receive a clean dataset plus an explicit run summary.

What this Actor does

• Reads a public subreddit listing through Reddit's public web representation.
• Supports new, hot, top, rising, and controversial ordering.
• Normalizes post identity, community, engagement, content, media, and provenance fields.
• Optionally requests public post detail data and up to 20 top-level comments.
• Stops at login, CAPTCHA, WAF, geofence, device, or rate-limit controls without bypassing them.

Input

Field	Type	Default	Description
subreddit	string	required	Community name without r/; 2–21 letters, numbers, or underscores.
sortBy	string	new	new, hot, top, rising, or controversial.
topTime	string	week	Time window for top or controversial.
maxItems	integer	50	Saved-post cap from 1 to 1,000.
deepScraping	boolean	true	Request each public post representation for richer details.
includeComments	boolean	false	Include bounded public comments; automatically enables detail requests.
headless	boolean	true	Run Chromium without a visible window.
proxyConfiguration	object	Apify Proxy	Standard Apify proxy configuration. A proxy does not bypass access controls.

Unknown fields and invalid values fail before crawling.

Example input

{
  "subreddit": "javascript",
  "sortBy": "new",
  "topTime": "week",
  "maxItems": 10,
  "deepScraping": false,
  "includeComments": false,
  "proxyConfiguration": { "useApifyProxy": true }
}

Dataset output

Every dataset item is a real post record with recordType: "subreddit_post". Required fields are recordType, position, postId, postFullname, postTitle, permalink, subreddit, sourceListingUrl, and scrapedAt. postId is bare and postFullname is the corresponding t3_<postId> value. Optional fields include author, scores, timestamps, post body, flair, media, awards, and comments.

{
  "recordType": "subreddit_post",
  "position": 1,
  "postId": "example123",
  "postFullname": "t3_example123",
  "postTitle": "Example discussion post",
  "permalink": "https://www.reddit.com/r/example/comments/example123/example_post/",
  "subreddit": "example",
  "score": 42,
  "numComments": 8,
  "sourceListingUrl": "https://www.reddit.com/r/example/new/?sort=new&t=week",
  "extractionMethod": "reddit-subreddit-rendered-json-or-dom",
  "scrapedAt": "2025-01-02T03:04:05.000Z"
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

The default key-value store contains OUTPUT, a JSON run summary with the requested scope, settings, records stored, failures, blocked requests, and an explicit dataAvailable flag. If no usable posts are available, the default dataset contains one four-field diagnostic record; diagnostics are never represented as successful dataset rows.

Limits and cost control

The Actor uses one listing request at a time, no blocked-request retries, bounded pagination, and at most three concurrent detail tabs. Disable deepScraping when listing fields are sufficient. Usage may include browser compute, proxy traffic, and the configured per-dataset-item event price.

Public-data and responsible use

This Actor reads only publicly available Reddit web data. It does not log in, alter browser identity, defeat challenges, use protected sessions, or evade access controls. Availability varies by Reddit, network, region, and proxy tier. Follow Reddit's terms and applicable privacy and data-protection requirements.

Local development

npm test
apify validate-schema
apify run --purge

The dataset validator rejects incomplete or malformed post rows and treats an empty dataset as a failed extraction check.