Reddit Full Scraper

Scrapes Reddit posts (and optionally comments) using Reddit's free public JSON API. No authentication required. Fast and lightweight — built with httpx, no browser, no Playwright.

Features

• Subreddit listings — scrape posts from any subreddit (hot, new, top, rising, controversial)
• Individual post URLs — scrape a single post with full details
• Search queries — search Reddit by keyword, optionally restricted to a subreddit
• Comment extraction — optionally extract comments (including nested replies)
• Pagination — automatically paginates through all available results up to your limit
• Time filters — for top/controversial sorting: hour, day, week, month, year, all
• No auth required — uses Reddit's public JSON API, no API key needed
• Rate-limit aware — automatic retry with backoff on 429 responses

Input

Field	Type	Default	Description
subredditUrls	array of strings	[]	Subreddit URLs to scrape (e.g., ["https://www.reddit.com/r/python/"])
postUrls	array of strings	[]	Individual post URLs (e.g., ["https://www.reddit.com/r/python/comments/abc123/"])
searchQueries	array of strings	[]	Search terms to find posts
searchSubreddit	string	null	Restrict search to a specific subreddit (e.g., "python"). Only used with searchQueries.
sort	string	"hot"	Sort order for listings and search: hot, new, top, rising, controversial, relevance
timeFilter	string	"all"	Time filter for top/controversial: hour, day, week, month, year, all
maxPosts	integer	100	Maximum number of posts to scrape per subreddit/search
extractComments	boolean	false	If true, extract comments when scraping individual post URLs
maxComments	integer	50	Maximum number of top-level comments to extract per post

Output

Each result is a dataset item with these fields:

Field	Type	Description
id	string	Reddit post ID
subreddit	string	Subreddit name
title	string	Post title
author	string	Author username
score	integer	Upvote score
upvoteRatio	number	Upvote ratio (e.g., 0.95)
numComments	integer	Number of comments
content	string	Post body text (selftext) or link URL
url	string	Full Reddit post URL
permalink	string	Relative permalink path
timestamp	number	Unix timestamp (UTC)
isSelf	boolean	Is a self/text post
domain	string	Domain for link posts
linkFlairText	string	Flair text
over18	boolean	NSFW flag
spoiler	boolean	Spoiler flag
stickied	boolean	Stickied/pinned flag
thumbnail	string	Thumbnail URL
awardCount	integer	Total awards received
comments	array	(Only if extractComments=true) List of comment objects with id, author, body, score, timestamp, depth, replies

Example Usage

Scrape a subreddit:

{
  "subredditUrls": ["https://www.reddit.com/r/python/"],
  "sort": "top",
  "timeFilter": "week",
  "maxPosts": 50
}

Scrape specific posts with comments:

{
  "postUrls": [
    "https://www.reddit.com/r/python/comments/abc123/",
    "https://www.reddit.com/r/javascript/comments/def456/"
  ],
  "extractComments": true,
  "maxComments": 100
}

Search all of Reddit:

{
  "searchQueries": ["python tutorial", "machine learning"],
  "sort": "relevance",
  "maxPosts": 25
}

Search within a subreddit:

{
  "searchQueries": ["async"],
  "searchSubreddit": "python",
  "sort": "top",
  "timeFilter": "year",
  "maxPosts": 30
}

Technical Details

• Built with httpx (async HTTP) — fast and lightweight
• Runs on apify/actor-python:3.12 Docker image
• No API key, no OAuth, no browser required
• Automatically handles Reddit's rate limiting with exponential backoff
• Polite pagination with small delays between requests
• Supports old.reddit.com, new.reddit.com, and www.reddit.com URL formats