Reddit Scraper

Scrape Reddit subreddits, posts with comments, user profiles, and search results by parsing server-rendered HTML. Works after the May 2026 .json API shutdown.

Why This Actor Exists

Reddit shut down its public .json API in May 2026, breaking many existing scrapers that relied on appending .json to Reddit URLs. This actor parses Reddit's server-rendered HTML directly — no API key, no .json endpoint, no authentication required.

Features

• Subreddit scraping — Get posts from any subreddit sorted by hot, new, top, or rising
• Post + comments — Scrape a single post with all its comments and replies
• User profiles — Get a user's posts and comments
• Search — Search Reddit for keywords, optionally restricted to a subreddit
• Pagination — Uses Reddit's after cursor for unlimited scrolling
• Time filtering — Filter top posts by hour, day, week, month, year, or all time
• Proxy support — Apify proxy integration for avoiding rate limits

Input

Field	Type	Required	Default	Description
mode	select	Yes	subreddit	subreddit, post, user, or search
subreddit	string	No	—	Subreddit name (without r/) for subreddit and search modes
username	string	No	—	Reddit username for user mode
postUrl	string	No	—	Full Reddit post URL for post mode
searchQuery	string	No	—	Search query for search mode
sort	select	No	hot	hot, new, top, rising
timeFilter	select	No	day	hour, day, week, month, year, all (top sort only)
maxResults	integer	No	100	Max results (1-1000)
proxyConfiguration	proxy	No	on	Apify proxy recommended

Output

Each result contains: title, author, score, upvote_ratio, num_comments, created_utc, url, selftext, subreddit, flair, permalink, and (for post mode) nested comments.

Technical Approach

• Fetches HTML from old.reddit.com (simpler HTML, fewer anti-bot measures)
• Falls back to www.reddit.com if old.reddit.com fails
• Parses embedded JSON data from <script> tags when available
• Falls back to HTML parsing with BeautifulSoup for structured extraction
• Uses browser-like User-Agent to avoid bot detection
• Handles 429 rate limiting with exponential backoff
• No external APIs, no API keys, no paid services