🤖 Reddit Scraper

Scrape Reddit posts, comments, and media from any subreddit or user profile. 🔓 No login required, no API keys needed.

🚀 What does Reddit Scraper do?

Reddit Scraper is a lightweight Apify Actor that extracts posts and comments from Reddit without using a browser. It uses Reddit's JSON API endpoints directly, making it fast and resource-efficient (runs on just 128-256MB memory).

Unlike browser-based scrapers, this Actor:

• ⚡ Starts instantly (no browser launch time)
• 💾 Uses minimal compute resources
• 🔄 Handles rate limiting automatically
• 💪 Supports resume on failure

📊 What data can you extract?

Data Type	Fields Extracted
📝 Posts	ID, title, author, subreddit, score, upvote ratio, comments count, post type, URL, text content, flair, awards, NSFW/spoiler flags
💬 Comments	ID, author, body text, score, depth, parent ID, timestamps
🖼️ Media	Image URLs, video URLs, gallery items, Reddit-hosted media

💡 Use cases

• 📈 Market Research - Monitor product discussions, brand mentions, and competitor analysis
• 🎭 Sentiment Analysis - Gather opinions on topics, products, or events for NLP processing
• 📰 Content Curation - Collect trending posts and media for content aggregation
• 🎓 Academic Research - Build datasets for social media studies and behavioral analysis
• 🎯 Lead Generation - Find potential customers discussing relevant topics
• 📊 Trend Monitoring - Track emerging topics and viral content in specific communities

🛠️ How to use Reddit Scraper

1. 🎯 Enter a target - Subreddit name (e.g., python, askreddit) or username
2. 🔢 Set the limit - How many posts to scrape (1-1000)
3. 📑 Choose sort order - new, hot, top, or rising
4. ⚙️ Enable options - Comments, media extraction, or media download
5. ▶️ Run the Actor - Results are saved to the dataset

📌 Example: Scrape r/python

{
  "target": "python",
  "limit": 100,
  "sort": "hot",
  "scrapeComments": true,
  "extractMediaUrls": true
}

👤 Example: Scrape a user's posts

{
  "target": "spez",
  "isUser": true,
  "limit": 50,
  "sort": "top",
  "timeframe": "year"
}

⚙️ Input parameters

Parameter	Type	Description	Default
target	string	Subreddit name or username (required)	-
isUser	boolean	Set true if target is a username	false
limit	integer	Maximum posts to scrape (1-1000)	10
sort	string	Sort order: new, hot, top, rising	new
timeframe	string	Time filter for top sort: hour, day, week, month, year, all	all
scrapeComments	boolean	Extract comments for each post	false
commentsLimit	integer	Max comments per post (1-500)	100
extractMediaUrls	boolean	Extract image/video URLs from posts	true
downloadMedia	boolean	Download media files to Key-Value Store	false
proxy	object	Proxy configuration (RESIDENTIAL recommended)	-

🌐 Proxy configuration

Reddit blocks most datacenter IPs. For reliable scraping, enable Apify Proxy with RESIDENTIAL group:

{
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}

📤 Output examples

📝 Post output

{
  "type": "post",
  "id": "1hj2abc",
  "title": "What's your favorite Python library?",
  "author": "pythonista123",
  "subreddit": "python",
  "created_utc": "2024-12-20T15:30:00",
  "permalink": "https://reddit.com/r/python/comments/1hj2abc/...",
  "url": "https://reddit.com/r/python/comments/1hj2abc/...",
  "score": 1542,
  "upvote_ratio": 0.96,
  "num_comments": 234,
  "num_crossposts": 5,
  "selftext": "I've been exploring different libraries and wanted to hear what everyone's using...",
  "post_type": "text",
  "is_nsfw": false,
  "is_spoiler": false,
  "flair": "Discussion",
  "total_awards": 3,
  "has_media": false,
  "media_urls": {
    "images": [],
    "videos": [],
    "galleries": []
  }
}

💬 Comment output

{
  "type": "comment",
  "comment_id": "kx7y9z",
  "post_permalink": "https://reddit.com/r/python/comments/1hj2abc/...",
  "post_title": "What's your favorite Python library?",
  "parent_id": "t3_1hj2abc",
  "author": "dev_guru",
  "body": "Pandas is absolutely essential for data work. Can't imagine doing analysis without it.",
  "score": 89,
  "created_utc": "2024-12-20T16:45:00",
  "depth": 0,
  "is_submitter": false
}

🖼️ Image post with media

{
  "type": "post",
  "id": "1hk3def",
  "title": "My home office setup",
  "post_type": "image",
  "has_media": true,
  "media_urls": {
    "images": [
      "https://i.redd.it/abc123.jpg",
      "https://preview.redd.it/xyz789.jpg"
    ],
    "videos": [],
    "galleries": []
  },
  "media_downloaded": true
}

💰 Cost estimation

This Actor uses minimal resources:

Memory	Speed	Cost per 1000 posts
128 MB	~2 posts/sec	~$0.05
256 MB	~3 posts/sec	~$0.08

💡 Note: Enabling scrapeComments or downloadMedia increases runtime and cost.

✅ Features

Feature	Supported
🏠 Subreddit scraping	✅ Yes
👤 User profile scraping	✅ Yes
💬 Comments extraction	✅ Yes
🔗 Media URL extraction	✅ Yes
📥 Media file download	✅ Yes
🖼️ Gallery support	✅ Yes
🔄 Resume on failure	✅ Yes
🌐 Proxy support	✅ Yes
🔓 No login required	✅ Yes
🔑 No API key needed	✅ Yes

⚙️ How it works

1. 📡 Fetches posts via Reddit's JSON API (old.reddit.com/r/{sub}.json)
2. 🔀 Falls back to mirrors if Reddit blocks the request (Redlib instances)
3. 🔍 Extracts structured data from JSON responses
4. 💬 Optionally fetches comments for each post
5. 📥 Optionally downloads media to Apify Key-Value Store
6. 💾 Saves progress for resume capability on long scrapes

⚠️ Limitations

• 🚦 Reddit rate limits apply (~100 requests per minute)
• 🔒 Some subreddits may be private or quarantined
• 📜 Very old posts (>1000 in listing) may not be accessible via pagination
• 📦 Media download is limited to 5 images + 2 videos per post to manage storage

❓ FAQ

📜 Is it legal to scrape Reddit?

Web scraping publicly available data is generally legal. However, always review Reddit's Terms of Service and robots.txt. This Actor only accesses public data and respects rate limits.

🌐 Why do I need a proxy?

Reddit blocks most datacenter IP addresses. Using Apify's RESIDENTIAL proxy group ensures reliable access.

🔄 How is this different from Reddit's official API?

Reddit's official API requires registration, has strict rate limits (100 requests/minute for OAuth), and recently introduced paid tiers. This Actor uses public JSON endpoints with no authentication needed.

🔒 Can I scrape private subreddits?

No, this Actor only accesses publicly available content.

📁 Where are downloaded media files stored?

Media files are saved to Apify's Key-Value Store with keys like media/{post_id}/image_0.jpg. You can access them via the Storage tab in Apify Console.