Reddit Scraper

Fast and reliable Reddit scraper that extracts posts and comments from any public subreddit. Data comes from the Arctic Shift Reddit archive, which picks up new posts and comments within minutes.

🚀 Key Features

• Simple Setup - Start scraping immediately with minimal configuration
• Fast & Efficient - Optimized for speed and reliability
• Rich Data - Extracts titles, text, scores, media, and engagement metrics
• Flexible Search - Browse subreddits, search by keyword, or scrape specific URLs
• Media Extraction - Automatically captures images and videos from posts

📋 What Data Can You Extract?

Each Reddit post/comment includes:

Field	Description
url	Permanent Reddit URL
id	Reddit unique identifier
title	Post title (posts only)
body	Text content
username	Author username
communityName	Subreddit name (e.g., r/python)
dataType	Content type: "post" or "comment"
score	Upvotes minus downvotes
upvote_ratio	Ratio of upvotes (0-1)
num_comments	Number of comments (posts only)
createdAt	ISO 8601 timestamp
isNsfw	NSFW flag
media	Array of image/video URLs (if present)

🎯 Use Cases

• Market Research - Monitor discussions about products, brands, or topics
• Sentiment Analysis - Analyze community opinions and trends
• Content Aggregation - Collect posts for newsletters or content curation
• Academic Research - Gather data for social media studies
• Trend Detection - Track emerging topics and viral content

💡 Input Configuration

Subreddit Scraping

Scrape posts from specific subreddits:

{
  "subreddits": ["python", "MachineLearning", "datascience"],
  "limit": 25,
  "sort": "new"
}

Parameters:

• subreddits (array, required) - List of subreddit names (without r/ prefix)
• limit (number, optional) - Posts per subreddit (1-100, default: 10)
• sort (string, optional) - Sort order: "new", "hot", "top" (default: "new"). Only "new" is exact; "hot" and "top" return the newest posts.
• keyword (string, optional) - Only posts whose title or text contains this keyword, newest first

Single URL Scraping

Scrape a specific post or comment by URL:

{
  "url": "https://www.reddit.com/r/python/comments/abc123/title/"
}

Parameters:

• url (string, required) - Full Reddit post or comment URL

📊 Output Example

[
  {
    "url": "https://www.reddit.com/r/bittensor_/comments/1or4vcv/james_altucher_on_bittensor/",
    "id": "t3_1or4vcv",
    "title": "James Altucher on Bittensor: The Only Tech That Matters Since Bitcoin",
    "body": "# Who Is James Altucher\n\nJames Altucher is not a typical crypto influencer...",
    "username": "Internal-Patience533",
    "communityName": "r/bittensor_",
    "dataType": "post",
    "parentId": null,
    "score": 29,
    "upvote_ratio": 1.0,
    "num_comments": 3,
    "createdAt": "2025-11-07T20:10:31Z",
    "isNsfw": false,
    "media": ["https://i.redd.it/2iyvqtsu6wzf1.png"]
  }
]

🔧 Advanced Usage

Keyword Filtering

Search for posts containing specific keywords:

{
  "subreddits": ["technology"],
  "keyword": "artificial intelligence",
  "limit": 50,
  "sort": "top"
}

Multiple Subreddits

Scrape from multiple communities in one run:

{
  "subreddits": ["python", "javascript", "golang", "rust"],
  "limit": 20,
  "sort": "hot"
}

📈 Performance

• Reliability: If the archive rate-limits a request, the actor retries it once from a different IP
• No proxy setup: The actor handles proxies itself

❓ Common Questions

Q: Can I scrape private subreddits? A: No, only public subreddits and posts are accessible.

Q: What about rate limits? A: The actor handles them. You don't need to configure a proxy.

Q: Can I scrape comments? A: Yes! The actor extracts both posts and can scrape individual comments via direct URL.

Q: How fresh is the data? A: New posts and comments appear within minutes. Score, comment count and upvote ratio are refreshed about 36 hours after posting, so very recent items can show early values (often a score of 1 and 0 comments).

---

Built with ❤️ by Macrocosmos