Reddit Scraper: Extract Posts, Comments, Search & AI Datasets

What does Reddit Scraper do?

The Reddit Scraper extracts clean, structured data from public Reddit pages without requiring a Reddit developer account or API credentials. It allows you to extract Reddit posts, scrape Reddit comments, extract user profiles, and run automated subreddit search queries at scale.

This Actor is highly optimized to run fast and bypass Reddit's anti-bot restrictions directly. Output can be exported in formats like JSON, CSV, or Excel, as well as specialized formats ready for AI ingestion (Chat fine-tuning JSONL or RAG-friendly Markdown documents).

---

Who is it for?

Reddit Scraper is built for anyone who needs Reddit data at scale without dealing with Reddit's restrictive official API:

• 🔬 Researchers & Academics — collect public opinion data, survey sentiment on topics, and build datasets for academic studies.
• 📊 Market Analysts & Competitor Intelligence — track brand mentions, product feedback, and competitor discussions across subreddits.
• 📈 SEO & Content Marketers — discover trending topics, find content ideas, and monitor keyword discussions.
• 🤖 AI/ML Engineers — gather training data, build sentiment analysis datasets, or feed LLM pipelines with real conversations.
• 📰 Journalists & Media Monitors — monitor communities for breaking stories and track public reactions to events.
• 🏢 Product Managers — collect user feedback from product subreddits, track feature requests, and follow bug reports.
• 💼 Lead Generation Teams — find potential customers asking for recommendations or solutions in your industry.

---

Why use Reddit Scraper?

• ✅ No API Keys or Accounts Required: Works immediately without Reddit developer credentials.
• 🔎 Subreddit Listings & Keyword Search: Easily fetch hot, new, top, rising, or relevance listings.
• 📦 Recursive Comment Threads: Scrape nested reply trees up to the configured nesting level.
• ⚡ High-Performance Direct Crawling: Bypasses typical cloud/datacenter IP blocks using Crawlee's advanced got-scraping engine.
• 🧠 AI-Ready Exports: Native support for SFT fine-tuning (JSONL) and RAG Markdown chunks.
• 💰 Pay-per-Event Economy: Optimized for minimal execution overhead.

---

What data can you extract?

Post Fields:

Field	Type	Description
type	String	Item type: 'post'
id	String	Unique identifier of the post
title	String	Post title
author	String	Username of the post author
subreddit	String	Subreddit name
score	Integer	Net upvotes (best-effort, may be 0 due to public path limits)
upvoteRatio	Number	Upvote percentage (0-1, best-effort)
numComments	Integer	Total comment count
createdAt	String	ISO 8601 timestamp of post creation
url	String	Full direct URL to the post
permalink	String	Relative URL path to the post
selfText	String	Body text of the post
domain	String	Link domain (for external link posts)
isVideo	Boolean	True if the post is a video link
isSelf	Boolean	True if the post is a text post
isNSFW	Boolean	True if the post is marked NSFW
isSpoiler	Boolean	True if the post is marked as a spoiler
thumbnail	String	URL to the post thumbnail image
subredditSubscribers	Integer	Total subscriber count of the subreddit
scrapedAt	String	Timestamp when this item was extracted

Comment Fields:

Field	Type	Description
type	String	Item type: 'comment'
id	String	Unique identifier of the comment
postId	String	Parent post identifier
postTitle	String	Title of the parent post
author	String	Username of the commenter
body	String	Text body of the comment
score	Integer	Net upvotes of the comment (best-effort)
createdAt	String	ISO 8601 timestamp of comment creation
depth	Integer	Nesting reply depth level (0 = top-level comment)
parentId	String	ID of the parent comment or post
scrapedAt	String	Timestamp when this item was extracted

---

How much does it cost to scrape Reddit?

This Actor runs on a standard consumption model. With Apify's free tier credits, you can scrape thousands of pages for free every month.

• 1,000 subreddit listing posts costs approximately $0.15 in compute usage.
• 1,000 comment threads costs approximately $0.25 in compute usage.

---

How to use Reddit Scraper (Step-by-Step)

1. Go to the Reddit Scraper on the Apify Console.
2. Add Reddit links to the Reddit Target URLs field. Any of these formats work:
   • https://www.reddit.com/r/technology/ (or simply r/technology)
   • https://www.reddit.com/r/AskReddit/comments/abc123/post-title/
   • https://www.reddit.com/user/spez/
3. Or enter a query in the Search Query Keyword field to search across Reddit.
4. Configure optional filters such as Sorting Mode, Max Comments per Thread, or Output Format Mode.
5. Click Start and wait for results.
6. Download your data as JSON, CSV, or Excel from the Dataset tab.

Searching Reddit with Keyword Filtering

To narrow down search results to specific keywords, pass the filterKeywords parameter. The scraper will automatically discard any posts that do not contain at least one of the specified terms in the title or self text body:

{
  "searchQuery": "best project management tools",
  "searchSubreddit": "productivity",
  "sort": "relevance",
  "timeFilter": "month",
  "maxPostsPerSource": 50,
  "filterKeywords": ["Notion", "Asana", "Monday"]
}

---

Input Parameters Table

Parameter	Type	Default	Description
urls	Array	—	Reddit URLs to scrape (subreddits, posts, users).
searchQuery	String	—	Search Reddit for posts matching this keyword/phrase.
searchSubreddit	String	—	Limit search query to this specific subreddit.
sort	Enum	hot	Sort order: hot, new, top, rising, relevance.
timeFilter	Enum	week	Time filter for top/relevance: hour, day, week, month, year, all.
maxPostsPerSource	Integer	100	Max posts to retrieve per URL or search. 0 = unlimited.
includeComments	Boolean	false	Enable to crawl the comments for each scraped post.
maxCommentsPerPost	Integer	100	Cap the total number of comments scraped from a single post.
commentDepth	Integer	3	Maximum nesting level for comment replies (1 to 10).
outputFormat	Enum	default	Output format: default, jsonl-finetune, rag-markdown.
filterKeywords	Array	[]	Keep only posts containing at least one keyword (case-insensitive).

---

Advanced AI Formats

To speed up machine learning workflows, this Actor supports three output modes:

1. Standard JSON (default)

Backward-compatible flat structures for general analytics. Each post and comment is exported as a separate row in the dataset.

2. SFT Chat JSONL (jsonl-finetune)

Formats thread discussions into OpenAI chat-style (system / user / assistant) JSONL objects, ready to train and fine-tune custom AI models.

• System message: Analytical role instruction.
• User message: Subreddit name + Post title + Post body text.
• Assistant message: Ordered comments serialized as a single string.

{
  "type": "finetune",
  "messages": [
    { "role": "system", "content": "You are analyzing a Reddit discussion..." },
    { "role": "user", "content": "r/AppIdeas — National Park animal sighting tracker\n\nI just went on a safari in Africa..." },
    { "role": "assistant", "content": "1. [user_a, score: 5] It needs a very easy and seamless way to report sightings...\n2. [user_b, score: 2] Consider the serious ethical issues..." }
  ],
  "metadata": {
    "postId": "t3_1uqye4t",
    "subreddit": "AppIdeas",
    "score": 1,
    "url": "https://www.reddit.com/r/AppIdeas/comments/1uqye4t/national_park_animal_sighting_tracker/"
  }
}

3. RAG Markdown Chunks (rag-markdown)

Combines the post body and top comments into a single Markdown document with a stable chunkId and flat metadata, optimized for vector database ingestion (Pinecone, Chroma, Qdrant).

{
  "type": "rag-chunk",
  "chunkId": "reddit-AppIdeas-t3_1uqye4t",
  "markdown": "# National Park animal sighting tracker\n\n**Subreddit:** r/AppIdeas\n**Author:** u/gdubrocks\n**Score:** 1\n\nI just went on a safari...\n\n## Top Comments\n\n### u/user_a (score: 5)\n\nIt needs a very easy and seamless way...\n\n### u/user_b (score: 2)\n\nConsider the serious ethical issues...",
  "metadata": {
    "source": "reddit",
    "subreddit": "AppIdeas",
    "postId": "t3_1uqye4t",
    "title": "National Park animal sighting tracker",
    "score": 1,
    "url": "https://www.reddit.com/r/AppIdeas/comments/1uqye4t/national_park_animal_sighting_tracker/"
  }
}

---

API Usage and Programmatic Integration

You can easily trigger Reddit Scraper programmatically using Apify's API and client libraries in Node.js, Python, or standard terminal commands.

Node.js (using apify-client)

Install client via npm install apify-client and invoke:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('karamelo/reddit-scraper').call({
    urls: ['https://www.reddit.com/r/technology/'],
    maxPostsPerSource: 100,
    sort: 'hot',
    includeComments: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python (using apify-client)

Install client via pip install apify-client and invoke:

from apify_client import ApifyClient

client = ApifyClient('YOUR_API_TOKEN')

run = client.actor('karamelo/reddit-scraper').call(run_input={
    'urls': ['https://www.reddit.com/r/technology/'],
    'maxPostsPerSource': 100,
    'sort': 'hot',
    'includeComments': False,
})

items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL command-line integration

Start the scraper run via HTTP POST:

curl "https://api.apify.com/v2/acts/karamelo~reddit-scraper/runs" \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -d '{
    "urls": ["https://www.reddit.com/r/technology/"],
    "maxPostsPerSource": 100,
    "sort": "hot"
  }'

To fetch items in JSON format once the run completes:

curl "https://api.apify.com/v2/datasets/DATASET_ID/items?format=json" \
  -H "Authorization: Bearer YOUR_API_TOKEN"

---

Use with AI Agents via Model Context Protocol (MCP)

This Scraper is designed to be accessible directly inside LLM tool pipelines and AI assistants.

Claude Code Setup

$claude mcp add --transport http apify "https://mcp.apify.com?tools=karamelo/reddit-scraper"

Claude Desktop / Cursor Setup

Add the server entry to your config file:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=karamelo/reddit-scraper"
    }
  }
}

Agent Integration via x402 (No Apify Account Required)

To run the Actor directly from agent wallets:

https://mcp.apify.com?tools=karamelo/reddit-scraper&payment=x402

---

FAQ, Disclaimers, and Support

⚖️ Legal Disclaimer: Our scraper only accesses publicly available posts and comments shared by users who chose to share them publicly. It does not log in, bypass any authentication barriers, or access private data. The US Ninth Circuit ruling in hiQ Labs v. LinkedIn (2022) established that scraping publicly available data does not violate the Computer Fraud and Abuse Act (CFAA). GDPR laws apply when storing personal data (such as usernames). Make sure you have a legitimate reason to scrape and store this data.

For feedback or feature requests, visit the Issues tab on the Actor page.