Reddit Scraper

What does Reddit Scraper do?

Reddit Scraper extracts posts, comments, and metadata from Reddit using the free Reddit JSON API. It supports three modes: browsing a subreddit by sort order (hot, new, top, rising), searching Reddit for keywords, and pulling post history for a user. No API key or authentication required — just append .json to any Reddit URL.

Powered by the Apify platform, this Actor runs in the cloud with built-in scheduling, monitoring, and data export. Results land in a structured dataset you can download as JSON, CSV, Excel, or use via the Apify API.

Use Cases

Market research and trend tracking

Monitor what developers are saying about your product, framework, or technology. Scrape /r/programming or /r/javascript by top posts weekly to spot emerging tools, pain points, and sentiment trends. The structured output (score, upvote ratio, comment count) gives you a quantitative signal on topic heat.

Community management

Track mentions of your brand or project across subreddits using the search mode. Filter by time range to catch conversations in the past day or week. Identify high-engagement threads where you should respond, and export the data for your CRM or social listening pipeline.

Content curation and aggregation

Build a feed of top posts from multiple subreddits. Scrape each subreddit in separate runs, combine the datasets, and publish the result as a newsletter, blog roundup, or Discord bot feed. The include_comments option enriches each post with its top 5 comments for fuller context.

Academic and social media research

Researchers studying online communities, discourse patterns, or information diffusion can collect Reddit data at scale. The clean JSON output with timestamps, scores, and thread metadata is ready for statistical analysis, NLP pipelines, or network graphs.

Competitive intelligence

Monitor competitor subreddits or industry-specific communities for product announcements, customer complaints, and feature requests. The upvote ratio and comment count provide a proxy for community sentiment. Schedule daily runs via Apify to catch new posts as they appear.

How to use Reddit Scraper

1. Install dependencies — Run pip install -r requirements.txt in a virtual environment.
2. Create an input file — Copy test-input.json or create your own. At minimum, set the mode field to subreddit, search, or user.
3. Run locally — Use apify run --input-file=test-input.json. The Actor will fetch posts from Reddit and push results to the local dataset under storage/datasets/.
4. Adjust settings — Tweak sort, time_range, and max_results to control what gets scraped. Enable include_comments for deeper posts.
5. Deploy to Apify — Run apify push to deploy your Actor to the Apify platform. From there you can schedule runs, trigger via API, and connect integrations like Zapier, Make, or Google Drive.

Input

The Actor accepts a JSON object with the following fields (configured in the Apify Console Input tab):

Field	Type	Default	Description
mode	string	subreddit	Scraping mode: subreddit, search, or user
subreddit	string	programming	Subreddit name (without r/), used in subreddit mode
query	string	python	Search keyword, used in search mode
username	string	spez	Reddit username (without u/), used in user mode
sort	string	hot	Sort order: hot, new, top, rising, controversial, relevance
time_range	string	month	Time filter for top/controversial: hour, day, week, month, year, all
max_results	integer	25	Max posts to scrape (1–100)
include_comments	boolean	false	Whether to fetch top 5 comments per post (increases API calls)

Example input

{
    "mode": "subreddit",
    "subreddit": "python",
    "sort": "top",
    "time_range": "month",
    "max_results": 10,
    "include_comments": true
}

Output

Each result is a JSON object pushed to the dataset. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel from the Apify Console.

Output example

{
    "title": "PEP 750 – Tag Strings For Writing Domain-Specific Languages",
    "url": "https://peps.python.org/pep-0750/",
    "permalink": "https://www.reddit.com/r/Python/comments/1jskkwi/pep_750_tag_strings_for_writing_domainspecific/",
    "author": "pythonhub",
    "score": 432,
    "upvote_ratio": 0.92,
    "num_comments": 89,
    "created_at": "2025-04-03T14:22:10+00:00",
    "subreddit": "Python",
    "selftext": "",
    "flair": null,
    "top_comments": [
        {"author": "dev_user", "body": "This is exciting for template libraries.", "score": 45, "created_at": "2025-04-03T15:00:00+00:00"}
    ]
}

Data fields

Field	Type	Description
title	string	Post title
url	string	Post URL (external link or Reddit permalink)
permalink	string	Reddit permalink to the post
author	string	Reddit username of the poster
score	integer	Net upvotes — downvotes
upvote_ratio	number	Upvote percentage (0.0–1.0)
num_comments	integer	Total comment count
created_at	string	ISO 8601 timestamp
subreddit	string	Source subreddit name
selftext	string	Post body text (truncated to 500 characters)
flair	string	Post flair text, if any
top_comments	array	Up to 5 top comments (only if include_comments is enabled)

Pricing / Cost estimation

How much does it cost to scrape Reddit? The Actor uses the free Reddit JSON API with zero third-party proxy costs. Apify platform charges are based on compute units consumed.

• Free tier: First 10 results per run are free of additional per-result charges.
• Per-result charge: Beyond 10 results, a small PPE (Pay Per Event) fee of $0.003 per result applies, billed via Actor.charge(). At 100 results, that's about $0.27 in PPE charges.
• Compute units: A run scraping 25 posts typically completes in a few seconds, using under 0.01 compute units. A full 100-post run with comments might use 0.02–0.05 compute units depending on response times.
• No proxy needed: Unlike many website scrapers, Reddit's JSON API does not block requests with a proper User-Agent header, so no Apify proxy is required — further reducing costs.

For most use cases (scheduled daily runs of 25–50 posts), the cost is well under $0.10 per month in compute + PPE fees.

Tips and Advanced options

• Max results and pagination: Reddit's API returns 25 posts per page by default. The Actor automatically handles pagination via the after cursor — setting max_results: 100 will fetch up to 4 pages. Each extra page is one additional API call.
• User-Agent header: Reddit may rate-limit requests with default Python HTTP client identifiers. This Actor sends ApifyActor/1.0 as the User-Agent. If you encounter rate limiting, add a custom UA or increase the interval between runs.
• Comment mode: Enable include_comments sparingly. Every post with comments enabled triggers an additional API call (up to 5 per post). For a 100-post scrape, that's up to 100 extra requests — still manageable but worth noting for rate-limit awareness.
• Sort relevance: The relevance sort option only works in search mode. In subreddit or user mode, the API ignores it and falls back to the default sort.
• NSFW content: The Reddit JSON API returns NSFW posts by default in subreddit and search results. The over_18 field is available in the raw response data if you need to filter.

FAQ, disclaimers, and support

Is scraping Reddit legal?

Reddit's Terms of Service permit automated access via their official API (including the .json endpoint). This Actor sends a descriptive User-Agent header, respects the structure of the API, and does not circumvent any access controls. Use responsibly: keep request volumes reasonable, cache results where possible, and respect Reddit's rate limits (60 requests per minute for authenticated clients, lower for unauthenticated).

What data can this scrape?

The Actor extracts what Reddit exposes through its JSON API: post titles, URLs, scores, comments (first 5), author names, and metadata. It cannot scrape private subreddits, user DM history, or deleted content.

Known limitations

• Max 100 results per run (Reddit API hard limit per page).
• Comments are limited to the top 5 per post.
• Reddit may change their API response structure; if output fields are unexpectedly empty, check the actor logs.
• Rate limiting kicks in around 60+ rapid requests. For large scrapes, add delays between runs or distribute across multiple Actor runs.

Need help?

Report issues or suggest features on the GitHub Issues page. For custom scraping solutions or enterprise use, contact the Apify team.