# Reddit Scraper (`ef12/reddit-scraper`) Actor Scrape Reddit posts and comments by subreddit, search query, or user. Get titles, scores, upvote ratios, comment counts, post bodies, and flairs via the Reddit JSON API. - **URL**: https://apify.com/ef12/reddit-scraper.md - **Developed by:** [Daniel Wilson](https://apify.com/ef12) (community) - **Categories:** Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Reddit Scraper ### What does Reddit Scraper do? **Reddit Scraper** extracts posts, comments, and metadata from Reddit using the free [Reddit JSON API](https://www.reddit.com/dev/api/). It supports three modes: browsing a **subreddit** by sort order (hot, new, top, rising), searching Reddit for **keywords**, and pulling post history for a **user**. No API key or authentication required — just append `.json` to any Reddit URL. Powered by the [Apify platform](https://apify.com), this Actor runs in the cloud with built-in scheduling, monitoring, and data export. Results land in a structured dataset you can download as JSON, CSV, Excel, or use via the Apify API. ### Use Cases #### Market research and trend tracking Monitor what developers are saying about your product, framework, or technology. Scrape `/r/programming` or `/r/javascript` by top posts weekly to spot emerging tools, pain points, and sentiment trends. The structured output (score, upvote ratio, comment count) gives you a quantitative signal on topic heat. #### Community management Track mentions of your brand or project across subreddits using the search mode. Filter by time range to catch conversations in the past day or week. Identify high-engagement threads where you should respond, and export the data for your CRM or social listening pipeline. #### Content curation and aggregation Build a feed of top posts from multiple subreddits. Scrape each subreddit in separate runs, combine the datasets, and publish the result as a newsletter, blog roundup, or Discord bot feed. The `include_comments` option enriches each post with its top 5 comments for fuller context. #### Academic and social media research Researchers studying online communities, discourse patterns, or information diffusion can collect Reddit data at scale. The clean JSON output with timestamps, scores, and thread metadata is ready for statistical analysis, NLP pipelines, or network graphs. #### Competitive intelligence Monitor competitor subreddits or industry-specific communities for product announcements, customer complaints, and feature requests. The upvote ratio and comment count provide a proxy for community sentiment. Schedule daily runs via Apify to catch new posts as they appear. ### How to use Reddit Scraper 1. **Install dependencies** — Run `pip install -r requirements.txt` in a virtual environment. 2. **Create an input file** — Copy `test-input.json` or create your own. At minimum, set the `mode` field to `subreddit`, `search`, or `user`. 3. **Run locally** — Use `apify run --input-file=test-input.json`. The Actor will fetch posts from Reddit and push results to the local dataset under `storage/datasets/`. 4. **Adjust settings** — Tweak `sort`, `time_range`, and `max_results` to control what gets scraped. Enable `include_comments` for deeper posts. 5. **Deploy to Apify** — Run `apify push` to deploy your Actor to the Apify platform. From there you can schedule runs, trigger via API, and connect integrations like Zapier, Make, or Google Drive. ### Input The Actor accepts a JSON object with the following fields (configured in the Apify Console Input tab): | Field | Type | Default | Description | | --- | --- | --- | --- | | `mode` | string | `subreddit` | Scraping mode: `subreddit`, `search`, or `user` | | `subreddit` | string | `programming` | Subreddit name (without r/), used in subreddit mode | | `query` | string | `python` | Search keyword, used in search mode | | `username` | string | `spez` | Reddit username (without u/), used in user mode | | `sort` | string | `hot` | Sort order: `hot`, `new`, `top`, `rising`, `controversial`, `relevance` | | `time_range` | string | `month` | Time filter for top/controversial: `hour`, `day`, `week`, `month`, `year`, `all` | | `max_results` | integer | `25` | Max posts to scrape (1–100) | | `include_comments` | boolean | `false` | Whether to fetch top 5 comments per post (increases API calls) | #### Example input ```json { "mode": "subreddit", "subreddit": "python", "sort": "top", "time_range": "month", "max_results": 10, "include_comments": true } ```` ### Output Each result is a JSON object pushed to the dataset. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel from the Apify Console. #### Output example ```json { "title": "PEP 750 – Tag Strings For Writing Domain-Specific Languages", "url": "https://peps.python.org/pep-0750/", "permalink": "https://www.reddit.com/r/Python/comments/1jskkwi/pep_750_tag_strings_for_writing_domainspecific/", "author": "pythonhub", "score": 432, "upvote_ratio": 0.92, "num_comments": 89, "created_at": "2025-04-03T14:22:10+00:00", "subreddit": "Python", "selftext": "", "flair": null, "top_comments": [ {"author": "dev_user", "body": "This is exciting for template libraries.", "score": 45, "created_at": "2025-04-03T15:00:00+00:00"} ] } ``` ### Data fields | Field | Type | Description | | --- | --- | --- | | `title` | string | Post title | | `url` | string | Post URL (external link or Reddit permalink) | | `permalink` | string | Reddit permalink to the post | | `author` | string | Reddit username of the poster | | `score` | integer | Net upvotes — downvotes | | `upvote_ratio` | number | Upvote percentage (0.0–1.0) | | `num_comments` | integer | Total comment count | | `created_at` | string | ISO 8601 timestamp | | `subreddit` | string | Source subreddit name | | `selftext` | string | Post body text (truncated to 500 characters) | | `flair` | string | Post flair text, if any | | `top_comments` | array | Up to 5 top comments (only if `include_comments` is enabled) | ### Pricing / Cost estimation How much does it cost to scrape Reddit? The Actor uses the free Reddit JSON API with zero third-party proxy costs. Apify platform charges are based on compute units consumed. - **Free tier**: First 10 results per run are free of additional per-result charges. - **Per-result charge**: Beyond 10 results, a small PPE (Pay Per Event) fee of $0.003 per result applies, billed via `Actor.charge()`. At 100 results, that's about $0.27 in PPE charges. - **Compute units**: A run scraping 25 posts typically completes in a few seconds, using under 0.01 compute units. A full 100-post run with comments might use 0.02–0.05 compute units depending on response times. - **No proxy needed**: Unlike many website scrapers, Reddit's JSON API does not block requests with a proper User-Agent header, so no Apify proxy is required — further reducing costs. For most use cases (scheduled daily runs of 25–50 posts), the cost is well under $0.10 per month in compute + PPE fees. ### Tips and Advanced options - **Max results and pagination**: Reddit's API returns 25 posts per page by default. The Actor automatically handles pagination via the `after` cursor — setting `max_results: 100` will fetch up to 4 pages. Each extra page is one additional API call. - **User-Agent header**: Reddit may rate-limit requests with default Python HTTP client identifiers. This Actor sends `ApifyActor/1.0` as the User-Agent. If you encounter rate limiting, add a custom UA or increase the interval between runs. - **Comment mode**: Enable `include_comments` sparingly. Every post with comments enabled triggers an additional API call (up to 5 per post). For a 100-post scrape, that's up to 100 extra requests — still manageable but worth noting for rate-limit awareness. - **Sort relevance**: The `relevance` sort option only works in `search` mode. In subreddit or user mode, the API ignores it and falls back to the default sort. - **NSFW content**: The Reddit JSON API returns NSFW posts by default in subreddit and search results. The `over_18` field is available in the raw response data if you need to filter. ### FAQ, disclaimers, and support #### Is scraping Reddit legal? Reddit's [Terms of Service](https://www.redditinc.com/policies/user-agreement) permit automated access via their official API (including the `.json` endpoint). This Actor sends a descriptive User-Agent header, respects the structure of the API, and does not circumvent any access controls. Use responsibly: keep request volumes reasonable, cache results where possible, and respect Reddit's rate limits (60 requests per minute for authenticated clients, lower for unauthenticated). #### What data can this scrape? The Actor extracts what Reddit exposes through its JSON API: post titles, URLs, scores, comments (first 5), author names, and metadata. It cannot scrape private subreddits, user DM history, or deleted content. #### Known limitations - Max 100 results per run (Reddit API hard limit per page). - Comments are limited to the top 5 per post. - Reddit may change their API response structure; if output fields are unexpectedly empty, check the actor logs. - Rate limiting kicks in around 60+ rapid requests. For large scrapes, add delays between runs or distribute across multiple Actor runs. #### Need help? Report issues or suggest features on the [GitHub Issues page](https://github.com/apify/apify-web-scraper). For custom scraping solutions or enterprise use, contact the Apify team. # Actor input Schema ## `mode` (type: `string`): What to scrape from Reddit ## `subreddit` (type: `string`): Subreddit name (without r/ prefix). Used in subreddit mode. ## `query` (type: `string`): Search term. Used in search mode. ## `username` (type: `string`): Reddit username (without u/ prefix). Used in user mode. ## `sort` (type: `string`): Sort order for posts ## `time_range` (type: `string`): Time range for top/controversial sorting ## `max_results` (type: `integer`): Maximum number of posts to scrape (max 100) ## `include_comments` (type: `boolean`): Include up to 5 top comments for each post (increases API calls per result) ## Actor input object example ```json { "mode": "subreddit", "subreddit": "programming", "query": "python", "username": "spez", "sort": "hot", "time_range": "month", "max_results": 25, "include_comments": false } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "subreddit": "programming", "query": "python", "username": "spez" }; // Run the Actor and wait for it to finish const run = await client.actor("ef12/reddit-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "subreddit": "programming", "query": "python", "username": "spez", } # Run the Actor and wait for it to finish run = client.actor("ef12/reddit-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "subreddit": "programming", "query": "python", "username": "spez" }' | apify call ef12/reddit-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=ef12/reddit-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Reddit Scraper", "description": "Scrape Reddit posts and comments by subreddit, search query, or user. Get titles, scores, upvote ratios, comment counts, post bodies, and flairs via the Reddit JSON API.", "version": "0.0", "x-build-id": "L8pT8zPle711fTAhX" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/ef12~reddit-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-ef12-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/ef12~reddit-scraper/runs": { "post": { "operationId": "runs-sync-ef12-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/ef12~reddit-scraper/run-sync": { "post": { "operationId": "run-sync-ef12-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "subreddit", "search", "user" ], "type": "string", "description": "What to scrape from Reddit", "default": "subreddit" }, "subreddit": { "title": "Subreddit", "type": "string", "description": "Subreddit name (without r/ prefix). Used in subreddit mode.", "default": "programming" }, "query": { "title": "Search Query", "type": "string", "description": "Search term. Used in search mode.", "default": "python" }, "username": { "title": "Username", "type": "string", "description": "Reddit username (without u/ prefix). Used in user mode.", "default": "spez" }, "sort": { "title": "Sort", "enum": [ "hot", "new", "top", "rising", "controversial", "relevance" ], "type": "string", "description": "Sort order for posts", "default": "hot" }, "time_range": { "title": "Time Range", "enum": [ "hour", "day", "week", "month", "year", "all" ], "type": "string", "description": "Time range for top/controversial sorting", "default": "month" }, "max_results": { "title": "Max Results", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum number of posts to scrape (max 100)", "default": 25 }, "include_comments": { "title": "Include Top Comments", "type": "boolean", "description": "Include up to 5 top comments for each post (increases API calls per result)", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```