# 👽 Reddit Scraper — Posts & Comments by Subreddit or Search (`iskoren/reddit-scraper`) Actor Scrape Reddit posts and comments from any subreddit or search query — scores, authors, timestamps, flairs, and full text. Export structured data for research and monitoring. - **URL**: https://apify.com/iskoren/reddit-scraper.md - **Developed by:** [Is Koren](https://apify.com/iskoren) (community) - **Categories:** Social media, News, Business - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $5.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## 👽 Reddit Scraper — Posts & Comments by Subreddit or Search Scrape **Reddit posts and comments** from any subreddit or keyword search — no Reddit API key, no OAuth, no login. This **Reddit Scraper** uses Reddit's public JSON endpoints to pull structured post data (title, score, author, upvote ratio, comment count, flair, and more) and, optionally, the **top comments** for each post. Point it at one or more subreddits, add search queries, choose how to sort, and get clean JSON, CSV, or Excel output you can plug into analytics, research, or monitoring workflows. ### ✨ What you get - Scrape **multiple subreddits** and **search queries** in a single run. - Sort by **hot, new, top, or rising** (with a time window for *top*). - Optionally fetch the **top comments** per post. - One clean record per post — ready for spreadsheets, dashboards, or databases. - Built-in **proxy support** (RESIDENTIAL recommended — Reddit rate-limits datacenter IPs). ### 🚀 Quick start Paste this into the input editor and hit **Start**: ```json { "subreddits": ["python"], "sort": "hot", "maxItems": 10 } ```` Want comments and a keyword search too? ```json { "subreddits": ["dataisbeautiful", "MachineLearning"], "searchQueries": ["large language models"], "sort": "top", "time": "month", "maxItems": 50, "includeComments": true, "maxComments": 5, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] } } ``` ### ⚙️ Input | Field | Type | Default | Description | |-------|------|---------|-------------| | `subreddits` | array of strings | `["python"]` | Subreddit names without `r/` (e.g. `python`, `dataisbeautiful`). | | `searchQueries` | array of strings | `[]` | Optional keyword searches run against Reddit's site-wide search. | | `sort` | select | `hot` | `hot`, `new`, `top`, or `rising`. | | `time` | select | `week` | Time window applied only when `sort` is `top`: `hour`, `day`, `week`, `month`, `year`, `all`. | | `maxItems` | integer | `25` | Max posts per source (subreddit or query). Range 1–200. | | `includeComments` | boolean | `false` | Fetch top comments for each post (slower). | | `maxComments` | integer | `10` | Max top-level comments per post when `includeComments` is on. Range 1–100. | | `proxyConfiguration` | proxy | RESIDENTIAL | Apify proxy settings. RESIDENTIAL recommended. | ### 📤 Output Each row is one Reddit post. Example: ```json { "id": "1abcd23", "subreddit": "python", "title": "What's your favourite underrated standard library module?", "author": "example_user", "score": 1423, "upvoteRatio": 0.97, "numComments": 312, "createdUtc": 1718600000.0, "createdIso": "2024-06-17T07:33:20+00:00", "url": "https://www.reddit.com/r/python/comments/1abcd23/...", "permalink": "https://www.reddit.com/r/python/comments/1abcd23/whats_your_favourite/", "selftext": "I'll start: functools...", "flair": "Discussion", "over18": false, "isVideo": false, "domain": "self.python", "matchedSource": "r/python", "comments": [ { "author": "commenter1", "body": "itertools, hands down.", "score": 88 } ] } ``` The `comments` array is included only when `includeComments` is enabled. `matchedSource` tells you which subreddit (`r/python`) or query (`search:large language models`) produced the row. ### 📥 Exporting your data Results land in the run's dataset. From the **Storage** tab (or the API) you can export to **JSON, CSV, Excel, HTML, RSS, or XML**, or fetch them programmatically: ``` https://api.apify.com/v2/datasets//items?format=csv ``` ### 💡 Tips - **Use RESIDENTIAL proxies.** Reddit aggressively rate-limits datacenter IPs and may return `429`. The actor retries `429`s automatically, but residential proxies are far more reliable. - **`maxItems` is per source.** Two subreddits with `maxItems: 50` can yield up to 100 posts. - **`time` only matters for `top`.** It's ignored for hot/new/rising. - **Comments cost extra requests.** Each post with `includeComments` triggers an additional fetch, so large runs are slower — keep `maxComments` modest. - **Pagination is automatic.** The actor follows Reddit's `after` cursor until your `maxItems` cap is hit. ### ❓ FAQ **Do I need a Reddit API key?** No. This scraper uses Reddit's public `.json` endpoints, so there's no API key, OAuth, or app registration. **Why am I getting fewer items than `maxItems`?** The source simply may not have that many posts (common for small or quiet subreddits, or narrow searches). **Can I scrape comments only?** The actor is post-centric — comments are attached to their parent post when `includeComments` is enabled. **Is this allowed?** You're responsible for complying with Reddit's terms and applicable laws. Scrape responsibly, keep volumes reasonable, and respect user privacy. **Why did some requests get rate-limited?** Reddit limits unauthenticated traffic. Residential proxies and modest `maxItems` keep runs smooth; the actor backs off and retries on `429`. ### 🔎 Use cases - Track sentiment and trending topics across communities. - Build datasets for NLP / market research. - Monitor brand, product, or competitor mentions via search queries. - Feed dashboards that surface the most-upvoted discussions in your niche. # Actor input Schema ## `subreddits` (type: `array`): Subreddit names to scrape, without the 'r/' prefix (e.g. 'python', 'dataisbeautiful'). ## `searchQueries` (type: `array`): Optional keyword searches run against Reddit's site-wide search. One record per matching post. ## `sort` (type: `string`): How to sort posts within each source. ## `time` (type: `string`): Time window applied only when sort is 'top'. ## `maxItems` (type: `integer`): Maximum number of posts to collect per subreddit or search query. ## `includeComments` (type: `boolean`): If enabled, fetch the top comments for each post (slower, extra requests). ## `maxComments` (type: `integer`): Maximum number of top-level comments to collect per post when 'Include comments' is enabled. ## `proxyConfiguration` (type: `object`): Proxy settings. RESIDENTIAL is strongly recommended because Reddit rate-limits datacenter IPs. ## Actor input object example ```json { "subreddits": [ "python" ], "sort": "hot", "time": "week", "maxItems": 25, "includeComments": false, "maxComments": 10, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "subreddits": [ "python" ] }; // Run the Actor and wait for it to finish const run = await client.actor("iskoren/reddit-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "subreddits": ["python"] } # Run the Actor and wait for it to finish run = client.actor("iskoren/reddit-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "subreddits": [ "python" ] }' | apify call iskoren/reddit-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=iskoren/reddit-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "👽 Reddit Scraper — Posts & Comments by Subreddit or Search", "description": "Scrape Reddit posts and comments from any subreddit or search query — scores, authors, timestamps, flairs, and full text. Export structured data for research and monitoring.", "version": "0.1", "x-build-id": "msB1dfc6Kf6Re7ylu" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/iskoren~reddit-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-iskoren-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/iskoren~reddit-scraper/runs": { "post": { "operationId": "runs-sync-iskoren-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/iskoren~reddit-scraper/run-sync": { "post": { "operationId": "run-sync-iskoren-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "subreddits": { "title": "Subreddits", "type": "array", "description": "Subreddit names to scrape, without the 'r/' prefix (e.g. 'python', 'dataisbeautiful').", "items": { "type": "string" } }, "searchQueries": { "title": "Search queries", "type": "array", "description": "Optional keyword searches run against Reddit's site-wide search. One record per matching post.", "items": { "type": "string" } }, "sort": { "title": "Sort", "enum": [ "hot", "new", "top", "rising" ], "type": "string", "description": "How to sort posts within each source.", "default": "hot" }, "time": { "title": "Time range (for 'top' sort)", "enum": [ "hour", "day", "week", "month", "year", "all" ], "type": "string", "description": "Time window applied only when sort is 'top'.", "default": "week" }, "maxItems": { "title": "Max items per source", "minimum": 1, "maximum": 200, "type": "integer", "description": "Maximum number of posts to collect per subreddit or search query.", "default": 25 }, "includeComments": { "title": "Include comments", "type": "boolean", "description": "If enabled, fetch the top comments for each post (slower, extra requests).", "default": false }, "maxComments": { "title": "Max comments per post", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum number of top-level comments to collect per post when 'Include comments' is enabled.", "default": 10 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings. RESIDENTIAL is strongly recommended because Reddit rate-limits datacenter IPs.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```