# Reddit Posts + Comments Scraper (`softwaresubs29/reddit-posts-comments-scraper`) Actor Scrapes Reddit posts and complete comment trees (including 'load more' branches) from subreddits, posts, user profiles, or keyword search. HTTP-only — no browser — for low compute cost. - **URL**: https://apify.com/softwaresubs29/reddit-posts-comments-scraper.md - **Developed by:** [Shab Codes](https://apify.com/softwaresubs29) (community) - **Categories:** Other, Developer tools, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 post scrapeds This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Reddit Posts + Comments Scraper Scrapes Reddit **posts and complete comment trees** from subreddits, individual posts, user profiles, or keyword search — including the comments hidden behind Reddit's "load more comments" and "continue this thread" stubs, which cheaper scrapers silently drop. Runs over plain HTTP against Reddit's public JSON endpoints (no browser, no HTML parsing), so compute cost per result stays low. ### No setup required Just paste Reddit URLs (or a search keyword), keep the default **RESIDENTIAL** proxy on, and click **Start** — **no Reddit account, API key, or credentials needed.** The scraper reads Reddit's public `old.reddit.com` pages with a real browser fingerprint and rotates residential IPs, retrying any blocked request on a fresh IP. > Residential proxy matters. Reddit blocks datacenter and single IPs hard; residential rotation is what keeps runs flowing. The default proxy setting is already RESIDENTIAL — leave it on. ### What it does - **Subreddit URLs** (`https://www.reddit.com/r/programming/`, including `/top`, `/new`, `/rising`, `/search?q=…` pages) → scrapes the post listing and each post's full comment tree, following pagination. - **Post URLs** (`https://www.reddit.com/r/…/comments/abc123/…`) → scrapes that post and its complete comment tree. - **User profile URLs** (`https://www.reddit.com/user/spez`) → scrapes the user's posts and comments. - **Search keyword** → searches all of Reddit and scrapes matching posts + comments. Deep comment threads are paginated by Reddit behind "load more" stubs. This scraper resolves them recursively via the `morechildren` API and re-fetches "continue this thread" branches, so you get the whole tree — not just what fits in the first response. ### Input | Field | Type | Description | |---|---|---| | `startUrls` | array | Subreddit, post, or user profile URLs. | | `searchKeyword` | string | Optional. Site-wide Reddit search; can be combined with `startUrls`. | | `maxItems` | integer | Hard cap on total results (posts + comments). Default `1000`. | | `maxCommentDepth` | integer | `0` = posts only, `1` = top-level comments only, `2` = + direct replies, … Empty = unlimited (complete trees). Deep replies are often noise — capping depth is the main cost lever. | | `timeFilter` | enum | `hour` / `day` / `week` / `month` / `year` / `all`. Applies to keyword search and `/top` listings. | | `proxyConfiguration` | object | Apify Proxy settings. **Defaults to RESIDENTIAL — keep it on.** Datacenter IPs get blocked by Reddit. | ### Output Posts and comments share one schema; `parentId` lets you reconstruct the tree client-side. ```json { "type": "comment", "id": "kxf2ab1", "parentId": "kxf19zq", "postId": "1c4xyz", "depth": 2, "author": "some_user", "text": "Comment body…", "score": 42, "subreddit": "programming", "createdAt": "2026-06-30T14:21:05.000Z", "permalink": "https://www.reddit.com/r/programming/comments/1c4xyz/comment/kxf2ab1/" } ```` Posts (`"type": "post"`) additionally carry `title`, `url` (external link for link posts), `numComments`, and `over18`; their `parentId` is `null`. Top-level comments have `depth: 1` and their `parentId` is the post id. **Deleted/removed content** is passed through as Reddit reports it (`[deleted]` / `[removed]` author or body) rather than erroring, so tree structure stays intact. ### Pricing (pay-per-event) This Actor charges per result, so runs that skip comments (`maxCommentDepth: 0`) only pay for posts: | Event | Charged when | |---|---| | `post_scraped` | A post is pushed to the dataset. | | `comment_scraped` | A comment is pushed to the dataset. | Set up in Apify Console → Actor → Publication → Monetization → Pay per event, with event names exactly `post_scraped` and `comment_scraped` (suggested: ~$0.003/post, ~$0.0025/comment ≈ $2.50–3.00 per 1,000 results). ### Limits & tips - **Private, quarantined, and gated subreddits** cannot be scraped even with credentials unless your account has access. The Actor logs a clear warning and skips them instead of retrying forever. - **Very large threads (10k+ comments):** run the post URL as its own run (optionally split by `maxCommentDepth` or `maxItems` windows) rather than bundling it with a big subreddit crawl, so one thread can't consume the whole budget. - **Rate limits:** Reddit rate-limits per IP. The Actor throttles itself and, when Reddit returns a block page, retires that IP and retries on a fresh residential one. Larger jobs go faster and more reliably on a paid Apify plan with a healthy residential pool. ### Legal notice — read before using - This Actor collects **publicly accessible** Reddit content. Scraping public web pages is broadly permitted in many jurisdictions, **but Reddit's User Agreement restricts automated collection of its content**; using this Actor may violate those terms, and you assume that risk. - Reddit posts and comments contain **personal data** (usernames, opinions, sometimes identifying details). Depending on how you store and use the data, **GDPR, CCPA, and similar privacy laws may apply** — including obligations around lawful basis, retention, and deletion requests. - This tool is provided for legitimate purposes such as research, moderation, and archiving of public content. You are solely responsible for ensuring your use complies with applicable laws and Reddit's terms. This is not legal advice. # Actor input Schema ## `startUrls` (type: `array`): Reddit URLs to scrape: subreddits (https://www.reddit.com/r/programming/), single posts (…/comments/abc123/…), or user profiles (https://www.reddit.com/user/spez). Subreddit sort pages like /top or /new also work. ## `searchKeyword` (type: `string`): Optional. Searches all of Reddit for this keyword and scrapes the matching posts (and their comments). Can be combined with Start URLs. ## `maxItems` (type: `integer`): Hard cap on the total number of results (posts + comments) pushed to the dataset. Use this to control cost. ## `maxCommentDepth` (type: `integer`): Caps how deep into comment threads the scraper goes. 0 = posts only (no comments), 1 = top-level comments only, 2 = top-level + direct replies, etc. Leave empty for unlimited depth (complete trees). Deep replies are often low-value — capping depth cuts cost significantly. ## `timeFilter` (type: `string`): Restricts keyword search results and /top listings to this time window. ## `proxyConfiguration` (type: `object`): Proxies to use. Reddit blocks datacenter and single IPs aggressively, so RESIDENTIAL proxies are the default and strongly recommended — the scraper retries blocked requests on fresh residential IPs. Datacenter will mostly get blocked. ## Actor input object example ```json { "startUrls": [ { "url": "https://www.reddit.com/r/programming/" } ], "maxItems": 1000, "timeFilter": "all", "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://www.reddit.com/r/programming/" } ], "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("softwaresubs29/reddit-posts-comments-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://www.reddit.com/r/programming/" }], "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("softwaresubs29/reddit-posts-comments-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://www.reddit.com/r/programming/" } ], "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call softwaresubs29/reddit-posts-comments-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=softwaresubs29/reddit-posts-comments-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Reddit Posts + Comments Scraper", "description": "Scrapes Reddit posts and complete comment trees (including 'load more' branches) from subreddits, posts, user profiles, or keyword search. HTTP-only — no browser — for low compute cost.", "version": "0.4", "x-build-id": "LxOVCSM2BSgxwecmE" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/softwaresubs29~reddit-posts-comments-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-softwaresubs29-reddit-posts-comments-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/softwaresubs29~reddit-posts-comments-scraper/runs": { "post": { "operationId": "runs-sync-softwaresubs29-reddit-posts-comments-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/softwaresubs29~reddit-posts-comments-scraper/run-sync": { "post": { "operationId": "run-sync-softwaresubs29-reddit-posts-comments-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrls": { "title": "Start URLs", "type": "array", "description": "Reddit URLs to scrape: subreddits (https://www.reddit.com/r/programming/), single posts (…/comments/abc123/…), or user profiles (https://www.reddit.com/user/spez). Subreddit sort pages like /top or /new also work.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "searchKeyword": { "title": "Search keyword", "type": "string", "description": "Optional. Searches all of Reddit for this keyword and scrapes the matching posts (and their comments). Can be combined with Start URLs." }, "maxItems": { "title": "Max items", "minimum": 1, "type": "integer", "description": "Hard cap on the total number of results (posts + comments) pushed to the dataset. Use this to control cost.", "default": 1000 }, "maxCommentDepth": { "title": "Max comment depth", "minimum": 0, "type": "integer", "description": "Caps how deep into comment threads the scraper goes. 0 = posts only (no comments), 1 = top-level comments only, 2 = top-level + direct replies, etc. Leave empty for unlimited depth (complete trees). Deep replies are often low-value — capping depth cuts cost significantly." }, "timeFilter": { "title": "Time filter", "enum": [ "hour", "day", "week", "month", "year", "all" ], "type": "string", "description": "Restricts keyword search results and /top listings to this time window.", "default": "all" }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxies to use. Reddit blocks datacenter and single IPs aggressively, so RESIDENTIAL proxies are the default and strongly recommended — the scraper retries blocked requests on fresh residential IPs. Datacenter will mostly get blocked.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```