# Reddit Scraper — Posts, Comments & Search (`crabwalker/reddit-scraper`) Actor Scrape public Reddit posts, comments, communities, users, and search results into structured datasets. - **URL**: https://apify.com/crabwalker/reddit-scraper.md - **Developed by:** [Crab Walker](https://apify.com/crabwalker) (community) - **Categories:** Social media, News - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.10 / 1,000 reddit results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Reddit Scraper — Posts, Comments, Communities & Users Fast, API-key-free extraction of public Reddit data. The Actor accepts the same input names and emits the same core output fields as `trudax/reddit-scraper-lite`, so existing Apify tasks and integrations can move over without remapping. It supports: - subreddit, listing, post, search, and user URLs - keyword searches for posts, comments, communities, users, and media - nested comment threads - pagination with independent post, comment, community, and user limits - newest, relevance, hot, top, rising, comments, and time-window sorting - post and comment date cutoffs for incremental collection - NSFW filtering, deduplication, media/gallery extraction, retries, and proxies - a public RSS fallback when Reddit blocks JSON listing endpoints - a Jina Reader fallback for public community/profile discovery when Reddit rejects all available cloud and proxy IPs The Actor uses lightweight HTTP requests rather than a browser. Detailed Reddit JSON responses provide engagement and media fields; when Reddit only permits RSS, the Actor still returns compatible records but fields unavailable in RSS are `null`. Community, profile, and keyword-comment search terms are sent to the public Jina Reader service because Reddit rejects these search pages from Apify cloud and proxy IP ranges. No credentials or private Reddit data are sent. ### Input Provide at least one `startUrls` item or one `searches` term: ```json { "startUrls": [ { "url": "https://www.reddit.com/r/webscraping/new/" } ], "maxItems": 100, "maxPostCount": 100, "skipCommunity": false, "proxy": { "useApifyProxy": true } } ```` Search example: ```json { "searches": ["web scraping"], "searchPosts": true, "searchComments": true, "searchCommunities": true, "sort": "new", "time": "month", "postDateLimit": "2026-07-01T00:00:00Z", "maxItems": 500 } ``` When `startUrls` is non-empty, search fields are ignored, matching the reference Actor. Set `ignoreStartUrls` to `true` for integrations that retain a prefilled URL. ### Output Every dataset item has a `dataType` of `post`, `comment`, `community`, or `user`. Compatible fields include: ```json { "id": "t3_abc123", "parsedId": "abc123", "url": "https://www.reddit.com/r/example/comments/abc123/example/", "username": "alice", "userId": "t2_xyz", "title": "Example", "communityName": "r/example", "parsedCommunityName": "example", "body": "Post text", "html": "

Post text

", "numberOfComments": 12, "upVotes": 42, "upVoteRatio": 0.95, "imageUrls": [], "videoUrls": [], "createdAt": "2026-07-23T01:00:00.000Z", "scrapedAt": "2026-07-23T01:01:00.000Z", "dataType": "post" } ``` `SUMMARY` in the default key-value store reports result counts by type, request count, and whether `maxItems` stopped the run. ### Local development ```bash npm install npm test npm run smoke npm start ``` The checked-in local `INPUT.json` runs a three-result live subreddit test with direct requests and exercises the RSS fallback. Use an Apify or custom proxy for detailed JSON, community profiles, user profiles, and heavily rate-limited comment threads. ### Pricing The intended model is pay per stored result with platform usage included. See [PRICING.md](./PRICING.md) for the exact Console events, initial price, and the calculator that preserves an 80% markup after Apify's 20% commission. ### Responsible use Scrape only public data. Respect Reddit's terms, applicable law, personal-data requirements, and reasonable request rates. This Actor does not bypass login, private communities, quarantines, or access controls. # Actor input Schema ## `startUrls` (type: `array`): Reddit post, subreddit, listing, search, or user URLs. When supplied, search fields are ignored unless Ignore start URLs is enabled. ## `skipComments` (type: `boolean`): Do not extract comments from post URLs. ## `skipUserPosts` (type: `boolean`): Do not extract activity from user profile URLs. ## `skipCommunity` (type: `boolean`): Do not save community details when scraping a subreddit URL. ## `includeMediaLinks` (type: `boolean`): Kept for input compatibility. This Actor always extracts all details available in Reddit's public response without an extra request. ## `searches` (type: `array`): One or more Reddit search queries. ## `searchCommunityName` (type: `string`): Restrict post and comment searches to this subreddit, without the r/ prefix. ## `ignoreStartUrls` (type: `boolean`): Ignore startUrls and use searches. Useful for integrations that keep a prefilled URL. ## `searchPosts` (type: `boolean`): Return posts matching each search term. ## `searchComments` (type: `boolean`): Return comments matching each search term when Reddit exposes comment search. ## `searchCommunities` (type: `boolean`): Return subreddit communities matching each search term. ## `searchUsers` (type: `boolean`): Return Reddit user profiles matching each search term. ## `searchMedia` (type: `boolean`): Search posts and retain only results containing images or video. ## `sort` (type: `string`): Sort Reddit search and listing results. ## `time` (type: `string`): Restrict top/search posts to the selected time window. ## `includeNSFW` (type: `boolean`): Include posts and communities marked over 18. ## `maxItems` (type: `integer`): Global dataset limit across posts, comments, communities, and users. ## `maxPostCount` (type: `integer`): Maximum posts saved from each subreddit, user, listing, or search. ## `postDateLimit` (type: `string`): Only save posts created on or after this ISO date/time. Forces newest-first listing order. ## `commentDateLimit` (type: `string`): Only save comments created on or after this ISO date/time. ## `maxComments` (type: `integer`): Maximum comments saved from each post, user page, or comment search. ## `maxCommunitiesCount` (type: `integer`): Maximum communities saved for each community search. ## `maxUserCount` (type: `integer`): Maximum users saved for each user search. ## `scrollTimeout` (type: `integer`): Compatibility field. Used as the request timeout for the HTTP implementation. ## `proxy` (type: `object`): Apify proxy or custom proxy configuration. Direct requests are used when useApifyProxy is false. ## `debugMode` (type: `boolean`): Log request-level diagnostic details. ## Actor input object example ```json { "startUrls": [ { "url": "https://www.reddit.com/r/webscraping/new/" } ], "skipComments": false, "skipUserPosts": false, "skipCommunity": false, "includeMediaLinks": false, "ignoreStartUrls": false, "searchPosts": true, "searchComments": false, "searchCommunities": false, "searchUsers": false, "searchMedia": false, "sort": "new", "time": "all", "includeNSFW": true, "maxItems": 10, "maxPostCount": 10, "maxComments": 10, "maxCommunitiesCount": 2, "maxUserCount": 2, "scrollTimeout": 40, "proxy": { "useApifyProxy": true }, "debugMode": false } ``` # Actor output Schema ## `dataset` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://www.reddit.com/r/webscraping/new/" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("crabwalker/reddit-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://www.reddit.com/r/webscraping/new/" }] } # Run the Actor and wait for it to finish run = client.actor("crabwalker/reddit-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://www.reddit.com/r/webscraping/new/" } ] }' | apify call crabwalker/reddit-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crabwalker/reddit-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Reddit Scraper — Posts, Comments & Search", "description": "Scrape public Reddit posts, comments, communities, users, and search results into structured datasets.", "version": "1.0", "x-build-id": "nOfi66JbAsYkoCYLx" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crabwalker~reddit-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crabwalker-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crabwalker~reddit-scraper/runs": { "post": { "operationId": "runs-sync-crabwalker-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crabwalker~reddit-scraper/run-sync": { "post": { "operationId": "run-sync-crabwalker-reddit-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrls": { "title": "Start URLs", "type": "array", "description": "Reddit post, subreddit, listing, search, or user URLs. When supplied, search fields are ignored unless Ignore start URLs is enabled.", "items": { "type": "object", "properties": { "url": { "title": "URL", "description": "A public reddit.com URL.", "type": "string" } }, "required": [ "url" ] } }, "skipComments": { "title": "Skip comments", "type": "boolean", "description": "Do not extract comments from post URLs.", "default": false }, "skipUserPosts": { "title": "Skip user posts", "type": "boolean", "description": "Do not extract activity from user profile URLs.", "default": false }, "skipCommunity": { "title": "Skip community", "type": "boolean", "description": "Do not save community details when scraping a subreddit URL.", "default": false }, "includeMediaLinks": { "title": "Include media links, votes, and comment count", "type": "boolean", "description": "Kept for input compatibility. This Actor always extracts all details available in Reddit's public response without an extra request.", "default": false }, "searches": { "title": "Search Term", "type": "array", "description": "One or more Reddit search queries.", "items": { "type": "string" } }, "searchCommunityName": { "title": "Community (optional)", "type": "string", "description": "Restrict post and comment searches to this subreddit, without the r/ prefix." }, "ignoreStartUrls": { "title": "Ignore start URLs", "type": "boolean", "description": "Ignore startUrls and use searches. Useful for integrations that keep a prefilled URL.", "default": false }, "searchPosts": { "title": "Search for posts", "type": "boolean", "description": "Return posts matching each search term.", "default": true }, "searchComments": { "title": "Search for comments", "type": "boolean", "description": "Return comments matching each search term when Reddit exposes comment search.", "default": false }, "searchCommunities": { "title": "Search for communities", "type": "boolean", "description": "Return subreddit communities matching each search term.", "default": false }, "searchUsers": { "title": "Search for users", "type": "boolean", "description": "Return Reddit user profiles matching each search term.", "default": false }, "searchMedia": { "title": "Search for media", "type": "boolean", "description": "Search posts and retain only results containing images or video.", "default": false }, "sort": { "title": "Sort search", "enum": [ "relevance", "hot", "top", "new", "rising", "comments" ], "type": "string", "description": "Sort Reddit search and listing results.", "default": "new" }, "time": { "title": "Filter by date (posts only)", "enum": [ "all", "hour", "day", "week", "month", "year" ], "type": "string", "description": "Restrict top/search posts to the selected time window.", "default": "all" }, "includeNSFW": { "title": "Include NSFW content", "type": "boolean", "description": "Include posts and communities marked over 18.", "default": true }, "maxItems": { "title": "Maximum number of items to save", "minimum": 1, "maximum": 2147483647, "type": "integer", "description": "Global dataset limit across posts, comments, communities, and users.", "default": 10 }, "maxPostCount": { "title": "Posts per listing", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum posts saved from each subreddit, user, listing, or search.", "default": 10 }, "postDateLimit": { "title": "Post date limit", "type": "string", "description": "Only save posts created on or after this ISO date/time. Forces newest-first listing order." }, "commentDateLimit": { "title": "Comment date limit", "type": "string", "description": "Only save comments created on or after this ISO date/time." }, "maxComments": { "title": "Comments per post/page", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum comments saved from each post, user page, or comment search.", "default": 10 }, "maxCommunitiesCount": { "title": "Community search limit", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum communities saved for each community search.", "default": 2 }, "maxUserCount": { "title": "User search limit", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum users saved for each user search.", "default": 2 }, "scrollTimeout": { "title": "Page scroll timeout (seconds)", "minimum": 1, "maximum": 2147483647, "type": "integer", "description": "Compatibility field. Used as the request timeout for the HTTP implementation.", "default": 40 }, "proxy": { "title": "Proxy configuration", "type": "object", "description": "Apify proxy or custom proxy configuration. Direct requests are used when useApifyProxy is false.", "default": { "useApifyProxy": true } }, "debugMode": { "title": "Debug Mode", "type": "boolean", "description": "Log request-level diagnostic details.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```