# Reddit Intelligence Scraper (`qaseemiqbal/reddit-intelligence-scraper`) Actor Collect public Reddit posts, comments, communities, and user profile data from searches, subreddit pages, Reddit URLs, and usernames. Export clean datasets for monitoring, research, and AI workflows. - **URL**: https://apify.com/qaseemiqbal/reddit-intelligence-scraper.md - **Developed by:** [Muhammad Qaseem Iqbal](https://apify.com/qaseemiqbal) (community) - **Categories:** Social media, AI - **Stats:** 2 total users, 1 monthly users, 60.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## 🚀 Reddit Intelligence Scraper Collect public Reddit posts, comments, communities, and user profile data from searches, subreddit pages, Reddit URLs, and usernames. 🔎 Use it to monitor conversations, research customer opinions, follow trends, and export clean Reddit data into spreadsheets, dashboards, databases, AI workflows, or automation tools. 📊 This Actor is designed to be practical for both non-technical users and data teams. ✅ You can start with a keyword or Reddit URL, choose how many results you want, and download the results from the Apify dataset when the run finishes. 📥 ### 🧠 What does this Actor do? Reddit Intelligence Scraper turns public Reddit pages into structured data. 🧾 Instead of manually copying posts and comments from Reddit, you can run the Actor and get organized records with useful details such as: - 📝 post title, body, author, subreddit, score, comment count, and URL - 💬 comment text, author, parent post, score, depth, and timestamp - 🏘️ subreddit/community name, description, subscriber count, and metadata - 👤 public user profile information, including karma and profile URL - 🏷️ optional sentiment labels, content categories, engagement metrics, media links, and raw payloads No Reddit API key, OAuth setup, or Reddit login is required for supported public pages. 🔓 ### 🎯 Common use cases - 📣 Track brand, product, or competitor mentions on Reddit - 📅 Monitor subreddit discussions on a schedule - 💡 Find customer pain points, feature requests, complaints, and praise - 🧵 Collect comments from a specific Reddit thread - 🔬 Research topics, communities, trends, and market language - 🤖 Build datasets for AI search, RAG, clustering, dashboards, or reports - 📤 Export Reddit data to CSV, Excel, Google Sheets, Make, Zapier, n8n, webhooks, or your own API workflow ### 📦 What Reddit data can it collect? | Data type | What you can collect | | --- | --- | | 📝 Posts | Search results, subreddit listings, direct post URLs, user submitted posts, `r/all`, and `r/popular` | | 💬 Comments | Comment search results and comment threads under posts when comment collection is enabled | | 🏘️ Communities | Subreddit metadata and community search results | | 👤 Users | Public Reddit user profile records and optional user activity inputs | The Actor works with several input styles, so you can start broad with keywords or stay precise with direct Reddit URLs. 🧭 ### ⚡ How to scrape Reddit on Apify 1. 🖥️ Open the Actor in Apify Console. 2. ➕ Add at least one source: - 🔎 keywords in **Search terms** - 🔗 Reddit links in **Direct Reddit URLs** - 🏘️ subreddit names or URLs in **Full subreddit scrape inputs** - 👤 Reddit usernames or profile URLs in **User profile inputs** 3. 🎚️ Set a result limit, such as `maxItems`. 4. ⚙️ Choose whether to include comments, media links, sentiment, or other optional data. 5. ▶️ Click **Start**. 6. 📥 Download the results from the **Dataset** tab as JSON, CSV, Excel, XML, or RSS. For a quick test, use a small limit such as `maxItems: 10`. 🧪 For scheduled monitoring, keep the limit modest and run the Actor repeatedly. 📅 ### 🎛️ Input options You only need one valid source to start. ✅ The most important fields are below. | Field | Plain-English meaning | Typical use | | --- | --- | --- | | 🔎 `searchTerms` | Keywords or phrases to search across Reddit | Brand monitoring, topic research, competitor tracking | | 🔗 `startUrls` | Direct Reddit URLs | Scrape a specific post, subreddit, user page, or Reddit search URL | | 🏘️ `subredditUrls` | Subreddit names or URLs | Collect posts from communities such as `r/startups` | | 👤 `userUrls` | Reddit usernames or profile URLs | Collect public user profile information | | 🎚️ `maxItems` | Maximum total records to save | Keep tests and production runs under control | | 💬 `crawlCommentsPerPost` | Also collect comments under each collected post | Thread research, sentiment, FAQ mining | | 🧵 `maxCommentsPerPost` | Comment limit for each post | Prevent very large threads from growing too much | | 🧭 `sort` and `time` | Reddit search ranking and time window | Newest posts, top posts this week, most commented posts, etc. | | 📍 `withinCommunity` | Search only inside one subreddit | Search for a topic within a specific community | | 🖼️ `includeMediaLinks` | Save image, video, gallery, and outbound link details | Media analysis or content discovery | | 😊 `sentimentAnalysis` | Add simple sentiment labels to posts and comments | Positive, negative, neutral, mixed, or uncertain | | 🏷️ `contentAnalysis` | Add topic/category labels to post records | Routing, grouping, research, and AI workflows | | 🛡️ `proxyConfiguration` | Optional Apify Proxy settings | Use Residential proxy when Reddit blocks cloud traffic | Advanced settings are available for date filters, comment depth, strict keyword matching, output style, raw data storage, and run reports. 🧰 ### 🧪 Example inputs #### 🔎 1. Quick keyword search Use this when you want a small sample of recent posts for a topic. ⚡ ```json { "searchTerms": ["AI video generator"], "sort": "new", "time": "week", "maxItems": 25, "maxPostsPerSearch": 25 } ```` #### πŸ“£ 2. Brand and competitor monitoring Use this to track mentions and include comments found through Reddit comment search. πŸ“‘ ```json { "searchTerms": ["Acme AI", "Acme pricing", "Acme alternative"], "searchPosts": true, "searchComments": true, "sort": "new", "time": "week", "maxItems": 150, "maxPostsPerSearch": 50, "maxCommentsCount": 50, "sentimentAnalysis": true } ``` #### 🏘️ 3. Scrape a subreddit Use this to collect posts from one or more communities. 🧭 ```json { "subredditUrls": ["r/startups"], "subredditSort": "new", "subredditTime": "month", "maxItems": 100, "maxPostsPerSubreddit": 100 } ``` #### 🧡 4. Collect a full post thread Use this when you already know the Reddit post URL and want the discussion under it. πŸ’¬ ```json { "startUrls": [ { "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/" } ], "crawlCommentsPerPost": true, "maxCommentsPerPost": 500, "commentDepthLimit": 0 } ``` #### πŸ’Έ 5. Low-cost test run Use this before a larger run to confirm your input works. βœ… ```json { "searchTerms": ["customer support software"], "maxItems": 10, "maxPostsPerSearch": 10, "crawlCommentsPerPost": false, "includeMediaLinks": false, "saveRawData": false, "writeHtmlReport": false } ``` ### πŸ“€ Output Results are saved to the default Apify dataset. πŸ“Š Each dataset item is one record. Possible record types: - πŸ“ `post` - πŸ’¬ `comment` - 🏘️ `community` - πŸ‘€ `user` Every record includes basic tracking fields such as: 🧾 | Field | Meaning | | --- | --- | | 🧩 `kind` | Type of record: post, comment, community, or user | | πŸ†” `id` | Reddit item ID | | πŸ”— `url` | Main Reddit URL for the item | | βœ… `canonicalUrl` | Normalized Reddit URL where available | | ⏱️ `scrapedAt` | When the Actor collected the record | | πŸ“ `source` | Which input produced the record | | πŸ” `sources` | Other inputs that found the same record, when duplicates are merged | ### πŸ“ Example post output ```json { "kind": "post", "id": "1hvoazn", "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/", "title": "My best cheesecake so far", "author": "example_user", "subreddit": "Baking", "createdAt": "2025-01-07T10:09:56.000Z", "score": 3489, "numComments": 43, "mediaType": "gallery", "hasMedia": true, "sentimentLabel": "positive", "contentCategoryLabel": "Food & Drink" } ``` The exact fields depend on the record type and the options you enable. βš™οΈ ### πŸ“‹ Run summary At the end of a run, the Actor writes `RUN-SUMMARY.json` to the key-value store. 🧾 This file is useful when you want a quick overview without opening the full dataset. The summary includes: - πŸ”’ total records saved - πŸ“¦ records by type - πŸ”Ž query and subreddit breakdowns - ⏭️ skipped items and why they were skipped - πŸ“ˆ request statistics - ⚠️ warnings and errors - πŸ†” IDs of the output dataset and key-value store If you enable `writeHtmlReport`, the Actor can also create a simple HTML report called `RUN-MAP.html`. πŸ—ΊοΈ ### πŸ’Έ Cost and performance tips This Actor is configured to keep costs low by default. βœ… - πŸ›‘οΈ Residential proxy is enabled by default because Reddit currently blocks direct Apify cloud traffic. - 🏠 For the cheapest successful tests, keep runs small and use direct Reddit URLs first. - 🎚️ Result limits are conservative by default. - πŸ” Request retries are disabled by default to avoid paying for repeated failed requests. - πŸ“ Raw data, media details, awards, and HTML reports are off by default. - πŸ’¬ Comments are only collected when you enable comment collection. To keep runs cheap: - πŸ§ͺ start with `maxItems` between 10 and 100 - πŸ’¬ keep `crawlCommentsPerPost` off unless you need thread-level discussion - πŸ“¦ keep `saveRawData` off unless you are debugging - πŸ—ΊοΈ keep `writeHtmlReport` off unless you need a visual report - πŸ”­ avoid `maximizeCoverage` unless recall matters more than speed and cost - πŸ›‘οΈ disable proxy only if direct access works for your run environment ### πŸ’³ Store pricing This Actor is designed for simple pay-per-result pricing on Apify Store. 🧾 Recommended paid events: | Event | What it means | | --- | --- | | πŸš€ `apify-actor-start` | A very small startup event charged automatically by Apify | | πŸ“¦ `apify-default-dataset-item` | One saved dataset record, such as a post, comment, community, or user | This keeps pricing easy to predict: the more records you save, the more you pay. Apify shows the run cost before and during execution, and you can control spend by setting `maxItems`, comment limits, and other result caps. 🎚️ ### πŸ“… Scheduling and integrations You can schedule this Actor in Apify Console to monitor Reddit regularly. ⏰ For example: - ⚑ every hour for fast-moving brand monitoring - πŸ“† once per day for subreddit tracking - πŸ“Š once per week for market research exports After each run, you can send the dataset to: - πŸ“— Google Sheets - 🧩 Make - ⚑ Zapier - πŸ”„ n8n - πŸͺ webhooks - ☁️ cloud storage - πŸ—„οΈ databases and warehouses - πŸ”Œ custom applications through the Apify API ### ⚠️ Important notes and limitations Reddit controls how much public data is available through its pages and listings. πŸ“Œ This affects all Reddit scrapers, not only this Actor. - πŸ”’ Some private, restricted, quarantined, deleted, removed, or login-gated content cannot be collected. - πŸͺŸ Reddit search and subreddit listings may expose only a limited window of results. - πŸ•°οΈ Very old posts may require narrower keywords, different sort options, or direct URLs. - 🚧 Reddit may rate limit or block traffic from cloud networks or proxies. - ❌ If every Reddit request is blocked, the Actor fails the run instead of silently returning an empty successful dataset. - βš™οΈ This version is HTTP-first and does not use a browser fallback. If a run is blocked by Reddit, try a smaller run first, reduce concurrency and request rate, try a direct post URL, use different inputs, or run again later. πŸ§ͺ Residential proxy settings are often the most reliable cloud option for Reddit, but they can increase cost and are not guaranteed to bypass every Reddit-side block. πŸ›‘οΈ ### ❓ FAQ #### βš–οΈ Is Reddit scraping legal? Scraping public Reddit data can be allowed in many cases, but you are responsible for how you collect, store, and use the data. πŸ›‘οΈ Always follow Reddit's terms, applicable laws, privacy rules, and the rules of any downstream platform where you use the data. #### πŸ”‘ Do I need a Reddit account or API key? No. βœ… This Actor is built for supported public Reddit pages and does not require a Reddit login or Reddit API key. #### πŸ’¬ Can it scrape comments? Yes. βœ… Enable `crawlCommentsPerPost` to collect comments under posts. You can control the amount with `maxCommentsPerPost` and `commentDepthLimit`. #### πŸ”— Can I scrape a specific Reddit post? Yes. βœ… Add the post URL to `startUrls`. If you also want the comments, enable `crawlCommentsPerPost`. #### 🏘️ Can I scrape a whole subreddit? Yes. βœ… Add a subreddit name such as `r/startups` or a full subreddit URL to `subredditUrls`. You can choose sorting options such as new, hot, top, rising, or most commented. #### πŸ“‰ Why did I get fewer results than expected? Common reasons include Reddit result limits, strict filters, date filters, duplicate removal, deleted or unavailable items, or Reddit blocking the request. πŸ” Check `RUN-SUMMARY.json` for warnings, errors, and skip counts. #### πŸͺŸ Why can't I always get more than about 1,000 posts from a subreddit or search? Reddit lists are not unlimited. πŸ“Œ Search pages and subreddit feeds often stop after a practical result window. To find more unique posts, try narrower keywords, different time windows, different sort options, or direct Reddit URLs. #### πŸ›‘οΈ Do I need proxies? On Apify cloud, usually yes. πŸ›‘οΈ Reddit is currently blocking direct cloud requests in our tests, while the `RESIDENTIAL` proxy group succeeded. Residential proxy traffic can increase cost, so keep test runs small and lower `maxItems` while testing. #### πŸ“€ Can I export the results? Yes. βœ… Apify datasets can be exported as JSON, CSV, Excel, XML, RSS, or accessed through the Apify API. #### πŸ€– Can I use the data with AI tools? Yes. βœ… The output is structured JSON, which makes it suitable for AI search, summarization, clustering, dashboards, and RAG workflows. Make sure your use of the data follows applicable privacy and platform rules. ### πŸ›‘οΈ Responsible use Use this Actor only for public Reddit data that you are allowed to collect and process. βœ… Do not use it to collect private, login-gated, sensitive, or harmful personal data. πŸ”’ Avoid publishing datasets in a way that exposes individuals unfairly or outside the purpose for which the data was collected. ### 🧰 Support If something does not work as expected, include: - πŸ†” the Apify run ID - πŸ“₯ your input JSON - πŸ“‹ the `RUN-SUMMARY.json` file - πŸ“ a short description of what you expected and what happened This makes it much easier to diagnose blocked requests, empty datasets, input mistakes, and result-limit questions. πŸ” # Actor input Schema ## `searchTerms` (type: `array`): Keywords or phrases to search across Reddit. ## `startUrls` (type: `array`): Direct Reddit URLs to scrape, such as post URLs, subreddit pages, user pages, all/popular listings, or Reddit search URLs. ## `subredditUrls` (type: `array`): Subreddits to traverse. Accepts names, r/name, or full subreddit URLs. ## `userUrls` (type: `array`): Reddit usernames or user profile URLs to scrape. ## `withinCommunity` (type: `string`): Optional subreddit restriction for search terms. ## `searchPosts` (type: `boolean`): Search for Reddit posts matching the search terms. ## `searchComments` (type: `boolean`): Search for Reddit comments matching the search terms. ## `searchCommunities` (type: `boolean`): Search for subreddits/communities matching the search terms. ## `searchUsers` (type: `boolean`): Search for public Reddit users matching the search terms. ## `searchMedia` (type: `boolean`): Keep media-bearing post results from search. ## `sort` (type: `string`): Ranking method for Reddit search results. ## `time` (type: `string`): Reddit-side time window for search results. ## `strictSearch` (type: `boolean`): Use stricter query construction where possible. ## `strictTokenFilter` (type: `boolean`): After collection, keep only records whose searchable text matches the important query tokens. ## `excludeTerms` (type: `array`): Terms that disqualify a record when found in searchable text. ## `subredditSort` (type: `string`): Ranking method for subreddit listing traversal. ## `subredditTime` (type: `string`): Time range for top-style subreddit traversal. ## `maximizeCoverage` (type: `boolean`): Expand broad queries and subreddits across complementary result views, then deduplicate. ## `crawlCommentsPerPost` (type: `boolean`): Collect comments for every post discovered from direct URLs, searches, or subreddits. ## `maxCommentsPerPost` (type: `integer`): Maximum comments to collect for each post. ## `maxCommentsCount` (type: `integer`): Maximum comments to collect from comment search per search term. ## `commentDepthLimit` (type: `integer`): Maximum nested comment depth. Use 0 for no depth limit. ## `commentSort` (type: `string`): Preferred comment ordering where supported. ## `commentDateFrom` (type: `string`): Optional lower date bound for comments. Accepts ISO datetime or YYYY-MM-DD. ## `commentDateTo` (type: `string`): Optional upper date bound for comments. Accepts ISO datetime or YYYY-MM-DD. ## `dateFrom` (type: `string`): Optional lower date bound for posts. Accepts ISO datetime or YYYY-MM-DD. ## `dateTo` (type: `string`): Optional upper date bound for posts. Accepts ISO datetime or YYYY-MM-DD. ## `forceSortNewForTimeFilteredRuns` (type: `boolean`): When a post date filter is active, use chronological traversal where possible. ## `includeNsfw` (type: `boolean`): Include posts or communities marked as NSFW/18+. ## `maxItems` (type: `integer`): Maximum dataset records to save across all record types. ## `maxPostsPerSearch` (type: `integer`): Maximum post records to collect for each search term. ## `maxPostsPerSubreddit` (type: `integer`): Maximum post records to collect for each subreddit input. ## `maxPostsPerUser` (type: `integer`): Maximum public user posts to collect per user input. ## `maxCommunitiesCount` (type: `integer`): Maximum community records to collect per community search. ## `maxUsersCount` (type: `integer`): Maximum user profile records to collect per user search. ## `maxRequestsPerRun` (type: `integer`): Safety cap for total queued request processing. ## `includeMediaLinks` (type: `boolean`): Extract media URLs, galleries, preview images, videos, and outbound links. ## `includeAwards` (type: `boolean`): Extract Reddit award/gilding metadata where available. ## `includeRawHtml` (type: `boolean`): Include HTML-formatted body fields where available. ## `saveRawData` (type: `boolean`): Save raw payloads to the default key-value store for debugging. ## `skipComments` (type: `boolean`): Disable comment extraction even for post URLs unless comment search is enabled. ## `skipUserPosts` (type: `boolean`): Do not collect public posts when scraping user profiles. ## `skipCommunityDetails` (type: `boolean`): Do not collect separate community metadata records for subreddit inputs. ## `sentimentAnalysis` (type: `boolean`): Add sentiment scores and labels to posts and comments. ## `contentAnalysis` (type: `boolean`): Classify posts into a stable topic taxonomy. ## `engagementAnalysis` (type: `boolean`): Add normalized engagement metrics. ## `languageDetection` (type: `boolean`): Detect likely language for post/comment text with a lightweight local heuristic. ## `fastMode` (type: `string`): Use HTTP/direct extraction when possible. ## `maxConcurrency` (type: `integer`): Maximum number of parallel tasks. ## `maxRequestsPerMinute` (type: `integer`): Rate limit for outbound requests. ## `requestTimeoutSecs` (type: `integer`): Timeout for individual requests. ## `maxRequestRetries` (type: `integer`): Maximum retries per request before failure. ## `retryHostFallback` (type: `boolean`): When enabled, retry failed Reddit JSON requests on old.reddit.com and api.reddit.com. Keep disabled for lowest cost because it can multiply failed requests. ## `scrollTimeoutSecs` (type: `integer`): Reserved for browser fallback. ## `debugMode` (type: `boolean`): Enable verbose logging and additional diagnostics. ## `outputMode` (type: `string`): Unified stores all records in the default dataset. Split also writes record-type datasets. ## `deduplicate` (type: `boolean`): Save each Reddit entity once per run using a stable dedupe key. ## `aggregateDuplicateSources` (type: `boolean`): Track all source contexts when the same entity is rediscovered. ## `writeRunSummary` (type: `boolean`): Write RUN-SUMMARY.json to the key-value store. ## `writeHtmlReport` (type: `boolean`): Write RUN-MAP.html dashboard to the key-value store. ## `proxyConfiguration` (type: `object`): Proxy settings. Apify cloud traffic to Reddit is commonly blocked without Residential proxy. Residential proxy is enabled by default for reliability; disable it only if direct access works for your use case. ## Actor input object example ```json { "searchTerms": [], "startUrls": [ { "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/" } ], "subredditUrls": [], "userUrls": [], "withinCommunity": "", "searchPosts": true, "searchComments": false, "searchCommunities": false, "searchUsers": false, "searchMedia": false, "sort": "new", "time": "all", "strictSearch": false, "strictTokenFilter": false, "excludeTerms": [], "subredditSort": "new", "subredditTime": "all", "maximizeCoverage": false, "crawlCommentsPerPost": false, "maxCommentsPerPost": 25, "maxCommentsCount": 50, "commentDepthLimit": 0, "commentSort": "confidence", "commentDateFrom": "", "commentDateTo": "", "dateFrom": "", "dateTo": "", "forceSortNewForTimeFilteredRuns": true, "includeNsfw": false, "maxItems": 100, "maxPostsPerSearch": 25, "maxPostsPerSubreddit": 50, "maxPostsPerUser": 25, "maxCommunitiesCount": 25, "maxUsersCount": 25, "maxRequestsPerRun": 500, "includeMediaLinks": false, "includeAwards": false, "includeRawHtml": false, "saveRawData": false, "skipComments": false, "skipUserPosts": true, "skipCommunityDetails": true, "sentimentAnalysis": false, "contentAnalysis": false, "engagementAnalysis": true, "languageDetection": false, "fastMode": "auto", "maxConcurrency": 6, "maxRequestsPerMinute": 60, "requestTimeoutSecs": 20, "maxRequestRetries": 0, "retryHostFallback": false, "scrollTimeoutSecs": 30, "debugMode": false, "outputMode": "unified", "deduplicate": true, "aggregateDuplicateSources": true, "writeRunSummary": true, "writeHtmlReport": false, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `runSummary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchTerms": [], "startUrls": [ { "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/" } ], "subredditUrls": [] }; // Run the Actor and wait for it to finish const run = await client.actor("qaseemiqbal/reddit-intelligence-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchTerms": [], "startUrls": [{ "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/" }], "subredditUrls": [], } # Run the Actor and wait for it to finish run = client.actor("qaseemiqbal/reddit-intelligence-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchTerms": [], "startUrls": [ { "url": "https://www.reddit.com/r/Baking/comments/1hvoazn/my_best_cheesecake_so_far/" } ], "subredditUrls": [] }' | apify call qaseemiqbal/reddit-intelligence-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=qaseemiqbal/reddit-intelligence-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Reddit Intelligence Scraper", "description": "Collect public Reddit posts, comments, communities, and user profile data from searches, subreddit pages, Reddit URLs, and usernames. Export clean datasets for monitoring, research, and AI workflows.", "version": "0.1", "x-build-id": "ZrUQKg4qQQRh3wju9" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/qaseemiqbal~reddit-intelligence-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-qaseemiqbal-reddit-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/qaseemiqbal~reddit-intelligence-scraper/runs": { "post": { "operationId": "runs-sync-qaseemiqbal-reddit-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/qaseemiqbal~reddit-intelligence-scraper/run-sync": { "post": { "operationId": "run-sync-qaseemiqbal-reddit-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchTerms": { "title": "Search terms", "type": "array", "description": "Keywords or phrases to search across Reddit.", "default": [], "items": { "type": "string" } }, "startUrls": { "title": "Direct Reddit URLs", "type": "array", "description": "Direct Reddit URLs to scrape, such as post URLs, subreddit pages, user pages, all/popular listings, or Reddit search URLs.", "default": [], "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "subredditUrls": { "title": "Full subreddit scrape inputs", "type": "array", "description": "Subreddits to traverse. Accepts names, r/name, or full subreddit URLs.", "default": [], "items": { "type": "string" } }, "userUrls": { "title": "User profile inputs", "type": "array", "description": "Reddit usernames or user profile URLs to scrape.", "default": [], "items": { "type": "string" } }, "withinCommunity": { "title": "Search within subreddit", "type": "string", "description": "Optional subreddit restriction for search terms.", "default": "" }, "searchPosts": { "title": "Search posts", "type": "boolean", "description": "Search for Reddit posts matching the search terms.", "default": true }, "searchComments": { "title": "Search comments", "type": "boolean", "description": "Search for Reddit comments matching the search terms.", "default": false }, "searchCommunities": { "title": "Search communities", "type": "boolean", "description": "Search for subreddits/communities matching the search terms.", "default": false }, "searchUsers": { "title": "Search users", "type": "boolean", "description": "Search for public Reddit users matching the search terms.", "default": false }, "searchMedia": { "title": "Search media posts", "type": "boolean", "description": "Keep media-bearing post results from search.", "default": false }, "sort": { "title": "Search sort", "enum": [ "relevance", "hot", "top", "new", "comments" ], "type": "string", "description": "Ranking method for Reddit search results.", "default": "new" }, "time": { "title": "Search time range", "enum": [ "all", "hour", "day", "week", "month", "year" ], "type": "string", "description": "Reddit-side time window for search results.", "default": "all" }, "strictSearch": { "title": "Strict Reddit search", "type": "boolean", "description": "Use stricter query construction where possible.", "default": false }, "strictTokenFilter": { "title": "Strict saved-record token filter", "type": "boolean", "description": "After collection, keep only records whose searchable text matches the important query tokens.", "default": false }, "excludeTerms": { "title": "Exclude terms", "type": "array", "description": "Terms that disqualify a record when found in searchable text.", "default": [], "items": { "type": "string" } }, "subredditSort": { "title": "Subreddit sort", "enum": [ "hot", "new", "top", "rising", "comments" ], "type": "string", "description": "Ranking method for subreddit listing traversal.", "default": "new" }, "subredditTime": { "title": "Subreddit time range", "enum": [ "all", "hour", "day", "week", "month", "year" ], "type": "string", "description": "Time range for top-style subreddit traversal.", "default": "all" }, "maximizeCoverage": { "title": "Maximum coverage mode", "type": "boolean", "description": "Expand broad queries and subreddits across complementary result views, then deduplicate.", "default": false }, "crawlCommentsPerPost": { "title": "Crawl comments for each post", "type": "boolean", "description": "Collect comments for every post discovered from direct URLs, searches, or subreddits.", "default": false }, "maxCommentsPerPost": { "title": "Max comments per post", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum comments to collect for each post.", "default": 25 }, "maxCommentsCount": { "title": "Max searched comments", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum comments to collect from comment search per search term.", "default": 50 }, "commentDepthLimit": { "title": "Comment depth limit", "minimum": 0, "maximum": 1000, "type": "integer", "description": "Maximum nested comment depth. Use 0 for no depth limit.", "default": 0 }, "commentSort": { "title": "Comment sort", "enum": [ "confidence", "top", "new", "controversial", "old", "qa" ], "type": "string", "description": "Preferred comment ordering where supported.", "default": "confidence" }, "commentDateFrom": { "title": "Comment date from", "type": "string", "description": "Optional lower date bound for comments. Accepts ISO datetime or YYYY-MM-DD.", "default": "" }, "commentDateTo": { "title": "Comment date to", "type": "string", "description": "Optional upper date bound for comments. Accepts ISO datetime or YYYY-MM-DD.", "default": "" }, "dateFrom": { "title": "Post date from", "type": "string", "description": "Optional lower date bound for posts. Accepts ISO datetime or YYYY-MM-DD.", "default": "" }, "dateTo": { "title": "Post date to", "type": "string", "description": "Optional upper date bound for posts. Accepts ISO datetime or YYYY-MM-DD.", "default": "" }, "forceSortNewForTimeFilteredRuns": { "title": "Force chronological traversal for date filters", "type": "boolean", "description": "When a post date filter is active, use chronological traversal where possible.", "default": true }, "includeNsfw": { "title": "Include NSFW content", "type": "boolean", "description": "Include posts or communities marked as NSFW/18+.", "default": false }, "maxItems": { "title": "Global max records", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum dataset records to save across all record types.", "default": 100 }, "maxPostsPerSearch": { "title": "Max posts per search term", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum post records to collect for each search term.", "default": 25 }, "maxPostsPerSubreddit": { "title": "Max posts per subreddit", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum post records to collect for each subreddit input.", "default": 50 }, "maxPostsPerUser": { "title": "Max posts per user", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum public user posts to collect per user input.", "default": 25 }, "maxCommunitiesCount": { "title": "Max communities", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum community records to collect per community search.", "default": 25 }, "maxUsersCount": { "title": "Max users", "minimum": 0, "maximum": 2147483647, "type": "integer", "description": "Maximum user profile records to collect per user search.", "default": 25 }, "maxRequestsPerRun": { "title": "Max requests per run", "minimum": 1, "maximum": 2147483647, "type": "integer", "description": "Safety cap for total queued request processing.", "default": 500 }, "includeMediaLinks": { "title": "Include media links", "type": "boolean", "description": "Extract media URLs, galleries, preview images, videos, and outbound links.", "default": false }, "includeAwards": { "title": "Include awards", "type": "boolean", "description": "Extract Reddit award/gilding metadata where available.", "default": false }, "includeRawHtml": { "title": "Include raw HTML body", "type": "boolean", "description": "Include HTML-formatted body fields where available.", "default": false }, "saveRawData": { "title": "Save raw Reddit payloads", "type": "boolean", "description": "Save raw payloads to the default key-value store for debugging.", "default": false }, "skipComments": { "title": "Skip comments", "type": "boolean", "description": "Disable comment extraction even for post URLs unless comment search is enabled.", "default": false }, "skipUserPosts": { "title": "Skip user posts", "type": "boolean", "description": "Do not collect public posts when scraping user profiles.", "default": true }, "skipCommunityDetails": { "title": "Skip community details", "type": "boolean", "description": "Do not collect separate community metadata records for subreddit inputs.", "default": true }, "sentimentAnalysis": { "title": "Sentiment analysis", "type": "boolean", "description": "Add sentiment scores and labels to posts and comments.", "default": false }, "contentAnalysis": { "title": "Content category analysis", "type": "boolean", "description": "Classify posts into a stable topic taxonomy.", "default": false }, "engagementAnalysis": { "title": "Engagement analysis", "type": "boolean", "description": "Add normalized engagement metrics.", "default": true }, "languageDetection": { "title": "Language detection", "type": "boolean", "description": "Detect likely language for post/comment text with a lightweight local heuristic.", "default": false }, "fastMode": { "title": "Fast mode", "enum": [ "auto", "always", "never" ], "type": "string", "description": "Use HTTP/direct extraction when possible.", "default": "auto" }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum number of parallel tasks.", "default": 6 }, "maxRequestsPerMinute": { "title": "Max requests per minute", "minimum": 1, "maximum": 6000, "type": "integer", "description": "Rate limit for outbound requests.", "default": 60 }, "requestTimeoutSecs": { "title": "Request timeout seconds", "minimum": 5, "maximum": 300, "type": "integer", "description": "Timeout for individual requests.", "default": 20 }, "maxRequestRetries": { "title": "Max request retries", "minimum": 0, "maximum": 20, "type": "integer", "description": "Maximum retries per request before failure.", "default": 0 }, "retryHostFallback": { "title": "Retry alternate Reddit hosts", "type": "boolean", "description": "When enabled, retry failed Reddit JSON requests on old.reddit.com and api.reddit.com. Keep disabled for lowest cost because it can multiply failed requests.", "default": false }, "scrollTimeoutSecs": { "title": "Browser scroll timeout seconds", "minimum": 5, "maximum": 300, "type": "integer", "description": "Reserved for browser fallback.", "default": 30 }, "debugMode": { "title": "Debug mode", "type": "boolean", "description": "Enable verbose logging and additional diagnostics.", "default": false }, "outputMode": { "title": "Output mode", "enum": [ "unified", "split" ], "type": "string", "description": "Unified stores all records in the default dataset. Split also writes record-type datasets.", "default": "unified" }, "deduplicate": { "title": "Deduplicate records", "type": "boolean", "description": "Save each Reddit entity once per run using a stable dedupe key.", "default": true }, "aggregateDuplicateSources": { "title": "Aggregate duplicate sources", "type": "boolean", "description": "Track all source contexts when the same entity is rediscovered.", "default": true }, "writeRunSummary": { "title": "Write run summary", "type": "boolean", "description": "Write RUN-SUMMARY.json to the key-value store.", "default": true }, "writeHtmlReport": { "title": "Write HTML report", "type": "boolean", "description": "Write RUN-MAP.html dashboard to the key-value store.", "default": false }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings. Apify cloud traffic to Reddit is commonly blocked without Residential proxy. Residential proxy is enabled by default for reliability; disable it only if direct access works for your use case.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```