# Reddit Scraper — Posts, Comments & Buying Intent Signals (`automationnation/reddit-intent-scraper`) Actor

Extract Reddit posts and comments with built-in buying-intent detection. Every item is scored for seeking-recommendation, pain-point, switching & comparison signals — deterministic, no LLM call. Pure HTTP architecture, 10x cheaper than browser scrapers. Filter to intent-only for lead gen.

- **URL**: https://apify.com/automationnation/reddit-intent-scraper.md
- **Developed by:** [Nathan Carter](https://apify.com/automationnation) (community)
- **Categories:** Social media, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 reddit post/comment extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Fast Reddit Intent Scraper (API Bypass & JSON)

Extract structured Reddit data at lightning speed. Unlike heavy, expensive browser-based scrapers, this Actor utilizes a pure HTTP/Cheerio architecture to pull posts and nested comments in seconds—without the enterprise API fees.

### 🎯 Why use this Actor?
When Reddit locked down its API, developers and marketers lost access to the best source of unfiltered buying intent on the internet. This scraper bypasses those limits using lightweight HTTP requests and rotating residential proxies.

* **Lightning Fast:** No headless browsers. Uses native HTTP and DOM parsing for instant extraction.
* **Cost-Effective:** Because it doesn't burn Playwright/Puppeteer compute, your Apify credits go 10x further.
* **Highly Reliable:** Built-in automatic retries, proxy rotation, and Rate Limit handling.

### 🎯 Built-in Intent Detection
Every post and comment is scored against 4 deterministic signal categories — no LLM call, so it's free and instant:
- **seeking_recommendation** — "looking for", "recommend a", "alternative to", "any suggestions for"
- **pain_point** — "struggling with", "frustrated with", "sick of"
- **switching** — "switching from", "cancelled my", "looking to replace"
- **comparison** — "vs", "which is better", "worth paying for"

Set `onlyIntentSignals: true` in the input to skip everything that matches zero categories — the output dataset is pre-filtered to actual signal, not a raw post/comment dump you have to filter yourself. Pass your own `intentKeywords` to match on product names or a specific pain point instead of the defaults.

### 📊 What Data Do You Get?
The scraper returns clean, structured JSON ready for LLMs, RAG pipelines, or CSV exports.
- `intentScore`: How many intent categories matched (0 = no signal).
- `matchedIntentLabels`: Which categories matched (e.g. `["seeking_recommendation"]`).
- `postTitle`: The title of the thread.
- `author`: The Reddit username.
- `textContent`: The actual post/comment body (clean text).
- `upvotes`: Current score.
- `commentCount`: Total engagement.
- `postUrl`: Direct link to the content.
- `depth` & `parentId`: Perfect for reconstructing nested comment trees in AI training.

### 💡 Use Cases
* **Lead Generation:** Set `onlyIntentSignals: true` and get only the posts/comments where someone is actively asking for a solution your product provides.
* **Social Listening:** Track brand mentions and competitor complaints (e.g., "alternative to [competitor]") with the `switching` and `comparison` signals.
* **AI Training Data:** Feed clean, scored JSON directly into your LLM or RAG pipeline without HTML bloat.

### ⚡ Pricing
* **Pay Per Event:** You only pay for successful extractions.

# Actor input Schema

## `startUrls` (type: `array`):

Subreddit, user, or post URLs to scrape (e.g. https://www.reddit.com/r/technology/).
## `maxItems` (type: `integer`):

Maximum number of posts + comments to return.
## `intentKeywords` (type: `array`):

Override the built-in buying-intent/pain-point/switching/comparison patterns with your own literal keywords or phrases (e.g. product names, specific pain points). Leave empty to use the defaults.
## `onlyIntentSignals` (type: `boolean`):

If enabled, posts/comments that match zero intent patterns are excluded from the output — the dataset is pre-filtered to actual signal instead of a full scrape.
## `proxyConfiguration` (type: `object`):

Proxy settings. Residential recommended to avoid Reddit's datacenter-IP blocking.

## Actor input object example

```json
{
  "startUrls": [
    "https://www.reddit.com/r/technology/"
  ],
  "maxItems": 1000,
  "onlyIntentSignals": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
````

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://www.reddit.com/r/technology/"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automationnation/reddit-intent-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://www.reddit.com/r/technology/"] }

# Run the Actor and wait for it to finish
run = client.actor("automationnation/reddit-intent-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://www.reddit.com/r/technology/"
  ]
}' |
apify call automationnation/reddit-intent-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=automationnation/reddit-intent-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Reddit Scraper — Posts, Comments & Buying Intent Signals",
        "description": "Extract Reddit posts and comments with built-in buying-intent detection. Every item is scored for seeking-recommendation, pain-point, switching & comparison signals — deterministic, no LLM call. Pure HTTP architecture, 10x cheaper than browser scrapers. Filter to intent-only for lead gen.",
        "version": "0.1",
        "x-build-id": "obXFxYrNZnb8w5F8T"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/automationnation~reddit-intent-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-automationnation-reddit-intent-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/automationnation~reddit-intent-scraper/runs": {
            "post": {
                "operationId": "runs-sync-automationnation-reddit-intent-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/automationnation~reddit-intent-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-automationnation-reddit-intent-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "startUrls"
                ],
                "properties": {
                    "startUrls": {
                        "title": "Start URLs",
                        "type": "array",
                        "description": "Subreddit, user, or post URLs to scrape (e.g. https://www.reddit.com/r/technology/).",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxItems": {
                        "title": "Max Items",
                        "minimum": 1,
                        "type": "integer",
                        "description": "Maximum number of posts + comments to return.",
                        "default": 1000
                    },
                    "intentKeywords": {
                        "title": "Custom Intent Keywords (optional)",
                        "type": "array",
                        "description": "Override the built-in buying-intent/pain-point/switching/comparison patterns with your own literal keywords or phrases (e.g. product names, specific pain points). Leave empty to use the defaults.",
                        "items": {
                            "type": "string"
                        }
                    },
                    "onlyIntentSignals": {
                        "title": "Only Return Intent Signals",
                        "type": "boolean",
                        "description": "If enabled, posts/comments that match zero intent patterns are excluded from the output — the dataset is pre-filtered to actual signal instead of a full scrape.",
                        "default": false
                    },
                    "proxyConfiguration": {
                        "title": "Proxy Configuration",
                        "type": "object",
                        "description": "Proxy settings. Residential recommended to avoid Reddit's datacenter-IP blocking.",
                        "default": {
                            "useApifyProxy": true,
                            "apifyProxyGroups": [
                                "RESIDENTIAL"
                            ]
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```