# Reddit Comment Scraper Pro | No Login | No Cookie (`rexreus/reddit-comment-scraper`) Actor

Scrape Reddit comments at scale with zero browser overhead. Extracts nested comment trees, deep replies, sentiment scores, and media URLs in bulk. 10x faster and ultra-low memory usage (<150MB). Perfect for AI training & NLP sentiment.

- **URL**: https://apify.com/rexreus/reddit-comment-scraper.md
- **Developed by:** [REXREUS D.O](https://apify.com/rexreus) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

<p align="center">
  <h1 align="center">Reddit Comment Scraper</h1>
  <img src="https://api.apify.com/v2/key-value-stores/06XtAnxPDRIXwrVmt/records/reddit-comment-scraper-banner.png" alt="High speed bulk Reddit comment scraping and sentiment analysis without browser bloat" align="center" width="100%">
  <p align="center">
    <strong>Extract Reddit comment trees, discussion hierarchies, and sentiment analysis at massive scale.</strong>
  </p>
  <p align="center">
    <em>Pure HTTP Tree Walker · Crawlee & got-scraping · Zero Browser Overhead (<150MB RAM) · Apify Residential Proxy Rotation</em>
  </p>
  <p align="center">
    <a href="#-quick-start"><img src="https://img.shields.io/badge/Quick_Start-Apify_Console-green?style=flat-square" alt="Quick Start"></a>
    <a href="https://apify.com"><img src="https://img.shields.io/badge/Runs_on-Apify-ff443e?style=flat-square" alt="Apify"></a>
    <a href="https://nodejs.org"><img src="https://img.shields.io/badge/Node.js-≥20-339933?style=flat-square&logo=node.js&logoColor=white" alt="Node.js"></a>
    <a href="https://www.typescriptlang.org/"><img src="https://img.shields.io/badge/TypeScript-5.3-blue?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript"></a>
    <a href="https://crawlee.dev/"><img src="https://img.shields.io/badge/Crawlee-v3-orange?style=flat-square" alt="Crawlee"></a>
  </p>
</p>

***

### ⚡ Why this Actor?

| Feature | What you get |
| :--- | :--- |
| **Pure HTTP Engine (No Browser)** | **10x faster and 90% cheaper** than browser scrapers (consumes <150MB RAM vs 1GB+ with Playwright). |
| **Anti-Bot TLS Fingerprinting** | Uses `got-scraping` to negotiate genuine browser TLS Client Hello (JA3/JA4) and HTTP/2 headers to bypass Fastly CDN blocks. |
| **Residential Proxy Auto-Rotation** | Seamlessly integrates with Apify `RESIDENTIAL` proxies; retires session and swaps to a clean IP immediately upon 429/403. |
| **Deep Collapsed Comments** | Automatically calls `POST /api/morechildren` in batches of 100 to extract deeply buried comments often missed by basic scrapers. |
| **Full Thread Hierarchy** | Outputs `threadId` (root comment), `ancestorIds` path array, and `depth` for zero-friction tree reconstruction in Pandas or BI tools. |
| **Lightweight Sentiment Analysis** | Evaluates in-memory polarity scores (`score`, `comparative`, `label`) for every comment at <0.01ms latency without expensive AI tokens. |
| **Link & Media Extractor** | Parses markdown hyperlinks, images, videos, and embedded GIFs (`i.redd.it`, Giphy, Imgur). |
| **Multi-Target Ingestion** | Supports direct post URLs, shortlinks (`redd.it`), raw post IDs, or whole subreddits. |

***

### 🚀 Quick start

#### Apify Console

1. Open **Reddit Comment Scraper** in the Apify Store and click **Try for free** or **Start**.
2. Paste target Reddit post URLs or subreddit names in the input form.
3. Click **Save & Run**.
4. Download structured results in JSON, CSV, or Excel from **Storage → Dataset**.

***

### 🔄 How it works

```text
Input (URLs / Subreddits / Post IDs)
  │
  ▼
[Input Normalizer] ──► Deduplicate IDs & extract endpoints
  │
  ▼
[Crawlee HttpCrawler] (got-scraping + SessionPool + Apify Residential Proxies)
  │
  ├──► GET /comments/{id}.json?raw_json=1&limit=500
  │      │
  │      ├─► [Comment Tree Parser] ──► Extract authors, body, scores, depth
  │      │
  │      ├─► [Sentiment Service] ──► In-memory polarity score & label
  │      │
  │      └─► [Hierarchy Mapper] ──► Calculate threadId & ancestorIds
  │
  ├──► POST /api/morechildren (Unfold collapsed comments in 100-ID batches)
  │
  ▼
[Streaming Dataset Output] ──► Actor.pushData() in 100-record chunks
```

***

### 📥 Input Reference

```json
{
  "startUrls": [
    { "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/" }
  ],
  "sort": "top",
  "maxCommentsPerPost": 100,
  "maxDepth": 10,
  "expandMoreComments": true,
  "enableSentiment": true,
  "extractMedia": true,
  "minScore": 0,
  "skipDeleted": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

| Field | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `startUrls` | Array | `[]` | List of target Reddit post URLs. |
| `subreddits` | Array | `[]` | Subreddit names to fetch latest posts from (e.g. `technology`, `webdev`). |
| `postIds` | Array | `[]` | Raw Reddit post IDs (e.g. `1i3b62l`). |
| `postsPerSubreddit` | Integer | `10` | Number of recent posts to crawl when subreddits are provided. |
| `sort` | String | `"confidence"` | Comment sort: `confidence` (Best), `top`, `new`, `controversial`, `old`, `qa`. |
| `maxCommentsPerPost` | Integer | `100` | Max comments to scrape per post (`0` = all reachable). |
| `maxDepth` | Integer | `10` | Max thread reply depth. |
| `expandMoreComments` | Boolean | `true` | When true, fetches collapsed "Load more" comments. |
| `minScore` | Integer | `0` | Excludes comments below this upvote threshold. |
| `skipDeleted` | Boolean | `true` | Excludes `[deleted]` and `[removed]` comments. |
| `keywordFilter` | Array | `[]` | Keeps only comments containing specified terms. |
| `dateFrom` / `dateTo` | String | `null` | ISO date filter range (`YYYY-MM-DD`). |
| `enableSentiment` | Boolean | `true` | Computes in-memory sentiment scores and labels. |
| `extractMedia` | Boolean | `true` | Parses hyperlinks and media URLs from markdown. |
| `proxyConfiguration` | Object | `RESIDENTIAL` | Apify Residential proxy configuration. |
| `maxConcurrency` | Integer | `15` | Concurrency limit (10–30 recommended). |

***

### 📤 Output Contract

Every scraped record pushed to the dataset adheres to this structure:

```json
{
  "id": "t1_k9z1abc",
  "postId": "t3_1i3b62l",
  "postTitle": "OpenAI Operator Live Demo",
  "postUrl": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/",
  "postAuthor": "tech_editor",
  "subreddit": "technology",
  "subredditId": "t5_2qh16",
  "parentId": "t1_k9y0xyz",
  "threadId": "t1_k9top1",
  "ancestorIds": ["t1_k9top1", "t1_k9y0xyz"],
  "author": "dev_guru",
  "authorFullname": "t2_8q4l1",
  "authorFlairText": "Software Engineer",
  "body": "This is great progress! The speed and accuracy are impressive: https://example.com/demo",
  "bodyHtml": "<p>This is great progress! The speed and accuracy are impressive: <a href=\"https://example.com/demo\">https://example.com/demo</a></p>",
  "score": 45,
  "createdUtc": 1772755200,
  "createdAtIso": "2026-03-06T00:00:00.000Z",
  "permalink": "/r/technology/comments/1i3b62l/openai_operator_live_demo/k9z1abc/",
  "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/k9z1abc/",
  "isSubmitter": false,
  "distinguished": null,
  "stickied": false,
  "depth": 2,
  "replyCount": 3,
  "edited": false,
  "wordCount": 12,
  "charCount": 92,
  "sentiment": {
    "score": 6,
    "comparative": 0.5,
    "label": "positive"
  },
  "extractedUrls": ["https://example.com/demo"],
  "mediaUrls": []
}
```

***

### 💰 Cost, Limits & Proxies

| Metric | Specification | Note |
| :--- | :--- | :--- |
| **Container Memory** | **512 MB** | Sufficient for high-concurrency HTTP streaming. |
| **Concurrency** | **15 – 30 workers** | Adjust via `maxConcurrency`. |
| **Speed** | **30 – 60 comments/sec** | Varies based on residential proxy latency. |
| **Compute Cost** | **~$0.01 – $0.02** | Per 10,000 comments scraped. |
| **Proxy Bandwidth** | **~15 MB** | Per 10,000 comments (~$0.15 on Apify Residential proxy). |
| **Proxy Type** | **Apify RESIDENTIAL** | Strongly recommended. Datacenter IPs are blocked by Reddit. |

***

### 🍳 Recipes

#### Recipe 1: Single Thread Deep Dive

```json
{
  "startUrls": [{ "url": "https://www.reddit.com/r/webdev/comments/17abcde/slug/" }],
  "maxCommentsPerPost": 0,
  "maxDepth": 20,
  "expandMoreComments": true,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}
```

#### Recipe 2: Subreddit Topic Sentiment Mining

```json
{
  "subreddits": ["technology", "artificial"],
  "postsPerSubreddit": 25,
  "maxCommentsPerPost": 100,
  "sort": "top",
  "enableSentiment": true,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}
```

***

### 🏛️ Architecture

```text
src/
├── main.ts                       # Composition & Actor lifecycle
├── domain/                       # Entity schemas & sentiment logic
│   ├── comment.entity.ts         # TypeScript models
│   └── sentiment.service.ts      # In-memory polarity calculator
├── input/                        # Trust boundary input validation
│   └── comment-input.ts          # URL normalizer & deduplicator
├── parsers/                      # Hierarchy & regex parsers
│   └── comment-tree.parser.ts    # Recursive flattener & tree mapping
└── clients/                      # Network & anti-bot transport
    └── reddit-json.client.ts     # got-scraping with 429/403 backoff & morechildren
```

***

### 🛠️ Development & Testing

```bash
## Run unit tests
npx vitest run

## Validate all schemas
apify validate-schema

## Run readiness audit
node scripts/validate-actor-readiness.js
```

***

### ❓ Frequently Asked Questions (FAQ)

**Q: Do I need a Reddit account or Reddit API token?**\
A: No. The Actor retrieves public discussions directly without requiring personal accounts or paid official Reddit API credentials.

**Q: Why do I need Residential Proxies?**\
A: Reddit's CDN (Fastly) blocks datacenter IP blocks (AWS, Google Cloud, DigitalOcean) with HTTP 403 Forbidden. Apify Residential proxies route requests through genuine residential IPs to ensure unblocked operation.

**Q: How does the Actor handle collapsed or "load more" comments?**\
A: The Actor identifies `kind: "more"` objects and automatically batches IDs into `POST /api/morechildren` requests (up to 100 IDs per call) until all reachable comments are retrieved.

**Q: Can I use this for academic or commercial NLP datasets?**\
A: Yes. The output includes `threadId` and `ancestorIds`, allowing you to reconstruct tree graphs for argument mining, conversational AI, and sentiment analysis.

**Q: Will this actor get blocked by Reddit?**\
A: The Actor uses `got-scraping` to negotiate real browser TLS Client Hello signatures and headers, combined with residential IP rotation on 429 or 403.

***

### ⚖️ Legal & Limitations

- **Public Data Only**: This Actor extracts publicly available comments and submissions. It cannot access private subreddits or quarantined/banned communities.
- **GDPR & Privacy**: Scraped data contains publicly posted usernames. Users are responsible for complying with relevant data protection regulations.

# Actor input Schema

## `startUrls` (type: `array`):

Direct URLs to Reddit posts or comment permalinks. Example: https://www.reddit.com/r/technology/comments/1i3b62l/openai\_operator\_live\_demo/

## `subreddits` (type: `array`):

Optional: list of subreddit names (without 'r/') to scrape comments from recent posts.

## `postIds` (type: `array`):

Optional: raw Reddit post IDs (e.g., '1i3b62l' or 't3\_1i3b62l').

## `postsPerSubreddit` (type: `integer`):

If subreddits are specified, number of recent posts to scrape per subreddit.

## `sort` (type: `string`):

How comments should be sorted within the thread.

## `maxCommentsPerPost` (type: `integer`):

Maximum number of comments to extract per post (0 = extract all reachable).

## `maxDepth` (type: `integer`):

Maximum reply depth (0 = top-level only, 1 = direct replies, 10 = deep threads).

## `expandMoreComments` (type: `boolean`):

When true, calls Reddit's morechildren API to retrieve deeply nested and collapsed comments.

## `minScore` (type: `integer`):

Exclude comments with net upvotes below this threshold.

## `skipDeleted` (type: `boolean`):

Skip comments marked as '\[deleted]' or '\[removed]'.

## `keywordFilter` (type: `array`):

Only include comments containing at least one of these keywords/phrases (case-insensitive). Leave empty to disable.

## `dateFrom` (type: `string`):

Only include comments posted after this date. Example: '2026-01-01'.

## `dateTo` (type: `string`):

Only include comments posted before this date. Example: '2026-12-31'.

## `enableSentiment` (type: `boolean`):

Calculate fast in-memory sentiment score, comparative value, and polarity label (positive/neutral/negative).

## `extractMedia` (type: `boolean`):

Parse external HTTP URLs and embedded media (GIFs, images) from the markdown content.

## `includePostContext` (type: `boolean`):

Attach post title, post author, and post URL to each comment record.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration. RESIDENTIAL proxy is strongly recommended for Reddit.

## `maxConcurrency` (type: `integer`):

Maximum number of simultaneous requests. Recommended: 10 - 30.

## `debugLog` (type: `boolean`):

Turn on verbose logging for troubleshooting.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/"
    }
  ],
  "postsPerSubreddit": 10,
  "sort": "confidence",
  "maxCommentsPerPost": 100,
  "maxDepth": 10,
  "expandMoreComments": true,
  "minScore": 0,
  "skipDeleted": true,
  "enableSentiment": true,
  "extractMedia": true,
  "includePostContext": true,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  },
  "maxConcurrency": 15,
  "debugLog": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rexreus/reddit-comment-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/" }] }

# Run the Actor and wait for it to finish
run = client.actor("rexreus/reddit-comment-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.reddit.com/r/technology/comments/1i3b62l/openai_operator_live_demo/"
    }
  ]
}' |
apify call rexreus/reddit-comment-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rexreus/reddit-comment-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/keC4Ipq6twwqHITGB/builds/bOLCGE7d2HUf8gRdu/openapi.json