# Reddit Scraper - Posts, Comments & Users (`titan_network/dataclawd-reddit-data-collector`) Actor

Scrape Reddit posts, comments, subreddits, users, activity, and search through 32 API endpoints with automatic pagination and structured Dataset output. Powered by Titan Network.

- **URL**: https://apify.com/titan\_network/dataclawd-reddit-data-collector.md
- **Developed by:** [Titan Network](https://apify.com/titan_network) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 37.5% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.14 / 1,000 item collecteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Reddit Scraper - Posts, Comments & Users

This **Reddit scraper** collects public posts, comments, subreddits, users, profiles, activity, and search results through the **Dataclawd API**. Choose a task or endpoint, collect one or more targets with automatic `after` pagination, and receive one Dataset row for each returned item.

It covers **32 endpoints** for subreddit listings and information, post details and comments, global popular posts, post/user/subreddit search, and user profiles and activity.

> Powered by Dataclawd API · Not affiliated with Reddit, Inc.

### What you can scrape

- Public Reddit posts, post comments, post duplicates, and global popular listings.
- Public subreddits, including listings, information, rules, similar communities, and subreddit search results.
- Public user profiles, activity, posts, comments, and user search results.

### Use cases

- Research public subreddit posts and comments around a topic or community.
- Compare public subreddits, rules, and related communities for community research.
- Build structured datasets from public Reddit users, activity, and search results.

### FAQ

**How does pagination work?** The Actor follows Reddit's `after` cursor when `paginationParam` is `auto`; use `maxItems`, `maxPages`, or `startCursor` to control or resume collection.

**Can I collect multiple targets?** Yes. Add parameter objects to `targets`; the Actor processes each target in sequence.

**Where do credentials come from?** Dataclawd API credentials are configured by the publisher for this Actor.

**What public data can I collect?** Collect only data you are authorized to collect from the public endpoint responses listed below; availability can vary by endpoint and platform state.

### Quick start

1. Open the Actor's **Input** tab.
2. Pick a **task** (采集任务), e.g. `subredditHot` — the endpoint is selected automatically.
3. Fill **subreddit**, e.g. `java`.
4. Optionally set `limit` (per-page count, max 100), `maxItems` / `maxPages` to control volume, or add multiple objects to **targets** to batch-collect several subreddits.
5. Click **Run** and open the **Dataset** tab when it finishes — every collected post is one row with the raw JSON in the `data` field.

### Input

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `task` | enum | — | **Recommended.** Collection intent, auto-maps to an endpoint (see Tasks below). Leave empty only if you set `endpoint` explicitly. |
| `endpoint` | enum | `auto` | Advanced. `auto` = resolve from `task`; or set an explicit endpoint to override. |
| `subreddit` | string | — | Subreddit name without `r/` (e.g. `java`). Needed by `postDetail`/`postComments`/`postDuplicates` and all `subreddit*` endpoints. |
| `username` | string | — | Reddit username without `u/` (e.g. `elonmusk`). Needed by all `user*` endpoints. |
| `article` | string | — | Post ID from the comments path (e.g. `1b7abcd`). Needed by `postDetail`/`postComments`/`postDuplicates`. |
| `q` | string | — | Search keyword. Needed by `search` / `searchSubreddits` / `searchUsers`. |
| `sort` | string | — | Listing sort: `hot`/`new`/`top`/`rising`/`best` (popular) or `confidence`/`top`/`new`/`controversial`/`old`/`random`/`qa`/`live` (comments). |
| `country` | string | — | ISO country code (e.g. `US`) for `popularCountry`. |
| `restrictSr` | string | — | Set `true` to restrict `search` to a subreddit (with `subreddit`). |
| `after` | string | *(empty)* | Reddit pagination cursor (e.g. `t3_1b7abcd`) to start from. |
| `limit` | integer | `25` | Items per page, max 100. |
| `t` | enum | *(empty)* | Time filter for `top` endpoints: `hour` / `day` / `week` / `month` / `year` / `all`. |
| `targets` | array of object | *(empty)* | Batch mode: each element is a param set, collected one after another. |
| `proxyUrl` | string | *(empty)* | Proxy used by Dataclawd's server when fetching Reddit. |
| `httpMethod` | enum | `GET` | Dataclawd Reddit endpoints are GET-only; `POST` returns 501. |
| `maxItems` | integer | `100` | Stop after this many items in total (max 100 000). |
| `maxPages` | integer | `10` | Max pages per target (max 100). |
| `startCursor` | string | *(empty)* | Alias for `after`; resume from a previous cursor. |
| `itemsPath` | string | *(empty)* | Dot-path to the item list, e.g. `data.children`. Leave empty for auto-detection (Reddit Listing `children` are auto-unwrapped). |
| `nextCursorPath` | string | *(empty)* | Dot-path to the next-page cursor. Leave empty for auto-detection (`data.after`). |
| `paginationParam` | enum | `auto` | Query param used for paging: `auto` (after), `cursor`, or `none`. |

#### Endpoints

#### Tasks

Pick a `task` and the Actor selects the endpoint for you. Fill the params below (`subreddit` / `username` / `article` / `q` …).

| Task | Endpoint | Required params |
| --- | --- | --- |
| `popular` 全局热门 | `popular` | — |
| `subredditHot` Subreddit 热门 | `subredditHot` | `subreddit` |
| `subredditNew` Subreddit 最新 | `subredditNew` | `subreddit` |
| `subredditTop` Subreddit 最佳 | `subredditTop` | `subreddit` |
| `postDetail` 帖子详情 | `postDetail` | `subreddit` + `article` |
| `postComments` 帖子评论 | `postComments` | `subreddit` + `article` |
| `search` 搜索 | `search` | `q` |
| `searchSubreddits` 搜索社区 | `searchSubreddits` | `q` |
| `searchUsers` 搜索用户 | `searchUsers` | `q` |
| `userProfile` 用户资料 | `userProfile` | `username` |
| `userPosts` 用户帖子 | `userPosts` | `username` |
| `userComments` 用户评论 | `userComments` | `username` |

**Posts** — `postDetail`, `postComments`, `postDuplicates`
**Popular / homepage** — `popular`, `popularBest`, `popularCountry`, `popularRising`, `popularTop`
**Search** — `search`, `searchSubreddits`, `searchUsers`
**Subreddit** — `subredditHot`, `subredditNew`, `subredditTop`, `subredditAbout`, `subredditComments`, `subredditControversial`, `subredditPosts`, `subredditRules`, `subredditSimilar`, `subredditsNew`, `subredditsPopular`
**User** — `userAbout`, `userActivity`, `userComments`, `userCommentsTop`, `userOverview`, `userPosts`, `userPostsHot`, `userPostsNew`, `userPostsTop`, `userProfile`

### Output (Dataset — one row per collected item)

| Field | Description |
| --- | --- |
| `platform` | Always `reddit`. |
| `endpoint` | The endpoint that produced this row. |
| `status` | `completed` (data collected) or `failed` (request-level failure). |
| `target` | Human-readable target summary, e.g. `subreddit=java`. |
| `page` | Page number this item came from (1-based). |
| `itemIndex` | Sequential index within the run (1-based). |
| `cursor` | Cursor used to fetch that page (`-1` = first page). |
| `data` | **The collected item** — the raw JSON returned by Dataclawd (a post or user object; Reddit Listing wrappers are unwrapped). |
| `errorStage` | Where a failure happened: `connect_failed` / `server_error` / `auth_error` / `rate_limited` / `request_error` / `parse_error` / `empty_result`. |
| `errorCode` | HTTP status code (if any), e.g. `429`. |
| `error` | Error message on failed rows. |

#### Run status & status message

While running, the Actor's **status message** is updated periodically (`开始采集 …` → `采集 … 中：已写入 n 条` → `完成：…`). The run **succeeds** (`SUCCEEDED`) as soon as **at least one item** is collected. If every target fails or returns nothing, the run ends `FAILED` with an explanatory message. Billing happens only for `completed` rows.

### How to verify the results

1. **Run status**: the run is `SUCCEEDED` when at least one item was written; the status message shows live progress.
2. **Per-item state**: open the run's **Dataset** — every collected item has `status=completed` with the raw JSON in `data`; failed rows explain why (`errorStage` / `errorCode` / `error`).
3. **Spot-check against the API**: re-run one target in the Dataclawd API Console and compare with the row's `data`.
4. **Billing**: only `completed` rows carry the `item-collected` pay-per-event charge.

> Developer note: for automated checks (run status, row schema, optional Dataclawd API replay), use `python3 tools/verify_output.py --run <RUN_ID> [--api]`.

### Pricing

Pay-per-event (PPE): **one `item-collected` event per successfully collected item**. Failed rows do **not** carry this event.

> Publishing note: define an `item-collected` event with your price in the Actor's **Monetization** tab. The platform's built-in synthetic event `apify-default-dataset-item` is charged automatically per default-dataset item when PPE is enabled; set its price to `0` if you don't want a per-item base fee.

### Configuration (publisher)

The following environment variables are configured on the Actor's version settings and are not visible to end users:

- `DATACLAWD_API_HOST` — Dataclawd API base URL (e.g. `https://api.dataclawd.example`).
- `DATACLAWD_API_KEY` — Dataclawd API key (sent as `X-API-Key`).

### Disclaimer

This Actor is not affiliated with, endorsed by, or sponsored by Reddit, Inc. Collect only data you are authorized to collect, and comply with Reddit's Terms of Service, the Dataclawd service terms, and applicable law.

***

### 中文简介

本 Actor 通过 **Dataclawd API** 批量采集 Reddit 数据（**32 个端点**）：subreddit 热门/最新/最佳/争议帖子、subreddit 资料/规则/相似社区、帖子详情与评论、全局热门帖子、帖子/社区/用户搜索，以及用户资料/动态/评论/帖子。选定采集端点、自动按 `after` 游标分页、自动提取条目列表（Reddit Listing 的 `children` 会自动解包为帖子对象），并把每条数据以 JSON 写入 Apify 默认 Dataset（成功行同时也是按事件计费的计费事件）。

- **输入**：`endpoint`（采集端点，必填）、`subreddit` / `username` / `article` / `q`（端点对应主体，`search*` 用 `q`）、`sort`/`country`/`restrictSr`（排序/国家/限定搜索）、`limit`（每页条数，最大 100）、`t`（top 端点时间范围）、`targets`（批量目标列表，可选）、`maxItems`/`maxPages`（采集量控制）、`after`/`startCursor`（续采）、`itemsPath`/`nextCursorPath`（高级：显式指定提取路径与游标路径）、`paginationParam`（分页参数）。
- **输出**：每行包含 `platform`、`endpoint`、`status`（`completed`/`failed`）、`target`、`page`、`itemIndex`、`cursor` 和采集到的原始数据 `data`；失败行带 `errorStage`（`connect_failed`/`server_error`/`auth_error`/`rate_limited`/`request_error`/`parse_error`/`empty_result`）、`errorCode`、`error`。运行中实时进度通过 Apify 状态消息展示，至少采到 1 条即运行成功（`SUCCEEDED`），全部失败/空结果则运行失败（`FAILED`）。
- **计费**：按事件计费（PPE），每条成功采集计一个 `item-collected` 事件；失败行不计费（发布时需在 Console Monetization 中配置该事件及价格）。
- **验证数据**：`status=completed` 即成功采集（数据在 `data` 字段）；开发时可运行 `python3 tools/verify_output.py --run <RUN_ID>` 校验。
- **免责声明**：本 Actor 与 Reddit, Inc. 无任何附属关系。请仅采集你被授权采集的数据，并遵守 Reddit 服务条款、Dataclawd 服务条款与当地法律。

# Actor input Schema

## `task` (type: `string`):

要采集的 Reddit 数据场景。选一个即可，Actor 会自动映射到对应端点；参数填在下方对应字段（subreddit/username/article/q 等）。留空则须在「高级选项」里手动指定 endpoint。 / The Reddit data scenario to collect. Pick one and the Actor maps it to the corresponding endpoint; fill in the related fields below (subreddit/username/article/q, etc.). Leave empty to manually specify the endpoint under Advanced options.

## `endpoint` (type: `string`):

高级选项：默认 auto（按「采集任务 task」自动选择端点）；也可手动指定具体端点覆盖 task。 / Advanced options: default auto (selects the endpoint automatically based on the collection task); you can also manually specify an endpoint to override task.

## `subreddit` (type: `string`):

subreddit 名称（不要带 r/ 前缀），例如 java。subreddit\* 与 postDetail 端点需要。 / Subreddit name (without the r/ prefix), e.g. java. Required by subreddit\* and postDetail endpoints.

## `username` (type: `string`):

Reddit 用户名（不要带 u/ 前缀），例如 elonmusk。user\* 端点需要。 / Reddit username (without the u/ prefix), e.g. elonmusk. Required by user\* endpoints.

## `article` (type: `string`):

帖子 ID（postDetail 端点必填，评论区路径中的 ID），例如 1b7abcd。 / Post ID (required by the postDetail endpoint; the ID in the comments path), e.g. 1b7abcd.

## `after` (type: `string`):

Reddit 分页游标（如 t3\_1b7abcd），从指定位置开始。留空从第一页开始。 / Reddit pagination cursor (e.g. t3\_1b7abcd) to start from that position. Leave empty to start from the first page.

## `limit` (type: `integer`):

每页返回条数，最大 100。 / Number of items returned per page, up to 100.

## `t` (type: `string`):

top 端点的时间过滤：hour / day / week / month / year / all。留空使用服务端默认。 / Time filter for top endpoints: hour / day / week / month / year / all. Leave empty for the server default.

## `targets` (type: `array`):

批量采集：每个元素是一组参数（覆盖上面各字段），Actor 会逐个目标采集。例如 \[{"subreddit":"java"},{"subreddit":"python"}]。 / Batch collection: each element is a set of parameters (overriding the fields above); the Actor collects each target one by one. For example \[{"subreddit":"java"},{"subreddit":"python"}].

## `proxyUrl` (type: `string`):

Dataclawd 服务端请求 Reddit 时使用的代理地址。 / The proxy address used by the Dataclawd server when requesting Reddit.

## `httpMethod` (type: `string`):

调用 Dataclawd 端点使用的 HTTP 方法（Reddit 端点在 Dataclawd 后端仅支持 GET，POST 会返回 501）。 / HTTP method used to call the Dataclawd endpoint (the Dataclawd backend only supports GET for Reddit endpoints; POST returns 501).

## `maxItems` (type: `integer`):

整个运行最多写入 Dataset 的条目数，达到即停止。 / Maximum number of items written to the Dataset for the whole run; stop once reached.

## `maxPages` (type: `integer`):

每个目标最多翻页次数（防止无限翻页）。 / Maximum number of pages per target (to prevent infinite pagination).

## `startCursor` (type: `string`):

从指定游标开始翻页（续采上次未完成的数据）。留空从第一页开始。 / Start pagination from the given cursor (to continue previously unfinished data). Leave empty to start from the first page.

## `itemsPath` (type: `string`):

点号路径，指定响应中条目列表的位置，例如 data.children。留空则自动探测（Reddit Listing 的 children 会被自动解包）。 / Dot-separated path to the item list in the response, e.g. data.children. Leave empty to auto-detect (the children of a Reddit Listing are unwrapped automatically).

## `nextCursorPath` (type: `string`):

点号路径，指定响应中下一页游标的位置。留空自动探测（优先 data.after）。 / Dot-separated path to the next-page cursor in the response. Leave empty to auto-detect (data.after is preferred).

## `paginationParam` (type: `string`):

翻页时使用的查询参数：auto 按端点自动选择（默认 after）；cursor 用于 X 风格分页；none 不翻页。 / Query parameter used for pagination: auto selects by endpoint (default after); cursor is for X-style pagination; none disables pagination.

## `q` (type: `string`):

search 系列端点必填，例如 spring boot。 / Required by search\* endpoints, e.g. spring boot.

## `sort` (type: `string`):

popular/post 评论的排序：hot / new / top / rising / best（popular）；confidence / top / new / controversial / old / random / qa / live（评论）。 / Sorting for popular/post comments: hot / new / top / rising / best (popular); confidence / top / new / controversial / old / random / qa / live (comments).

## `country` (type: `string`):

popular/country 端点的国家/地区过滤（ISO 国家代码），例如 US / GB / DE。 / Country/region filter for popular/country endpoints (ISO country code), e.g. US / GB / DE.

## `restrictSr` (type: `string`):

search 端点：restrictSr=true 时只在 subreddit 内搜索。 / For search endpoints: when restrictSr=true, search only within the subreddit.

## Actor input object example

```json
{
  "task": "subredditHot",
  "endpoint": "auto",
  "subreddit": "java",
  "limit": 25,
  "targets": [],
  "httpMethod": "GET",
  "maxItems": 5,
  "maxPages": 1,
  "paginationParam": "auto"
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset of collected Reddit records — one item per record.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "task": "subredditHot",
    "endpoint": "auto",
    "subreddit": "java",
    "targets": [],
    "maxItems": 5,
    "maxPages": 1
};

// Run the Actor and wait for it to finish
const run = await client.actor("titan_network/dataclawd-reddit-data-collector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "task": "subredditHot",
    "endpoint": "auto",
    "subreddit": "java",
    "targets": [],
    "maxItems": 5,
    "maxPages": 1,
}

# Run the Actor and wait for it to finish
run = client.actor("titan_network/dataclawd-reddit-data-collector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "task": "subredditHot",
  "endpoint": "auto",
  "subreddit": "java",
  "targets": [],
  "maxItems": 5,
  "maxPages": 1
}' |
apify call titan_network/dataclawd-reddit-data-collector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,titan_network/dataclawd-reddit-data-collector"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/XHCPy2MgAI3FBDcoj/builds/pnxRx1n1f4OeVY1Js/openapi.json