Reddit Scraper - Posts, Comments & Users

This Reddit scraper collects public posts, comments, subreddits, users, profiles, activity, and search results through the Dataclawd API. Choose a task or endpoint, collect one or more targets with automatic after pagination, and receive one Dataset row for each returned item.

It covers 32 endpoints for subreddit listings and information, post details and comments, global popular posts, post/user/subreddit search, and user profiles and activity.

Powered by Dataclawd API · Not affiliated with Reddit, Inc.

What you can scrape

• Public Reddit posts, post comments, post duplicates, and global popular listings.
• Public subreddits, including listings, information, rules, similar communities, and subreddit search results.
• Public user profiles, activity, posts, comments, and user search results.

Use cases

• Research public subreddit posts and comments around a topic or community.
• Compare public subreddits, rules, and related communities for community research.
• Build structured datasets from public Reddit users, activity, and search results.

FAQ

How does pagination work? The Actor follows Reddit's after cursor when paginationParam is auto; use maxItems, maxPages, or startCursor to control or resume collection.

Can I collect multiple targets? Yes. Add parameter objects to targets; the Actor processes each target in sequence.

Where do credentials come from? Dataclawd API credentials are configured by the publisher for this Actor.

What public data can I collect? Collect only data you are authorized to collect from the public endpoint responses listed below; availability can vary by endpoint and platform state.

Quick start

1. Open the Actor's Input tab.
2. Pick a task (采集任务), e.g. subredditHot — the endpoint is selected automatically.
3. Fill subreddit, e.g. java.
4. Optionally set limit (per-page count, max 100), maxItems / maxPages to control volume, or add multiple objects to targets to batch-collect several subreddits.
5. Click Run and open the Dataset tab when it finishes — every collected post is one row with the raw JSON in the data field.

Input

Field	Type	Default	Description
task	enum	—	Recommended. Collection intent, auto-maps to an endpoint (see Tasks below). Leave empty only if you set endpoint explicitly.
endpoint	enum	auto	Advanced. auto = resolve from task; or set an explicit endpoint to override.
subreddit	string	—	Subreddit name without r/ (e.g. java). Needed by postDetail/postComments/postDuplicates and all subreddit* endpoints.
username	string	—	Reddit username without u/ (e.g. elonmusk). Needed by all user* endpoints.
article	string	—	Post ID from the comments path (e.g. 1b7abcd). Needed by postDetail/postComments/postDuplicates.
q	string	—	Search keyword. Needed by search / searchSubreddits / searchUsers.
sort	string	—	Listing sort: hot/new/top/rising/best (popular) or confidence/top/new/controversial/old/random/qa/live (comments).
country	string	—	ISO country code (e.g. US) for popularCountry.
restrictSr	string	—	Set true to restrict search to a subreddit (with subreddit).
after	string	(empty)	Reddit pagination cursor (e.g. t3_1b7abcd) to start from.
limit	integer	25	Items per page, max 100.
t	enum	(empty)	Time filter for top endpoints: hour / day / week / month / year / all.
targets	array of object	(empty)	Batch mode: each element is a param set, collected one after another.
proxyUrl	string	(empty)	Proxy used by Dataclawd's server when fetching Reddit.
httpMethod	enum	GET	Dataclawd Reddit endpoints are GET-only; POST returns 501.
maxItems	integer	100	Stop after this many items in total (max 100 000).
maxPages	integer	10	Max pages per target (max 100).
startCursor	string	(empty)	Alias for after; resume from a previous cursor.
itemsPath	string	(empty)	Dot-path to the item list, e.g. data.children. Leave empty for auto-detection (Reddit Listing children are auto-unwrapped).
nextCursorPath	string	(empty)	Dot-path to the next-page cursor. Leave empty for auto-detection (data.after).
paginationParam	enum	auto	Query param used for paging: auto (after), cursor, or none.

Endpoints

Tasks

Pick a task and the Actor selects the endpoint for you. Fill the params below (subreddit / username / article / q …).

Task	Endpoint	Required params
popular 全局热门	popular	—
subredditHot Subreddit 热门	subredditHot	subreddit
subredditNew Subreddit 最新	subredditNew	subreddit
subredditTop Subreddit 最佳	subredditTop	subreddit
postDetail 帖子详情	postDetail	subreddit + article
postComments 帖子评论	postComments	subreddit + article
search 搜索	search	q
searchSubreddits 搜索社区	searchSubreddits	q
searchUsers 搜索用户	searchUsers	q
userProfile 用户资料	userProfile	username
userPosts 用户帖子	userPosts	username
userComments 用户评论	userComments	username

Posts — postDetail, postComments, postDuplicates Popular / homepage — popular, popularBest, popularCountry, popularRising, popularTop Search — search, searchSubreddits, searchUsers Subreddit — subredditHot, subredditNew, subredditTop, subredditAbout, subredditComments, subredditControversial, subredditPosts, subredditRules, subredditSimilar, subredditsNew, subredditsPopular User — userAbout, userActivity, userComments, userCommentsTop, userOverview, userPosts, userPostsHot, userPostsNew, userPostsTop, userProfile

Output (Dataset — one row per collected item)

Field	Description
platform	Always reddit.
endpoint	The endpoint that produced this row.
status	completed (data collected) or failed (request-level failure).
target	Human-readable target summary, e.g. subreddit=java.
page	Page number this item came from (1-based).
itemIndex	Sequential index within the run (1-based).
cursor	Cursor used to fetch that page (-1 = first page).
data	The collected item — the raw JSON returned by Dataclawd (a post or user object; Reddit Listing wrappers are unwrapped).
errorStage	Where a failure happened: connect_failed / server_error / auth_error / rate_limited / request_error / parse_error / empty_result.
errorCode	HTTP status code (if any), e.g. 429.
error	Error message on failed rows.

Run status & status message

While running, the Actor's status message is updated periodically (开始采集 … → 采集 … 中：已写入 n 条 → 完成：…). The run succeeds (SUCCEEDED) as soon as at least one item is collected. If every target fails or returns nothing, the run ends FAILED with an explanatory message. Billing happens only for completed rows.

How to verify the results

1. Run status: the run is SUCCEEDED when at least one item was written; the status message shows live progress.
2. Per-item state: open the run's Dataset — every collected item has status=completed with the raw JSON in data; failed rows explain why (errorStage / errorCode / error).
3. Spot-check against the API: re-run one target in the Dataclawd API Console and compare with the row's data.
4. Billing: only completed rows carry the item-collected pay-per-event charge.

Developer note: for automated checks (run status, row schema, optional Dataclawd API replay), use python3 tools/verify_output.py --run <RUN_ID> [--api].

Pricing

Pay-per-event (PPE): one item-collected event per successfully collected item. Failed rows do not carry this event.

Publishing note: define an item-collected event with your price in the Actor's Monetization tab. The platform's built-in synthetic event apify-default-dataset-item is charged automatically per default-dataset item when PPE is enabled; set its price to 0 if you don't want a per-item base fee.

Configuration (publisher)

The following environment variables are configured on the Actor's version settings and are not visible to end users:

• DATACLAWD_API_HOST — Dataclawd API base URL (e.g. https://api.dataclawd.example).
• DATACLAWD_API_KEY — Dataclawd API key (sent as X-API-Key).

Disclaimer

This Actor is not affiliated with, endorsed by, or sponsored by Reddit, Inc. Collect only data you are authorized to collect, and comply with Reddit's Terms of Service, the Dataclawd service terms, and applicable law.

---

中文简介

本 Actor 通过 Dataclawd API 批量采集 Reddit 数据（32 个端点）：subreddit 热门/最新/最佳/争议帖子、subreddit 资料/规则/相似社区、帖子详情与评论、全局热门帖子、帖子/社区/用户搜索，以及用户资料/动态/评论/帖子。选定采集端点、自动按 after 游标分页、自动提取条目列表（Reddit Listing 的 children 会自动解包为帖子对象），并把每条数据以 JSON 写入 Apify 默认 Dataset（成功行同时也是按事件计费的计费事件）。

• 输入：endpoint（采集端点，必填）、subreddit / username / article / q（端点对应主体，search* 用 q）、sort/country/restrictSr（排序/国家/限定搜索）、limit（每页条数，最大 100）、t（top 端点时间范围）、targets（批量目标列表，可选）、maxItems/maxPages（采集量控制）、after/startCursor（续采）、itemsPath/nextCursorPath（高级：显式指定提取路径与游标路径）、paginationParam（分页参数）。
• 输出：每行包含 platform、endpoint、status（completed/failed）、target、page、itemIndex、cursor 和采集到的原始数据 data；失败行带 errorStage（connect_failed/server_error/auth_error/rate_limited/request_error/parse_error/empty_result）、errorCode、error。运行中实时进度通过 Apify 状态消息展示，至少采到 1 条即运行成功（SUCCEEDED），全部失败/空结果则运行失败（FAILED）。
• 计费：按事件计费（PPE），每条成功采集计一个 item-collected 事件；失败行不计费（发布时需在 Console Monetization 中配置该事件及价格）。
• 验证数据：status=completed 即成功采集（数据在 data 字段）；开发时可运行 python3 tools/verify_output.py --run <RUN_ID> 校验。
• 免责声明：本 Actor 与 Reddit, Inc. 无任何附属关系。请仅采集你被授权采集的数据，并遵守 Reddit 服务条款、Dataclawd 服务条款与当地法律。