Reddit Community Scraper

Reddit Community Scraper creates a structured dataset of posts collected from public Reddit communities (subreddits). Each dataset item represents one post and can include the title, self-text, author, score, upvote ratio, comment count, flair, timestamps, and media flags. Query the source by community URL or subreddit name, control sort order and time filter, limit results with maxItems, and retrieve records through the Apify Dataset API or export them as JSON, CSV, Excel, or XML.

Dataset at a glance

Property	Value
Source	reddit.com (public subreddits)
Record unit	One post
Input methods	Community URLs (startUrls) or subreddit names (searchQueries)
Main identifiers	id (post ID), url
Delivery	Apify Dataset and API
Export formats	JSON, CSV, Excel, XML
Update model	Fresh records per Actor run
Pricing	$2 per 1,000 posts

Coverage and available records

The Actor collects posts from one or more public Reddit communities.

• URL-based entry: Pass full community URLs in startUrls (for example https://www.reddit.com/r/AI_Agents/ or https://old.reddit.com/r/python/).
• Name-based entry: Pass subreddit names in searchQueries (for example AI_Agents, python); URLs are built automatically. Both inputs can be combined.
• Sort order: sortBy selects hot, new, top, rising, or controversial.
• Time filter: timeFilter applies to top and controversial and limits results to hour, day, week, month, year, or all.
• Result limit: maxItems caps the number of posts collected (0 = unlimited).

Known exclusions: private, quarantined, and age-restricted communities that require login are not accessible; each run captures community state at run time (no historical snapshots); the Actor does not fetch full comment threads (only the numComments count is returned).

Data dictionary

Field names below match dataset record JSON properties exactly.

Field	Type	Nullable	Description	Example
id	string	No	Reddit post identifier; best stable deduplication key	1vhilqp
title	string	No	Post title	Which AI agent platform is best?
sourceQuery	string	Yes	The community source that produced the record	r/AI_Agents
selftext	string	Yes	Post body text (empty for link posts)	We're looking at AI agent platforms...
author	string	Yes	Author username	elementary_constable
subreddit	string	Yes	Subreddit name without the r/ prefix	AI_Agents
subredditNamePrefixed	string	Yes	Subreddit name with the r/ prefix	r/AI_Agents
score	number	Yes	Post score	34
ups	number	Yes	Upvote count	34
upvoteRatio	number	Yes	Upvote ratio (0–1)	0.97
numComments	number	Yes	Number of comments	16
totalAwardsReceived	number	Yes	Total awards received	0
createdAt	string	Yes	Post creation time (ISO 8601)	2026-08-06T22:20:48+00:00
url	string	Yes	Post URL	https://www.reddit.com/r/AI_Agents/comments/1vhilqp/...
permalink	string	Yes	Post permalink path	/r/AI_Agents/comments/1vhilqp/...
fullUrl	string	Yes	Full post URL	https://www.reddit.com/r/AI_Agents/comments/1vhilqp/...
domain	string	Yes	Link domain (e.g. self.AI_Agents)	self.AI_Agents
linkFlairText	string	Yes	Post flair text	Resource Request
over18	boolean	Yes	Whether the post is marked NSFW	false
spoiler	boolean	Yes	Whether the post is marked a spoiler	false
stickied	boolean	Yes	Whether the post is stickied	false
isSelf	boolean	Yes	Whether the post is a text (self) post	true
isVideo	boolean	Yes	Whether the post contains video	false
isGallery	boolean	Yes	Whether the post is an image gallery	false
thumbnailUrl	string	Yes	Thumbnail URL (empty when absent)	""
postHint	string	Yes	Reddit post type hint	self
galleryUrls	array	Yes	Gallery image URLs	[]
gilded	number	Yes	Gilding count	0
distinguished	string	Yes	Distinguished author flag (mod/admin)	""
pinned	boolean	Yes	Whether the post is pinned	false
locked	boolean	Yes	Whether the post is locked	false
archived	boolean	Yes	Whether the post is archived	false

Example dataset record

A representative record produced from searchQueries: ["AI_Agents"]:

{
  "id": "1vhilqp",
  "title": "Which AI agent platform is best for enterprise voice support?",
  "sourceQuery": "r/AI_Agents",
  "selftext": "We're looking at AI agent platforms for enterprise voice support...",
  "author": "elementary_constable",
  "subreddit": "AI_Agents",
  "subredditNamePrefixed": "r/AI_Agents",
  "score": 34,
  "ups": 34,
  "upvoteRatio": 0.97,
  "numComments": 16,
  "totalAwardsReceived": 0,
  "createdAt": "2026-08-06T22:20:48+00:00",
  "url": "https://www.reddit.com/r/AI_Agents/comments/1vhilqp/...",
  "permalink": "/r/AI_Agents/comments/1vhilqp/...",
  "fullUrl": "https://www.reddit.com/r/AI_Agents/comments/1vhilqp/...",
  "domain": "self.AI_Agents",
  "linkFlairText": "Resource Request",
  "over18": false,
  "spoiler": false,
  "stickied": false,
  "isSelf": true,
  "isVideo": false,
  "isGallery": false,
  "thumbnailUrl": "",
  "postHint": "",
  "galleryUrls": [],
  "gilded": 0,
  "distinguished": "",
  "pinned": false,
  "locked": false,
  "archived": false
}

Query and input reference

Input	Type	Required	Default	Accepted values	Description
startUrls	array	No	—	Reddit community URLs	URL-based entry point
searchQueries	array	No	—	Subreddit names (AI_Agents, python)	Name-based entry point; URLs built automatically
sortBy	string	No	hot	hot, new, top, rising, controversial	Post sort order
timeFilter	string	No	all	hour, day, week, month, year, all	Time filter for top/controversial
maxItems	integer	No	0	0 or a positive integer	Maximum posts; 0 = unlimited
proxyConfiguration	object	No	Apify residential proxy	Apify proxy groups or custom proxies	Residential proxies are recommended

Minimal request:

{ "searchQueries": ["AI_Agents"], "sortBy": "hot" }

Advanced request:

{
  "searchQueries": ["AI_Agents", "python"],
  "sortBy": "top",
  "timeFilter": "month",
  "maxItems": 100,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

Retrieve the data through the API

1. Start the Actor with a JSON input (console or API).
2. Wait for the run to finish, or use a synchronous endpoint for an inline response.
3. Retrieve items from the run's default dataset.
4. Paginate or export the dataset.

Python example:

from apify_client import ApifyClient

client = ApifyClient("YOUR-APIFY-TOKEN")

run = client.actor("datascrapers/reddit-scraper").call(run_input={
    "searchQueries": ["AI_Agents", "python"],
    "sortBy": "hot",
    "maxItems": 100,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["id"], item["title"], item["score"])

Apify generates ready-to-run Python, JavaScript, and cURL examples on the Actor's API tab. Do not put a real API token in shared code or URLs.

Data quality and record handling

• Conditional fields: Link posts have empty selftext; text posts have isSelf: true. Fields Reddit does not expose for a post are returned as null or empty.
• Source changes: Reddit page structure and values can change; unreadable fields are returned as null rather than fabricated.
• Deduplication: Each run appends fresh records; the Actor does not deduplicate across runs. Use id as the stable key and filter repeated runs against previously stored IDs.
• Rate limits: Reddit blocks datacenter IPs. Residential proxies (enabled by default) are required for consistent coverage.
• Normalization: createdAt is normalized to ISO 8601; scores and counts are read as numbers.

Export and pipeline examples

Destination	Recommended method	Typical use
PostgreSQL / Supabase	Dataset API poll or webhook consumer	Store posts for trend and sentiment pipelines
Google Sheets	Apify Google Sheets integration	Share community snapshots with teams
S3 / cloud storage	Scheduled export via Apify scheduler	Archival of community snapshots
BI tools	CSV / JSON export	Content and engagement analysis

Pricing and cost examples

The Actor uses pay-per-event pricing, billed per post record:

Event	Trigger	Rate
Result	Every post pushed to the dataset	$2 per 1,000 posts

A one-time Actor start event of $0.00005 applies to each run (effectively $0).

Posts	Estimated base cost
1,000	$2.00
10,000	$20.00
100,000	$200.00

Compute units consumed by the run are billed by your Apify plan. Estimates depend on the verified pricing model and the options selected for the run.

Limitations and responsible data use

• The Actor collects publicly accessible post data from public Reddit communities only.
• Field availability depends on what Reddit renders at run time; some values can be null or missing, and site changes can alter fields.
• The Actor does not provide historical snapshots unless you store them yourself.
• Reddit requires residential proxies for reliable access; without them, coverage may degrade due to blocking.
• You are responsible for compliance with Reddit's terms of service, applicable privacy law, and any contractual obligations before using the data.

Dataset questions

What does one dataset item represent?

One Reddit post. Comment counts are included as numComments, but full comment threads are not collected.

Which field should I use as a unique identifier?

id is the stable Reddit post identifier and the recommended deduplication key. url is a reasonable secondary key.

Are fields nullable or conditional?

Yes. Link posts have empty selftext, and media fields (thumbnailUrl, postHint, galleryUrls) are empty when the post has no media.

Can I retrieve the records as CSV or JSON?

Yes. The dataset can be exported as JSON, CSV, Excel, or XML from the Apify Console, and queried through the Dataset API.

Does the Actor return historical data?

No. Each run captures the state of the community at run time. To track changes, schedule repeated runs and store the outputs yourself.

What counts as a billable result?

Each post pushed to the dataset is one billable result, charged at $2 per 1,000 posts. Use maxItems to control cost per run.

Related datasets from Data Scrapers

• Twitter X Profile Scraper — Public post data from X for cross-platform discussion monitoring.
• Instagram Post Scraper — Public Instagram posts for social content research.
• YouTube Comment Scraper — Video comment text for sentiment and audience research.
• LinkedIn Profile Scraper — Professional profile data for audience and community analysis.

Data Scrapers support

Need an additional field, record type, or export workflow? Contact Data Scrapers at stardustspotlight@gmail.com. Include a sample source URL, required fields, expected record volume, and preferred delivery format.