Reddit Posts Scraper

Reddit Posts Scraper — Extract Posts, Comments and Search Results

Reddit Posts Scraper pulls posts and nested comment threads from any public subreddit, Reddit URL, or site-wide keyword search, and returns clean, typed JSON — no Reddit account, no OAuth app, no browser. Every run returns the same structured fields — post title, author, score, real post date, permalink, flair, NSFW/spoiler flags, outbound link, and recursively nested comments — ready to load into a spreadsheet, a database, or an LLM context window. Point it at a subreddit name, a full Reddit URL, or a search phrase, mix all three in one run, and results stream to the dataset as each post finishes.

---

What is Reddit Posts Scraper?

Reddit Posts Scraper is an Apify Actor that scrapes public Reddit listing, search, and comment pages and returns them as a normalized dataset of posts with nested comment threads. It reads the old.reddit.com HTML surface rather than Reddit's official API, so it needs no Reddit account, no app registration, and no OAuth token — every field it returns is already visible to an anonymous visitor.

It runs three input modes in a single job: named subreddits, full Reddit URLs, and free-text keyword searches. Each source is sorted (Hot, New, Top, or Rising), optionally time-windowed, and paginated automatically past Reddit's 100-post-per-page listing limit up to the number of posts you ask for.

Key capabilities:

• Three source types in one run — subreddit names (news, r/news), full Reddit URLs (subreddit pages, search pages), and free-text keyword searches, mixed freely in the same input list
• Nested comment threads — recursive replies fetched per post, not a flat list, up to a configurable comment count
• Sort and time control — hot, new, top, rising, with an optional time window for top/rising
• Automatic proxy fallback — direct → datacenter → residential, with fresh-IP retries when Reddit blocks a request
• Real-time dataset writes — each post is pushed as soon as it and its comments are scraped, not batched at the end
• Transparent, uncharged failures — a post that fails to scrape is still written to the dataset as a diagnostic row, and that row is never billed
• User post history, undocumented in the input label but present in the code — a u/username, user/username, or full /user/<name>/ URL in startUrls pulls that user's submitted posts instead of a subreddit or search, even though the input field is titled "Reddit URLs / Subreddits / Keywords"

No Reddit account or login is required to run it — the Actor never authenticates with Reddit, and it only reads pages any anonymous visitor can already load.

---

What data can you get with Reddit Posts Scraper?

Every run returns one result type — Reddit posts — with comments nested inside each post row.

Result Type	Extracted Fields	Primary Use Case
Post	post_id, title, author, author_fullname, score, num_comments, created_utc, published_at, permalink, post_url, body, image_url, thumbnail_url, subreddit, subreddit_type, over_18, spoiler, domain, outbound_url, num_crossposts, link_flair_text, comments, success, error_message	Trend research, social listening, sentiment datasets, brand monitoring

Nested comment threads

Every post row carries its own comments array — each comment is { author, body, score, created_utc, replies[] }, and replies is the same shape recursively, so a full sub-thread arrives nested inside the post that started it rather than as a separate flat list you have to re-assemble with a parent_id. Comments are collected in Reddit's confidence ("Best") order and walked to a fixed recursion depth of 8 levels; threads deeper than that, and Reddit's "load more comments" stubs, are not expanded. Set maxComments to 0 to skip comment collection entirely and get post metadata only — useful when you only need the post-level signal and want the run to finish faster.

"comments": [
  {
    "author": "commenter1",
    "body": "This matches what I've seen too.",
    "score": 412,
    "created_utc": 1784417796,
    "replies": [
      {
        "author": "commenter2",
        "body": "Source?",
        "score": 58,
        "created_utc": 1784418010,
        "replies": []
      }
    ]
  }
]

---

Why not build a Reddit scraper yourself?

Reddit's .json endpoints (the ones a lightweight script would normally call) return a hard 403 or a Cloudflare wall from datacenter and cloud IPs across proxy tiers and TLS fingerprints — that wall is what makes a DIY script unreliable in practice, not a lack of publicly visible data. Reddit Posts Scraper works around it by parsing the old.reddit.com HTML listing, search, and comment pages instead, which carry the same data in rich data-* attributes, using Chrome TLS impersonation so requests fingerprint like a real browser rather than a default HTTP client.

Making that reliable at scale means three things a one-off script usually skips: a proxy ladder that escalates from direct to datacenter to residential on a block, with a fresh IP per retry; a block detector that treats a non-200 status, a too-short body, or an "access denied" page as a failure rather than parsing it into empty results; and a comment-tree walker that recurses through Reddit's nested sitetable markup instead of assuming a flat list. That's infrastructure to build, test, and keep working through layout changes — before you've scraped a single subreddit.

Reddit Posts Scraper handles all of it and writes typed JSON rows to a dataset you can query, export, or schedule — no proxy account to manage, no HTML parser to maintain.

Pagination past Reddit's 100-post-per-page listing cap is handled the same way: the Actor follows the after/count pagination tokens Reddit's own "next" link exposes, requesting another page until maxPosts is reached or Reddit stops returning a next-page token — you set a target, not a page count. Between individual post-comment fetches the Actor waits a short randomized interval (roughly 0.3–0.7 seconds), and roughly 0.8–1.4 seconds between listing pages, so a run self-throttles rather than firing requests as fast as the network allows.

---

How to scrape Reddit with Reddit Posts Scraper

1. Open Reddit Posts Scraper on the Apify Store and click Try for free (or Run, if you already have it saved).
2. Under Reddit URLs / Subreddits / Keywords, add your sources to startUrls — one per line. Mix subreddit names (technology), full URLs (https://www.reddit.com/r/technology/), and search phrases (artificial intelligence) freely.
3. Set Sort order (hot, new, top, rising) and, if you picked top or rising, a Time filter. Set Maximum posts per source and Maximum comments per post.
4. Click Start. Posts stream into the dataset live — you don't have to wait for the whole run to finish to start reading results.
5. Open the Output tab and export as JSON, CSV, or Excel, or pull results through the Apify API / apify-client.

{
  "startUrls": ["technology", "r/artificial", "reddit data scraping"],
  "maxPosts": 50,
  "maxComments": 10,
  "sortOrder": "top",
  "timeFilter": "week"
}

How to scrape multiple subreddits, keywords, and URLs in one run

startUrls is a list — add as many subreddits, URLs, and search phrases as you want, and the Actor works through them in order, applying the same sortOrder, timeFilter, maxPosts, and maxComments to every source. Duplicate subreddits (however they were entered — by name, r/ prefix, or URL) are merged so you don't pay to scrape the same community twice. There is no separate concurrency setting exposed in the input; each source in the list is scraped in turn within a single run.

Site-wide keyword search, no subreddit specified:

{
  "startUrls": ["remote work productivity"],
  "sortOrder": "new",
  "maxPosts": 100,
  "maxComments": 0
}

Fast metadata-only sweep across several communities (skip comments entirely with maxComments: 0 when you only need post-level signal):

{
  "startUrls": ["technology", "artificial", "MachineLearning", "programming"],
  "sortOrder": "rising",
  "maxPosts": 25,
  "maxComments": 0
}

One user's recent post history:

{
  "startUrls": ["u/spez"],
  "sortOrder": "new",
  "maxPosts": 20,
  "maxComments": 5
}

---

⬇️ Input

Reddit Posts Scraper takes one required field and five optional ones. Every parameter below is reproduced exactly from the Actor's input schema.

Parameter	Required	Type	Constraints	Description
startUrls	Yes	array (string list)	at least one entry	One item per line. Accepts full URLs (e.g. https://www.reddit.com/r/news/), subreddit names (e.g. news or r/news), or search keywords (e.g. artificial intelligence, which searches Reddit). Duplicate subreddits are merged.
maxPosts	No	integer	min 1, max 1000, default 10	Max number of posts to scrape per subreddit or keyword. If you have 3 sources and set 50, you can get up to 150 posts total.
maxComments	No	integer	min 0, max 1000, default 5	Max comments to fetch for each post. Set to 0 to skip comments and only get post metadata (faster).
sortOrder	No	string (enum)	hot, new, top, rising; default top	How Reddit should sort the posts. Hot = trending now, New = latest first, Top = most upvoted, Rising = gaining traction.
timeFilter	No	string (enum)	hour, day, week, month, year, all; default week	Time range for results. Only applies when Sort order is Top or Rising. Ignored for Hot and New.
proxyConfiguration	No	object (proxy editor)	default {"useApifyProxy": false}	Choose which proxies to use. If Reddit blocks a request, the Actor automatically falls back: no proxy → datacenter → residential. Recommended for large runs or when you hit blocks.

proxyConfiguration is a standard Apify proxy object, not a credential field — there is no API key or token to supply anywhere in this input; the Actor needs only your Apify account to run.

Sort order and time filter reference

The console shows friendly labels for sortOrder and timeFilter; the value actually sent in the input is the plain enum string in the left column.

sortOrder value	Console label	timeFilter value	Console label
hot	Hot	hour	Past hour
new	New	day	Past 24 hours
top (default)	Top	week (default)	Past week
rising	Rising	month	Past month
—	—	year	Past year
—	—	all	All time

timeFilter is read only when sortOrder is top or rising; for hot and new it's accepted as valid input but has no effect on the request the Actor sends.

Example input

{
  "startUrls": [
    "https://www.reddit.com/r/news/",
    "news",
    "artificial intelligence"
  ],
  "maxPosts": 50,
  "maxComments": 10,
  "sortOrder": "top",
  "timeFilter": "week",
  "proxyConfiguration": { "useApifyProxy": false }
}

Common pitfall: timeFilter is silently ignored unless sortOrder is top or rising — if you set sortOrder: "hot" with timeFilter: "month", the time filter has no effect, because Reddit itself doesn't accept a time window on Hot or New listings. A second pitfall: any entry in startUrls containing a space is treated as a keyword search, not a subreddit — "artificial intelligence" searches Reddit site-wide rather than looking for a subreddit of that name, while "news" (no space) is treated as a subreddit. If you want to search inside one specific subreddit, use a Reddit search URL for that subreddit rather than a bare keyword.

How startUrls entries are classified

Each line in startUrls is matched in this order, so it's worth knowing which bucket your entry lands in:

You enter	Treated as	Example
A URL starting with http whose path starts with /r/<name>/	Subreddit	https://www.reddit.com/r/technology/
A URL whose path starts with /user/<name>/ or /u/<name>/	User post history	https://www.reddit.com/user/spez/
Any other http URL	Scraped as-is, rewritten onto old.reddit.com	A search-results URL, a specific post URL
Text starting with r/	Subreddit (the r/ prefix is stripped)	r/news → subreddit news
Text starting with u/ or user/	User post history	u/spez → user spez
Plain alphanumeric/underscore text, 2–30 characters, no spaces	Subreddit	news, artificial
Anything else (contains a space, punctuation, etc.)	Keyword search	artificial intelligence, "exact phrase"

startUrls also accepts {"url": "..."} objects, not just plain strings, in addition to a single newline-separated string — useful if you're building the input programmatically through the Apify API rather than typing into the console field, since the array editor in the console only ever produces plain strings.

---

⬆️ Output

Results are written to the Actor's default dataset as typed JSON, one row per post, in real time as each post finishes scraping. Export as JSON, CSV, Excel, XML, or HTML from the Output tab, or read the dataset through the Apify API or apify-client.

Fields whose value is empty or unknown are omitted from the row entirely rather than written as null — so a successful post row can carry up to 24 keys, but a given row only has the keys that actually have a value. image_url and thumbnail_url are the exception: they default to an empty string "" rather than being dropped, so they're always present. A post that fails to scrape (blocked page, parse error) is written as a smaller, fixed diagnostic row — post_id, title, author, subreddit, permalink, comments: [], success: false, error_message — and that row is never billed: only successfully scraped posts count toward the charged row_result event. To keep only billed rows when processing the dataset yourself, filter on success == true.

Scraped results

[
  {
    "post_id": "1v0arhr",
    "title": "Study finds remote work productivity gains hold steady three years in",
    "author": "example_user",
    "created_utc": 1784417559,
    "num_comments": 6182,
    "score": 65176,
    "permalink": "/r/news/comments/1v0arhr/study_finds_remote_work_productivity_gains/",
    "image_url": "https://apnews.com/article/example",
    "thumbnail_url": "",
    "body": "",
    "comments": [
      {
        "author": "commenter1",
        "body": "This matches what I've seen too.",
        "score": 412,
        "created_utc": 1784417796,
        "replies": [
          {
            "author": "commenter2",
            "body": "Source?",
            "score": 58,
            "created_utc": 1784418010,
            "replies": []
          }
        ]
      }
    ],
    "subreddit": "news",
    "success": true,
    "published_at": "2026-07-18T23:32:39Z",
    "post_url": "https://www.reddit.com/r/news/comments/1v0arhr/study_finds_remote_work_productivity_gains/",
    "author_fullname": "t2_1wdmiwti2j",
    "subreddit_type": "public",
    "over_18": false,
    "spoiler": false,
    "domain": "apnews.com",
    "outbound_url": "https://apnews.com/article/example",
    "num_crossposts": 26,
    "link_flair_text": "Article"
  },
  {
    "post_id": "1v0b2xz",
    "title": "What's the best way to explain LLM context windows to a non-technical exec?",
    "author": "curious_pm",
    "created_utc": 1784412300,
    "num_comments": 214,
    "score": 892,
    "permalink": "/r/artificial/comments/1v0b2xz/whats_the_best_way_to_explain_llm_context/",
    "image_url": "",
    "thumbnail_url": "",
    "body": "Trying to write a one-paragraph explanation for our leadership deck...",
    "comments": [],
    "subreddit": "artificial",
    "success": true,
    "published_at": "2026-07-18T22:05:00Z",
    "post_url": "https://www.reddit.com/r/artificial/comments/1v0b2xz/whats_the_best_way_to_explain_llm_context/",
    "subreddit_type": "public",
    "over_18": false,
    "spoiler": false,
    "domain": "self.artificial",
    "num_crossposts": 0
  },
  {
    "post_id": "1v0c9pq",
    "title": "",
    "author": "unknown",
    "subreddit": "news",
    "permalink": "",
    "comments": [],
    "success": false,
    "error_message": "TimeoutError: Request timed out after 40 seconds"
  }
]

The second row shows a search-result post: rows sourced from Reddit's search-results page carry fewer fields than subreddit-listing rows — no author_fullname, link_flair_text, or outbound_url, because Reddit's search cards don't expose them — while a listing-sourced row (first example) carries the full set. The third row is an uncharged failure — success: false, a short error_message, and none of the optional fields, because the post was never successfully parsed.

Notes on data coverage

• upvote_ratio and award/gilding counts are not on the old.reddit.com HTML surface this Actor reads, and are intentionally left out of every row rather than faked with a placeholder value.
• link_flair_text is only present on posts that actually carry a flair, so its coverage varies by subreddit — some communities flair almost every post, others never use flair.
• body reflects a text (self) post's content; link and media posts carry an empty or omitted body, with the destination in outbound_url/domain instead.
• ⚠️ Comment collection stops at a fixed recursion depth of 8 replies, and Reddit's "load more comments" placeholders are not expanded — a very deep sub-thread is truncated rather than fully walked, and this depth is not configurable through the input.
• num_comments is Reddit's own reported total comment count for the post, while the returned comments array only holds what maxComments and the depth-8 limit actually pulled — the two numbers are expected to diverge on any post with more discussion than you asked for.

---

How can I use the data extracted with Reddit Posts Scraper?

• Market and trend researchers: pull top/week posts across a set of subreddits and read score, num_comments, and link_flair_text to see what's actually resonating in a community this week, not just what's pinned.
• AI engineers and LLM developers: feed title, body, and the nested comments array straight into an LLM as grounded context, or index them into a vector store for a Reddit-aware retrieval agent — the JSON is already typed and consistent across runs.
• Brand and social-listening teams: run a keyword search for a brand or product name, then use subreddit, score, and created_utc to spot where and how fast a mention is spreading, and over_18/domain to filter what's worth a human look.
• Data engineers: join on the stable post_id and author_fullname (Reddit's t2_... id, not a mutable display name) when building a warehouse table that needs to survive a username change.
• Content and SEO researchers: run sortOrder: "top" with a timeFilter of month or year across a niche subreddit to find durable, high-score questions and discussions worth building content around, rather than one-day spikes.

Each of these workflows reads the same dataset shape — there's no separate "analytics" output to configure, so the fields above are what every consumer downstream (a notebook, a dashboard, an agent) works from directly.

---

📊 How do you monitor a subreddit or keyword over time?

Reddit discussions move fast, and a single snapshot only tells you what a subreddit or search term looked like at one moment. The discipline here is simple: run the same startUrls on a schedule, and diff the result against the previous run's dataset on post_id.

Two signals matter most between runs: new post_ids that weren't in the previous dataset (new discussion volume), and a rising score or num_comments on a post_id you already have (a post gaining traction after your last check). Set sortOrder: "new" if you mainly care about catching posts as they appear, or sortOrder: "rising" if you want to catch momentum before a post peaks. Set maxComments low or 0 for a fast polling run that just tracks post-level signals, and raise it only on the runs where you need the discussion itself.

A typical workflow: schedule a run every few hours across a fixed list of subreddits or search terms, load each run's dataset, and alert when a post_id you're tracking crosses a score or num_comments threshold, or when error_message starts showing up more than usual (a signal Reddit is blocking that source harder than before). Reddit Posts Scraper doesn't push results anywhere on its own — schedule it with Apify Schedules in the Apify Console, and pull each run's dataset through the Apify API or apify-client to do the diffing on your side.

---

Integrate Reddit Posts Scraper and automate your workflow

Reddit Posts Scraper works with any language or tool that can send an HTTP request, since it runs as a standard Apify Actor with a documented API.

REST API with Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run = client.actor("<YOUR_USERNAME>/reddit-posts-scraper").call(run_input={
    "startUrls": ["technology", "artificial intelligence"],
    "maxPosts": 50,
    "maxComments": 10,
    "sortOrder": "top",
    "timeFilter": "week",
})

for post in client.dataset(run["defaultDatasetId"]).iterate_items():
    if post.get("success", True):
        print(post["subreddit"], post["title"], post["score"])

Works the same way in Node.js, Go, or Ruby — any client that can call the Apify API.

curl "https://api.apify.com/v2/acts/<YOUR_USERNAME>~reddit-posts-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -X POST \
  -H "Content-Type: application/json" \
  -d '{
        "startUrls": ["technology", "artificial intelligence"],
        "maxPosts": 20,
        "maxComments": 5,
        "sortOrder": "top",
        "timeFilter": "week"
      }'

run-sync-get-dataset-items runs the Actor and returns the dataset rows directly in the response — convenient for quick or scripted pulls. For a large maxPosts value that may run past a few minutes, start the run with the regular /runs endpoint instead and poll for completion, or fetch the dataset once you get a run-finished webhook.

Scheduled monitoring and delivery

Set up a recurring run from the Actor's Schedule tab in the Apify Console — hourly, daily, or any cron expression — so it re-scrapes the same sources without manual triggering. Combine a schedule with an Apify webhook on the run-succeeded event to trigger downstream processing (a script, another Actor, or a notification) as soon as each scheduled run finishes, or poll the dataset via the API on your own interval.

---

Is it legal to scrape Reddit posts?

Yes — scraping publicly accessible Reddit content is generally permitted, and Reddit Posts Scraper only returns what any visitor sees without logging in: post text, scores, comments, and the poster's public username. It doesn't access private, quarantined, or login-gated communities.

Because the output includes usernames and other content tied to identifiable posters, treat it as personal data for GDPR and CCPA purposes if your use case involves EU or California residents — collect only what you need, and avoid republishing or re-identifying users beyond what your legal basis supports. A pseudonymous Reddit username is still often enough to identify a real person when combined with post history, so the same care that applies to any user-generated-content dataset applies here. Using scraped Reddit data to train a model carries a different risk profile than using it for internal, time-bounded monitoring, so weigh the two differently. Consult your legal team for commercial use cases involving bulk data storage or redistribution.

---

❓ Frequently asked questions

Does Reddit Posts Scraper need a Reddit account, cookies, or API key?

No. It reads Reddit's public old.reddit.com pages directly and needs no Reddit login, cookie, OAuth app, or API key. The only credential involved is your own Apify account token, used to run the Actor and read the dataset back.

How do I scrape multiple subreddits, URLs, and keywords in one run?

Add each one as a separate line in startUrls — the field is a list, and every entry is processed against the same sortOrder, timeFilter, maxPosts, and maxComments settings. Duplicate subreddits are automatically merged so they aren't scraped twice.

How does Reddit Posts Scraper handle Reddit's blocking?

It escalates through a proxy ladder — direct, then datacenter, then residential — retrying with a fresh IP on each attempt when a request comes back blocked (HTTP 403/429/5xx, a suspiciously short page, or an "access denied" response). Requests are sent with Chrome TLS impersonation rather than a default HTTP client signature. If proxyConfiguration isn't enabled, the Actor still tries direct requests first before escalating.

How many posts and comments does Reddit Posts Scraper return per source?

Up to maxPosts (1–1000, default 10) posts per subreddit, URL, or keyword — pagination past Reddit's 100-post listing page happens automatically to reach your target. Up to maxComments (0–1000, default 5) top-level comments and their nested replies per post, walked to a fixed recursion depth of 8. These are input-schema limits you set, not a platform rate limit the Actor discovered on its own.

Why is my time filter being ignored?

timeFilter only applies when sortOrder is top or rising — Reddit's Hot and New listings don't accept a time window, so the Actor ignores timeFilter for those two sort orders rather than erroring.

What happens when a post fails to scrape?

It's still written to the dataset, as a smaller diagnostic row: post_id, title, author, subreddit, permalink, an empty comments array, success: false, and an error_message describing what went wrong. That row is not billed — only rows where scraping succeeded count toward the charged event. Filter on success == true to work with billed rows only.

How do I monitor a subreddit or keyword over time?

Schedule the same startUrls to run repeatedly with Apify Schedules, then diff each run's dataset against the previous one on post_id — new ids are new discussion, a rising score or num_comments on a known id is a post gaining traction. See How do you monitor a subreddit or keyword over time? above.

Does Reddit Posts Scraper work with Claude, ChatGPT, and AI agent frameworks?

Yes, as a standard HTTP endpoint through the Apify API — any agent framework that can make an HTTP call (LangChain, CrewAI, a custom tool definition, or a direct API call from Claude or ChatGPT function-calling) can trigger a run and read back typed JSON. This Actor does not currently document an MCP server integration.

How does Reddit Posts Scraper compare to other Reddit scrapers?

Checked on the Apify Store on 26 July 2026: harshmaur/reddit-scraper documents four data types (posts, comments, user profiles, communities) with 70+ post fields and an MCP integration; betterdevsscrape/reddit-scraper documents score/flair/domain/author filters and a date-windowing mode to pull past Reddit's ~1,000-post listing cap; parseforge/reddit-posts-scraper documents a free tier of ~35 post fields with comment threading, author profiles, and NSFW/spoiler flags gated behind a paid plan. Reddit Posts Scraper returns one result type — posts with nested comment threads included in every run at no extra gate — and always writes an uncharged diagnostic row for a post that fails, rather than silently dropping it.

Can I use Reddit Posts Scraper without managing proxies or platform credentials?

Yes. The default input runs with no proxy at all (useApifyProxy: false), and the Actor still escalates through datacenter and residential Apify Proxy tiers automatically if a request gets blocked, provided you enable proxyConfiguration. You never need a Reddit account, and if you do enable Apify Proxy, you don't need to create or rotate proxy sessions yourself — the Actor manages the ladder and IP rotation internally.

Can Reddit Posts Scraper pull a specific user's post history?

Yes, though it's not called out in the input field's title. Enter u/username, user/username, or a full https://www.reddit.com/user/<name>/ URL in startUrls, and the Actor pulls that user's submitted posts (sorted by the same sortOrder you set) instead of treating the entry as a subreddit or search term.

What formats does startUrls accept besides plain text lines?

The console input field is a simple one-line-per-entry list, but the underlying input also accepts an array of {"url": "..."} objects, or a single string with entries separated by newlines — useful when you're constructing the input from another script or Actor rather than typing into the Apify Console form.

---

💬 Your feedback

Found a bug or missing a field? Open an issue from the Actor's page in the Apify Console (Issues tab) with a sample source and what you expected to see — it helps get a fix out faster. If you're happy with the results, a review on the Actor's Apify Store listing helps other users find it.