Reddit Historical Archive Scraper

Export old Reddit posts and comments by subreddit, author, keyword, score, and date range.

The Reddit Historical Archive Scraper turns a bounded historical window into clean datasets for research, social listening, OSINT, moderation analysis, NLP, and data backfills.

It reads the public Arctic Shift archive directly. No Reddit login, OAuth app, API key, cookie, or proxy is required.

• 🗓️ Search exact UTC date windows instead of only recent Reddit listings.
• 🧵 Export both historical posts and historical comments.
• 🎯 Narrow by subreddit, author, keyword, and minimum score.
• 📦 Keep posts and comments in separate run-scoped datasets.
• 🔁 Schedule repeatable archive exports or call them through API and MCP.
• 💾 Download JSON, CSV, Excel, XML, HTML, or JSONL from Apify.

Run Reddit Historical Archive Scraper

What does Reddit Historical Archive Scraper do?

Reddit's normal listings emphasize recent content and expose limited history. This Actor queries a historical archive so you can backfill older conversations within an explicit date window.

For every matching post, it can return:

• title and selftext;
• author and subreddit;
• UTC and ISO timestamps;
• score, upvote ratio, and comment count;
• flair, domain, permalink, and outbound URL;
• NSFW, self-post, video, locked, sticky, and spoiler flags.

For every matching comment, it can return:

• full comment body;
• author and subreddit;
• UTC and ISO timestamps;
• score and controversiality;
• post ID and parent ID;
• permalink, submitter status, and award count.

Every row includes the archive request URL and the requested filters for reproducibility.

Who is this Reddit archive extractor for?

Researchers and academics

Create bounded historical samples for discourse, community, event, and longitudinal studies. Preserve the exact requested dates and provenance with each row.

Social-listening teams

Backfill brand or topic mentions in selected communities. Schedule fixed windows to maintain an internal historical corpus.

OSINT analysts

Trace public discussions around known authors, communities, events, or phrases without relying on current Reddit search ranking.

ML, NLP, and RAG builders

Export flat text records for sentiment classification, topic modeling, retrieval, evaluation, or training-data preparation.

Moderators and community teams

Review old high-scoring posts, archived comments, recurring themes, or author activity in a defined period.

Why use this historical Reddit scraper?

• ✅ Historical depth — query date windows that ordinary listings may not expose.
• ✅ No credentials — no Reddit developer application or personal account is needed.
• ✅ Bounded jobs — every run requires an after date, before date, and subreddit or author.
• ✅ Analysis-ready records — numbers remain numbers, booleans remain booleans, and timestamps include ISO values.
• ✅ Two datasets — posts and comments stay separate instead of creating a sparse mixed table.
• ✅ Resilient requests — transient failures use bounded backoff and rate-limit headers.
• ✅ Deduplication — duplicate archive IDs are removed within a run.
• ✅ Apify workflows — use schedules, webhooks, integrations, API clients, and MCP.

What Reddit data can you extract?

Data group	Example fields	Why it helps
Identity	recordType, redditId, author, subreddit	Join, group, and deduplicate records
Time	createdUtc, createdAt	Build timelines and time-series analyses
Engagement	score, numComments, upvoteRatio	Rank and filter archived discussions
Post text	title, selftext, linkFlairText	Topic, intent, and sentiment analysis
Comment text	body, linkId, parentId	Conversation and reply analysis
URLs	permalink, url, sourceUrl	Trace records back to Reddit and the archive query
Flags	over18, locked, stickied, spoiler	Segment moderation and content states
Provenance	requestedAfter, requestedBefore, filter fields	Reproduce an export later

Optional source fields are omitted when the archive has no value. Required identity and timestamp fields are always present for saved rows.

How to scrape old Reddit posts and comments

1. Open the Actor input page.
2. Choose Posts, Comments, or both under Record types.
3. Enter at least one subreddit or author.
4. Set the After date and Before date in YYYY-MM-DD format.
5. Optionally enter a keyword and minimum score.
6. Choose oldest-first or newest-first ordering.
7. Set a safe maximum number of archive records.
8. Click Start.
9. Open Posts or Comments in the Output and Storage views.
10. Export the dataset or connect it to your next workflow.

A one-day window and 20-record cap are prefilled for a fast first run.

Input parameters

Input	Type	Required	Description
recordTypes	array	No	posts, comments, or both; default is both
subreddits	string array	Conditional	Names without r/; at least a subreddit or author is required
authors	string array	Conditional	Usernames without u/; combines with subreddit when both are set
keyword	string	No	Matches post title/selftext or comment body within the selected scope
after	date string	Yes	Inclusive UTC lower bound in YYYY-MM-DD
before	date string	Yes	Exclusive UTC upper bound in YYYY-MM-DD
minScore	integer	No	Save only records with this score or higher
sort	string	No	asc for oldest first or desc for newest first
maxItems	integer	No	Global output limit across posts and comments
batchSize	integer	No	Archive records requested per page, from 1 to 100
maxPages	integer	No	Global request safety limit; default 100 pages

Keyword and score filters are applied to fetched archive pages before records are saved. Sparse filters may inspect more records than they return, so maxPages prevents unbounded scans.

When both subreddit and author lists are supplied, the Actor evaluates each subreddit-author combination. Keep arrays focused to avoid a large cross-product.

Example inputs

Export posts and comments from one subreddit

{
  "recordTypes": ["posts", "comments"],
  "subreddits": ["python"],
  "after": "2024-01-01",
  "before": "2024-01-02",
  "maxItems": 100
}

Export one author's old posts

{
  "recordTypes": ["posts"],
  "authors": ["spez"],
  "after": "2023-01-01",
  "before": "2024-01-01",
  "sort": "asc",
  "maxItems": 200
}

Find keyword-matching historical comments

{
  "recordTypes": ["comments"],
  "subreddits": ["technology"],
  "keyword": "privacy",
  "after": "2024-01-01",
  "before": "2024-02-01",
  "minScore": 1,
  "maxItems": 1000
}

Output datasets

The Actor defines two run-scoped datasets. They belong to the current run and follow its retention settings.

• Posts uses the default dataset alias.
• Comments uses the comments dataset alias.

This design prevents post-only and comment-only columns from being mixed into one sparse table. The Run page's main Export button exports Posts; use the Storage dataset picker for Comments.

Example post output

{
  "recordType": "post",
  "redditId": "18vks3q",
  "author": "friendlyfitnessguy",
  "subreddit": "Python",
  "createdUtc": 1704068225,
  "createdAt": "2024-01-01T00:17:05.000Z",
  "score": 0,
  "permalink": "https://www.reddit.com/r/Python/comments/18vks3q/",
  "title": "code competition topics",
  "numComments": 0,
  "over18": false,
  "isSelf": true,
  "isVideo": false,
  "locked": false,
  "stickied": false,
  "spoiler": false,
  "totalAwardsReceived": 0,
  "requestedAfter": "2024-01-01",
  "requestedBefore": "2024-01-02"
}

Example comment output

{
  "recordType": "comment",
  "redditId": "kfrp2h4",
  "author": "startup_biz_36",
  "subreddit": "Python",
  "createdUtc": 1704067267,
  "createdAt": "2024-01-01T00:01:07.000Z",
  "score": 1,
  "body": "yeah when dealing with big datasets...",
  "linkId": "18vgrc9",
  "parentId": "t3_18vgrc9",
  "controversiality": 0,
  "isSubmitter": false,
  "locked": false,
  "stickied": false,
  "totalAwardsReceived": 0
}

How much does it cost to scrape historical Reddit data?

The Actor uses pay-per-event pricing:

• a small one-time start charge covers input validation and run startup;
• each saved historical post or comment is one result event;
• Apify subscription tiers receive volume discounts shown on the Pricing tab.

You are not charged a result event for duplicate IDs, filtered-out rows, or empty archive pages. Set maxItems to control the maximum number of billable records before starting.

The Apify Free plan includes platform credits, so small test exports may fit within your available credits. Check the live Pricing tab for the exact price applicable to your account tier.

Pagination, ordering, and deduplication

Arctic Shift returns bounded pages ordered by creation timestamp. The Actor advances a timestamp cursor until it reaches the requested date boundary, maxItems, or the maxPages safety limit.

• asc starts at the After date and moves forward.
• desc starts at the Before date and moves backward.
• IDs are deduplicated separately for posts and comments.
• maxItems applies across every selected type and scope.
• maxPages bounds source requests when keyword or score filters are sparse.
• A safe request-time budget stops very broad or slow archive scans before the Apify run timeout, preserving all records already saved.
• Keyword matching is case-insensitive against post title/selftext or comment body.
• Transient HTTP failures are retried a limited number of times.

An upstream outage fails clearly instead of silently falling back to a login-dependent Reddit surface.

Tips for reliable archive exports

• 🎯 Start with one subreddit, a short date window, and a low maxItems.
• 🗓️ Split multi-year research into monthly or yearly runs for simpler checkpoints.
• 🔍 Combine a keyword with a subreddit or author; the archive does not allow unbounded global full-text searches.
• 📊 Use minScore to reduce low-engagement records after validating the source coverage.
• ↕️ Use oldest-first ordering for incremental backfills.
• 💾 Save the requested dates with your downstream job state.
• 🔁 Schedule non-overlapping windows to avoid downloading the same archive period repeatedly.
• 🧪 Review both dataset schemas before importing them into a strict database.

Integrations and automation workflows

Google Sheets and data warehouses

Export CSV for a one-off analysis, or use the Dataset API to load rows into BigQuery, Snowflake, PostgreSQL, or a lakehouse.

Make, Zapier, and n8n

Start the Actor on a schedule, wait for completion, then route post and comment dataset items into dashboards, alerts, or storage.

Webhooks

Attach a run-succeeded webhook to trigger sentiment analysis, embedding generation, moderation review, or an ETL job.

Research pipelines

Run one date partition per task, preserve the input JSON and sourceUrl, and merge results by recordType + redditId.

Social-listening backfills

Use a keyword plus selected brand-relevant communities, then enrich the returned text with your taxonomy or classifier.

Run with the Apify API using cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~reddit-historical-archive-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "recordTypes": ["posts", "comments"],
    "subreddits": ["python"],
    "after": "2024-01-01",
    "before": "2024-01-02",
    "maxItems": 100
  }'

Keep tokens in environment variables or a secret manager. Do not commit them to source control.

Run with the Apify JavaScript client

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/reddit-historical-archive-scraper').call({
  recordTypes: ['posts', 'comments'],
  subreddits: ['python'],
  after: '2024-01-01',
  before: '2024-01-02',
  maxItems: 100,
});

const posts = await client.dataset(run.defaultDatasetId).listItems();
console.log(posts.items);

Use the run's storage mapping to access the Comments dataset when comments are enabled.

Run with the Apify Python client

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/reddit-historical-archive-scraper").call(run_input={
    "recordTypes": ["posts"],
    "authors": ["spez"],
    "after": "2023-01-01",
    "before": "2024-01-01",
    "maxItems": 200,
})

posts = client.dataset(run["defaultDatasetId"]).list_items().items
print(posts)

Poll asynchronous runs when your application cannot wait for a long historical export.

Use Reddit Historical Archive Scraper with MCP

Apify's MCP server lets Claude Code, Claude Desktop, Cursor, VS Code, and other MCP clients call the Actor as a tool.

Claude Code

$claude mcp add --transport http apify "https://mcp.apify.com?tools=automation-lab/reddit-historical-archive-scraper"

Claude Desktop, Cursor, or VS Code

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/reddit-historical-archive-scraper"
    }
  }
}

Example prompts:

• “Export the top historical Python subreddit posts from January 2024.”
• “Find archived privacy comments in r/technology and summarize recurring concerns.”
• “Build a JSON dataset of one author's public Reddit posts during 2023.”
• “Compare historical comment themes across two bounded monthly runs.”

Scheduling historical Reddit backfills

Use an Apify Schedule for repeatable partitions. For example, run a closed monthly window after the archive has had time to ingest it.

A practical incremental pattern is:

1. Store the end of the last successful window.
2. Use that date as the next run's after value.
3. Set a later non-overlapping before date.
4. Keep oldest-first ordering.
5. Merge downstream by recordType + redditId.
6. Alert if a run fails because the community archive is unavailable.

Do not use a moving unbounded window; explicit dates make reruns auditable.

Data source, freshness, and limitations

This Actor depends on the community-maintained Arctic Shift archive. It is not operated by Reddit or Apify and has no guaranteed uptime or completeness SLA.

Archive coverage can differ by year, subreddit, author, deletion state, or ingestion lag. Scores and comment counts reflect the archived record and may not match today's live Reddit values. Deleted or removed content may be absent, partially preserved, or marked with source placeholders.

The Actor does not:

• log in to Reddit;
• bypass private communities or access controls;
• recover content missing from the archive;
• promise real-time Reddit monitoring;
• enrich records from a separate live Reddit API;
• reconstruct nested comment trees beyond parent and link identifiers.

Use a live Reddit scraper when freshness matters more than deep historical coverage.

Is scraping historical Reddit data legal?

This Actor accesses a public archive endpoint and returns public-facing archive records. Whether a particular use is lawful depends on your jurisdiction, purpose, retention, and handling of personal data.

You are responsible for:

• following applicable laws and contractual obligations;
• respecting privacy, deletion, and data-subject requests;
• applying appropriate minimization and retention controls;
• avoiding harassment, profiling, doxxing, or harmful surveillance;
• verifying that your downstream use has a valid legal basis.

Public availability does not remove ethical or compliance responsibilities. Consult qualified counsel for regulated or high-risk projects.

FAQ

Can I scrape Reddit history without an API key?

Yes. This Actor reads an anonymous public archive and does not require Reddit OAuth credentials.

How far back can it search?

As far as Arctic Shift has records for the chosen subreddit, author, and date window. Coverage is not guaranteed for every period.

Can I search all Reddit globally by keyword?

No. To protect the archive from unbounded work, this Actor requires at least one subreddit or author. Keyword searches stay inside that scope.

Why did my successful run return no records?

The selected archive window and filters may have no matching records. Try a known active subreddit, shorten or adjust the dates, remove minScore, and verify spelling without r/ or u/ prefixes.

Why did the run fail with an archive error?

Arctic Shift may be temporarily unavailable or rate-limited. The Actor already performs bounded retries; rerun later rather than switching to a login-dependent source.

Are comments included automatically?

Both Posts and Comments are preselected. You can disable either record type to focus the run and reduce output.

Where are comments in the output?

Open the run's Storage tab and choose the Comments dataset. Posts remain in the default dataset used by the Run page Export button.

Are result rows deduplicated?

Yes, within each run by recordType + redditId. Deduplicate across separate runs downstream if their date windows overlap.

Can it return a full threaded conversation?

It returns linkId and parentId, which let you reconstruct relationships when all required comments are present. It does not guarantee every branch exists in the archive.

Can I schedule it?

Yes. Use Apify Schedules with fixed non-overlapping date windows for periodic historical exports.

Related Reddit scrapers

• Reddit Scraper — scrape live Reddit posts, comments, searches, and current community data.
• Reddit Historical Archive Scraper — use this Actor for bounded historical archive exports.

Choose the live Reddit scraper for current activity and this archive scraper for old date-range backfills.

Support

If a run behaves unexpectedly, include:

• the run URL;
• the exact input without secrets;
• the expected subreddit, author, and date window;
• whether Posts, Comments, or both were selected;
• a sample missing or malformed record ID.

This makes it easier to distinguish an Actor issue from an archive coverage or availability issue.