Reddit Historical Archive Scraper - Old Posts by Date
Pricing
from $1.05 / 1,000 results
Pushshift alternative to scrape old Reddit posts and comments without an API key. Full-text comment search, user history, export to CSV/JSON.
Search works again. PullPush has closed, and the request layer was asking the wrong network for each host.
A search for a plain word like shoes failed on the first query with HTTP 429,
and the run's advice — switch to the residential proxy group — was wrong.
Measured from three separate networks (a home connection, Apify's datacenter
pool and Apify's residential pool), PullPush answers every free request with
"this website does not provide free scraping resources". It is not an IP block,
so no proxy group restores it. PullPush is now dropped from a run the moment it
says so, instead of being retried for six minutes.
shoes subreddit:malefashionadvice or
shoes author:someuser is read out of the query and sent straight to
Arctic-Shift, which is faster and deeper than going via the feed.maxRetries: 0. Drawing a clean address
is not the same as hammering a source that has refused us — and the source
that has, PullPush, is no longer retried at all.Measured after the change, from the platform: the reported failing input
(shoes, from 2026-09-01, 5 items, maxRetries: 0, default proxy) returns 5
rows in 15-24 seconds across four consecutive runs.
Investigated the seven columns reported as empty — the extraction is correct;
the shipped example was the problem. selftext, authorFlairText, media,
galleryImages, edited, distinguished and suggestedSort were all mapped
to the right source fields. They came back empty because the Actor's prefilled
example ran r/technology, a subreddit that shares nothing but external links.
Verified against the live archive: of 100 recent r/technology posts, 0% are self
posts, 0% carry flair, 0% carry media or galleries, and 0% are distinguished or
have a pinned sort. There is nothing there to extract.
Changed the prefilled example from ["technology"] to
["personalfinance", "wallstreetbets", "pics"] at 40 posts each, so a first run
actually exercises the schema: post bodies, flair, media, galleries, pinned
sorts and moderator posts all appear. Measured against the live archive these
subreddits return selftext on 55.8%, authorFlairText on 31.7%,
suggestedSort on 33.3%, galleryImages on 10.0% and media on 2.5% of a
120-row run — every one of them zero before.
edited and distinguished stay near zero on recent posts by nature: the
archives snapshot a post within minutes of publication, and moderator posts
are rare in ordinary traffic. Both fill properly once you query a historical
window, which is what this Actor is for — a run over r/announcements and
r/AmItheAsshole before 2024-01-01 returns edited on 26.2% and
distinguished on 48.8% of rows.
Documented, in the README and in each field's description, exactly which post
type each of these columns depends on, so an empty column is recognisable as
"this subreddit has no such posts" rather than a broken scrape.
Fixed silent data loss on any subreddit containing a gallery post. The
dataset schema declared galleryImages as a string while the Actor has always
emitted an array of image objects. Pushing a single Reddit gallery post
therefore failed schema validation and aborted that whole subreddit — every
remaining row for it was lost, and the run still reported success. Verified on
a live run before the fix: r/wallstreetbets and r/pics both died with "Schema
validation failed" and contributed 0 of their 40 rows. galleryImages is now
declared as an array and those targets complete. This also explains part of why
galleryImages looked empty in audits: the rows that would have carried it were
the exact rows being rejected.
isGallery was emitted as undefined for non-gallery posts, so the column
vanished from exports entirely instead of reading false. It now defaults to
false.
No change to the parsers, the backends or the set of output columns. Nothing was removed: every one of the seven columns is real Reddit data that fills for the right subreddit.
selftext, authorFlairText, media, galleryImages, edited, distinguished and others); these are under review.body, parent/link IDs, vote fields, and reconstructed depth.proxyConfiguration default {useApifyProxy:true}). Every request mints a fresh rotating proxy session (new IP) per attempt, with retry/backoff and IP rotation on 403/429/5xx and Cloudflare 520-524 (522). RESIDENTIAL is documented as the escalation if AUTO/datacenter IPs stay blocked.Actor.fail) with a clear "enable Apify Proxy / switch to RESIDENTIAL" message instead of masquerading as an empty success. A valid-but-empty query (reachable backend, no matching data) still exits cleanly.{} now returns a sensible default (recent posts from a popular subreddit) instead of erroring — provided subreddits/posts/users/searches behave exactly as before.