Substack Scraper — Free Posts & Publications
Pricing
$1.80 / 1,000 free post scrapeds
Substack Scraper — Free Posts & Publications
Extract free Substack posts from publication URLs, custom domains or publication search. Get clean text, HTML, bylines, dates and engagement. No paywall bypass.
Pricing
$1.80 / 1,000 free post scrapeds
Rating
0.0
(0)
Developer
abhinav subedi
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 hours ago
Last modified
Categories
Features
Share
Extract publicly available free Substack posts from newsletter subdomains and custom domains. Each row contains publication identity, post metadata, author bylines, engagement and optional full HTML/plain text. Export from Apify as JSON, CSV, Excel or through the dataset API.
Quick start
{"publicationUrls": ["astralcodexten.substack.com", "https://www.lennysnewsletter.com", "noahpinion.substack.com"],"maxPostsPerPublication": 10,"publishedAfter": "2026-01-01","includeBody": true}
Provide at least one publication URL or subdomain. Custom domains must actually host a Substack publication; not every newsletter does.
Inputs
| Field | Default | Meaning |
|---|---|---|
publicationUrls | empty | Array of bare Substack subdomains, HTTPS publication URLs or custom domains |
maxPostsPerPublication | 10 | Maximum accepted free posts per publication (1–1000) |
publishedAfter, publishedBefore | none | Inclusive ISO dates/timestamps; date alone means midnight UTC |
includeBody | true | Include bodyText and bodyHtml; false omits both fields |
requestDelayMillis | 1500 | Pause before each request, at least 1000ms |
maxArchivePages | 100 | Safety cap per publication, up to 1000 pages of 12 |
Sample output
{"publicationName": "Astral Codex Ten","publicationUrl": "https://www.astralcodexten.com","publicationSubdomain": "astralcodexten","title": "The AI Forecasting Agenda","subtitle": "...","url": "https://www.astralcodexten.com/p/the-ai-forecasting-agenda","slug": "the-ai-forecasting-agenda","publishedAt": "2026-11-19T20:29:45.205Z","audience": "everyone","bylines": [{"name":"Scott Alexander","handle":"astralcodexten","profileUrl":"https://substack.com/@astralcodexten"}],"reactions": {"❤":141},"likeCount": 141,"commentCount": 139,"wordCount": 3638,"coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/4125aad4-afa1-465e-ac49-4f4926beff76_1536x1024.png","bodyText": "Full freely accessible article text…","bodyHtml": "<p>Full freely accessible article HTML…</p>"}
Counts change over time. Missing publication metadata falls back to the input host; the subdomain is null for unidentified custom domains. likeCount uses Substack's reaction count, with the heart count as fallback. Byline objects contain only name, handle and profile URL: no author email, biography, subscriber profiles or comments are collected. Body content remains publisher-authored public content and may itself mention people.
Pricing
Target price: $0.0018 per result ($1.80 per 1,000 posts), one result charge event per saved post, with or without bodies. No charge events for skipped paid posts, errors or empty results. This is below the popular competitor's $0.002–$0.0023 full-content price checked at launch. Apify's live pricing panel is authoritative; monetization requires the owner's payout/billing setup and may remain unavailable until completed. Platform usage charges may apply as shown by Apify.
Limits, reliability and responsible use
- No credentials, subscriber cookies, paywall bypass, or paid previews. Only
audience: everyonedetail responses with a nonempty body and no truncation/paywall flags are accepted. This check runs even when bodies are omitted. Archive excerpts (truncated_body_text) are never used as full content. - One request at a time. Requests pause between calls; HTTP 429 and server errors retry up to three times with exponential backoff and respect
Retry-After. Access-denied responses are not bypassed. - Archives are requested in pages of 12 with
sort=new, an emptysearchparameter and an offset. If the public archive rejects an offset with HTTP 400, the scraper continues through publicprevious_post_sluglinks. Both paths obey the same free-content checks, rate limit and scan bound. Substack's undocumented endpoints can change; failures yield partial results with explicit errors. A bounded scan can return fewer posts than requested. Repeated pages and links are detected. - Dataset URLs are deduplicated within a run. No stored history across runs. Metadata-only or empty-body posts are intentionally skipped.
- Inspect the
SUMMARYkey-value record for result/skipped counts and per-publication errors. A run with errors and no results fails; a legitimately empty archive succeeds with zero results. - Public access does not grant republication rights. Respect publisher terms, copyright, privacy and applicable law. Use for permitted research/analysis; do not redistribute full articles without permission. Render untrusted HTML safely.
Development
Node.js 22+. npm ci, npm test, then apify run --purge --input-file example-input.json. Deploy only this directory using apify push. The folder-local publish.mjs configures only this Actor, never other Actors in the account.