Substack Scraper: Posts, Full Text, Newsletters & Leaderboards

Export Substack newsletter data to JSON, CSV or Excel: every post of a publication with its date, likes, comments and restacks; the full text and HTML of free posts, ready for analysis or an AI/RAG pipeline; publication profiles with subscriber counts and subscription prices; and lists of publications from Substack's category bestseller leaderboards. It reads the same public JSON that Substack's own website loads, so there is no login, no browser and no API key.

One Actor, four jobs:

Mode	What you get
Posts of publications	Every post in a publication's archive, newest first: title, subtitle, link, date, type (newsletter, podcast, thread), free or paid, word count, likes, comments, restacks, authors, tags, cover image. Optionally the full content too.
Full content of specific posts	For each post link: everything above plus the post as plain text and as HTML, its outbound links and a word count. Free posts in full; paid posts as the public preview Substack shows to everyone.
Publication info	Name, author, description, subscriber count or label, paid-subscriber label, monthly, annual and founding prices, bestseller badge, contributors, sections, launch date, and whether the writer opted out of AI crawlers.
Find publications	Substack's category leaderboards (Bestsellers, Rising, Top) for Technology, Business, Finance, Culture, U.S. Politics and the other top-level categories.

Why use this one

• Custom domains just work. Give lennysnewsletter.com, lenny.substack.com, just lenny, a profile link such as substack.com/@lenny, or any post link. The same publication given twice in different spellings is scraped and billed once.
• The whole archive, not the first page. Substack's archive often returns a short first page; the Actor pages by post count until the archive really ends, and stops cleanly at your limit.
• Text a model can use. Post HTML is turned into clean paragraphs and bullet lists with footnotes kept as [1]. "Subscribe now" buttons, image toolbars, embedded tweets and other posts' teasers are left out. The original HTML is kept in contentHtml.
• Honest about paywalls. Every content row says contentAccess: full (free post), preview (the public preview of a paid post) or none. Paywalls are never bypassed.
• Respects AI opt-outs. Substack lets writers block AI crawlers. aiCrawlersBlocked on content and publication rows tells you which ones did, so your pipeline can leave them out.
• Monitoring built in. Turn on Only new posts since the last run, schedule the Actor, and each run returns only posts published since the previous one.
• Market data in one table. Publication rows carry subscriber labels as Substack displays them ("Over 1,200,000 subscribers", "Tens of thousands of paid subscribers") next to the actual prices, so you can size and price a niche in one run.

How to use it

1. Choose What to scrape.
2. Add Publications, Post URLs or Categories, depending on the mode.
3. Click Start, then download the results or pull them through the Apify API.

Example input: the last six months of two newsletters, with full text

{
  "mode": "posts",
  "publications": ["https://www.lennysnewsletter.com", "astralcodexten.substack.com"],
  "postedAfter": "2026-04-01",
  "includeContent": true,
  "maxPostsPerPublication": 500
}

Example input: a daily feed of new posts

{
  "mode": "posts",
  "publications": ["https://www.noahpinion.blog", "https://www.thefp.com"],
  "onlyNewPosts": true
}

Schedule it daily in Apify Console. The first run returns the newest posts up to your limit; later runs return only what was published since.

Example input: the bestsellers in two categories

{
  "mode": "searchPublications",
  "categories": ["technology", "finance"],
  "categoryRanking": "paid"
}

Example input: profiles of publications you already know

{
  "mode": "publicationDetails",
  "publications": ["lenny", "https://newsletter.semianalysis.com", "https://substack.com/@petergyang"]
}

Output

Each item has a type field: post or publication.

Post (Posts mode)

{
  "type": "post",
  "postId": 215694124,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "postedAt": "2026-09-29T13:15:57Z",
  "postType": "newsletter",
  "audience": "everyone",
  "isPaywalled": false,
  "wordCount": 1276,
  "reactionCount": 291,
  "commentCount": 7,
  "restackCount": 5,
  "authors": ["Lenny Rachitsky"],
  "authorHandles": ["lenny"],
  "tags": [],
  "coverImageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-0e02-4190-a590-1236719b99de_9213x6142.jpeg",
  "publicationId": 10845,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com"
}

Post with content (Full content mode, or Posts mode with Include full post content) adds:

{
  "title": "Open Thread 455",
  "audience": "everyone",
  "wordCount": 143,
  "updatedAt": "2026-10-13T00:21:04Z",
  "contentAccess": "full",
  "contentText": "This is the weekly visible open thread. Post about anything you want, ask random questions, whatever. ACX has an unofficial subreddit, Discord, and bulletin board, and in-person meetups around the world. ...\n\n1: Meetup updates: ...",
  "contentHtml": "<p>This is the weekly visible open thread. ...</p>",
  "contentWordCount": 141,
  "links": ["https://www.reddit.com/r/slatestarcodex/", "https://discord.gg/RTKtdut"]
}

wordCount is Substack's count for the whole post; contentWordCount counts the text you received, so on a paid post's preview the two differ.

Publication

{
  "type": "publication",
  "publicationId": 10845,
  "name": "Lenny's Newsletter",
  "url": "https://www.lennysnewsletter.com",
  "subdomain": "lenny",
  "customDomain": "www.lennysnewsletter.com",
  "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.",
  "authorName": "Lenny Rachitsky",
  "authorProfileUrl": "https://substack.com/@lenny",
  "subscriberCount": 1200000,
  "subscriberCountApprox": "1.2M+",
  "subscriberCountLabel": "Over 1,200,000 subscribers",
  "paidSubscriberLabel": "Tens of thousands of paid subscribers",
  "bestsellerTier": 10000,
  "monthlyPrice": 20.0,
  "annualPrice": 200.0,
  "foundingPrice": 400.0,
  "priceCurrency": "USD",
  "hasPodcast": true,
  "aiCrawlersBlocked": false,
  "sections": ["Community wisdom", "How I AI"],
  "firstPostAt": "2019-06-14T15:00:48Z"
}

Leaderboard rows also carry rank, category, categoryRanking and listTitle (for example "Bestsellers in Technology").

Pricing

Pay only for results:

Result	Price
Post (archive row)	$2.00 per 1,000
Post with full content	$5.00 per 1,000
Publication (info or leaderboard row)	$5.00 per 1,000

The default input (50 posts from one newsletter) costs about $0.10. Set a maximum spend per run in the run options and the Actor stops cleanly when it is reached.

Paywalls, previews and AI opt-outs

The Actor sees what any logged-out reader sees. Free posts come back complete. For a paid post Substack serves the preview the writer chose, which can be a few paragraphs or nothing at all (common for paid discussion threads); those rows have contentAccess set to preview or none and isPaywalled set to true. There is no option to log in or unlock paid posts.

Writers can tell Substack to block AI crawlers. When they have, aiCrawlersBlocked is true. If you are building a training set or a retrieval index, filter on it.

Limits to know about

These come from Substack, not from this Actor:

• Posts in hidden sections are not listed. Writers can hide a section's posts from the publication's archive (podcast summaries, community digests, for example). The Actor lists what the archive lists.
• Subscriber numbers are what the writer shows. Many publications hide exact counts; subscriberCount is then empty and subscriberCountLabel or subscriberCountApprox carries Substack's rounded wording when there is one. A young publication's ranking line is a launch date, which is not exported as a subscriber label.
• Leaderboards are short. When checked, the Technology Bestsellers list stopped at 100 publications, Rising at 75 and Top at 525. Leaderboards exist only for the top-level categories, not for subcategories such as "Artificial Intelligence".
• No keyword search. Substack's publication search answers requests from cloud servers, Apify's included, with an empty list, so publications are found through the category leaderboards instead.
• Popular-first order cannot stop early on a date filter, because dates are not in order; the Actor reads pages until your limit or the end of the archive.
• Rate limits. If Substack answers a burst with HTTP 429, the Actor waits and retries. Keep Max parallel requests at 3 or so for large runs.
• Only new posts. The Actor remembers the newest post it saved per publication. If a run stops early (max items or spending limit), older new posts it did not reach are not returned by later runs.

FAQ

Do I need a Substack account? No. Everything here is public.

Does it work with custom domains? Yes. Use the domain the newsletter lives on, its name.substack.com address, or a profile link.

Can I get the comments? Not yet. Each post row carries the comment count.

Can I scrape a paid newsletter I subscribe to? No. The Actor never logs in, so paid posts come back as their public preview.

How do I keep a dataset of a newsletter up to date? Run Posts mode once with a high limit and Include full post content, then schedule the same input with Only new posts since the last run turned on.

Feedback

Found a bug or need a field that is missing? Open an issue on the Actor's Issues tab.