Substack Scraper

What does Substack Scraper do?

Substack Scraper collects posts from any Substack newsletter, whether it lives on name.substack.com or on its own custom domain. For every post you get the title, subtitle, authors, publish date, likes, comments, restacks, free or paid status, word count and cover image, and, if you want it, the full text of free posts. Paste newsletter links, single post links or author profiles, or search all of Substack by keyword.

Try it with the prefilled newsletter: open the Actor, click Start, and the 20 newest posts of Lenny's Newsletter arrive in a few seconds. Because it runs on the Apify platform, you also get API access, scheduled runs, integrations (Google Sheets, Make, Zapier, webhooks), run monitoring and automatic retries through Apify Proxy when Substack is slow to answer.

Why scrape Substack?

• Creator and market research: see which posts of a newsletter earn the most likes, comments and restacks, and how often it publishes.
• Competitor tracking: follow the newsletters in your niche and get every new post as structured data on a schedule.
• Content and trend analysis: search Substack for a topic and compare what dozens of writers publish about it.
• Sponsorship and partnership scouting: find active newsletters on a subject and gauge their engagement before reaching out.
• Reading lists, archives and AI datasets: export the full text of free posts for analysis, summaries or model training.

How to scrape Substack newsletters

1. Copy the address of a newsletter, such as https://example.substack.com or a custom domain like https://www.lennysnewsletter.com. You can leave out https://.
2. Paste it into Substack links, one link per line. A link to a single post (…/p/post-name), one section of a newsletter (…/s/section-name), a search of its archive (…/archive?search=pricing) or an author's profile (https://substack.com/@handle) works too.
3. Optionally, type words into Search Substack to find posts across all newsletters.
4. Set Max posts per newsletter or search, choose Newest first or Most popular first, and turn on Include full text of free posts if you need the text.
5. Click Start, then download the results when the run finishes.

Input

Everything is set on the Input tab. In JSON, the input looks like this:

{
    "start_urls": [
        "https://www.lennysnewsletter.com",
        "astralcodexten.substack.com/archive?sort=top",
        "https://www.noahpinion.blog/p/this-thing-will-fail",
        "https://www.lennysnewsletter.com/s/community-wisdom",
        "https://substack.com/@lenny"
    ],
    "search_queries": ["product management"],
    "max_posts_per_newsletter": 50,
    "sort": "new",
    "include_full_text": false
}

Field	What it does
start_urls	Newsletter homepages or /archive pages (on substack.com or a custom domain), one section of a newsletter (/s/section-name), a search of one newsletter's archive (/archive?search=your+words), post links (/p/post-name, quote share links /i/123/…, substack.com/home/post/p-123, substack.com/inbox/post/123, open.substack.com/pub/name/p/post-name), author profiles (substack.com/@handle) and Substack-wide search pages (substack.com/search/your%20words). Notes, tag (/t/…) and podcast (/podcast) pages are not supported yet and are listed in FAILED_INPUTS, as are a search inside a section (/s/…?search=…, which Substack answers with nothing; use /archive?search=…) and any other page of a newsletter, so a link is never widened to the whole newsletter. With or without https://; API callers can also send {"url": "…"} objects.
search_queries	Optional. Words or phrases; each runs Substack's post search across all newsletters, most relevant first. Up to 500 characters each; longer searches are listed in FAILED_INPUTS.
max_posts_per_newsletter	Stops after this many posts for each newsletter, section, profile or search. Default: 50. Single post links always give their one post.
sort	new (newest first, the default) or top (most popular first), also for section links. A link that already ends in ?sort=top or ?sort=new keeps its own order. Searches, including archive searches, are in Substack's best-match order.
include_full_text	Adds a text field with the plain text of each free post. Default: off. Paid posts never get text.
proxy_configuration	Optional; leave it empty. Substack answers the Actor directly, and the Actor switches to Apify's datacenter and then residential proxies by itself if it is ever refused.

Output

Each post becomes one row. Here is a real free post, saved with full text on (the text is shortened here):

{
    "id": 215694124,
    "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
    "title": "All of the Lenny & Friends Summit talks are now online!",
    "subtitle": "Plus, some reflections and takeaways from the day",
    "newsletter_name": "Lenny's Newsletter",
    "newsletter_url": "https://www.lennysnewsletter.com",
    "authors": ["Lenny Rachitsky"],
    "author_handles": ["lenny"],
    "published_at": "2026-09-29T13:15:57.512Z",
    "audience": "free",
    "is_paywalled": false,
    "post_type": "newsletter",
    "reactions": 283,
    "comments": 6,
    "restacks": 5,
    "word_count": 1276,
    "cover_image": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-0e02-4190-a590-1236719b99de_9213x6142.jpeg",
    "text": "👋 Hey there, I’m Lenny. Each week, I share deeply researched product, growth, and career advice. …\n\nI’m excited to share that every main-stage talk from the recent Lenny & Friends Summit is now online. …",
    "source": "https://www.lennysnewsletter.com",
    "scraped_at": "2026-10-10T14:03:07.146048+00:00"
}

A paid post from the same newsletter has "audience": "paid", "is_paywalled": true and no text. The Output tab has two views, Posts and Full text. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data fields

Field	Description
id	Substack's id for the post.
url	Link to the post.
title	The post's title.
subtitle	The subtitle, when the post has one.
newsletter_name	The newsletter's name.
newsletter_url	The newsletter's address (its custom domain when it has one).
authors	Names of the post's authors, guests included.
author_handles	Their Substack handles; each profile is at https://substack.com/@handle.
published_at	When the post was published (UTC, ISO 8601).
audience	free (anyone can read it), paid (paid subscribers), founding (founding members only) or subscribers (free sign-up required).
is_paywalled	true for every post that is not free to read without subscribing.
post_type	Substack's kind of post: newsletter for a written post, podcast for an episode, and so on.
reactions	Likes (hearts) on the post.
comments	Number of comments, replies included.
restacks	Number of restacks.
word_count	Substack's word count for the whole post, also shown for paid posts.
cover_image	The cover image URL, when the post has one.
text	Only with include_full_text, and only for free posts: the post as plain text, one paragraph per line, footnotes as [1].
source	The link or search the post was found through, so posts from a section or archive-search link can be told apart.
scraped_at	When the row was saved.

How much does it cost to scrape Substack?

Substack Scraper uses pay-per-event pricing, so you pay only for results: a fee for each post saved, plus a separate fee for each post saved with its full text, which takes an extra request per post. See the Pricing tab for the exact prices.

Inputs that return nothing are never saved as rows, so you are not charged for them, and each post is saved and charged only once per run, even when several links lead to it. If you set a maximum cost for a run, the Actor stops as soon as it is reached. Apify's free plan includes monthly credit you can use to try it.

Tips for scraping Substack

• Leave full text off when you only need titles, dates and engagement: runs are faster and cheaper, because a whole page of 25 posts arrives in one request.
• Most popular first (sort: top) with a small limit gives a newsletter's best-performing posts in one go.
• Whole archives: set max_posts_per_newsletter to a large number such as 10000. The Actor pages through the archive until it ends.
• Monitoring: schedule a daily run with a small limit (10–20) and newest first to catch new posts.
• One section or topic of a newsletter: paste a section link (https://www.lennysnewsletter.com/s/community-wisdom) or an archive search (https://www.lennysnewsletter.com/archive?search=pricing) instead of the whole newsletter. Some sections, such as members' digests, are not listed by popularity on Substack; use newest first for them.
• Search finds posts, not newsletters. To scrape a whole newsletter you found in a search, paste its address into start_urls. Substack's search shows some posts on more than one page; each is saved once and counted once toward the limit.
• Profiles: substack.com/@handle gives the posts of that person's own newsletter. For posts they wrote as a guest elsewhere, paste those newsletters' links.

FAQ, disclaimers and support

Does it unlock paid posts? No. The Actor reads Substack exactly as a logged-out visitor does and never bypasses a paywall. Paid, founding-member and subscriber-only posts are saved with their public details (title, subtitle, authors, date, likes, comments, restacks, word count, cover image) and marked is_paywalled: true, but never with their text, even when Include full text is on.

Does it work with custom domains? Yes. Any newsletter hosted on Substack works, whether its address is name.substack.com or its own domain.

What happens with a wrong or deleted link? Links that are not Substack newsletters, newsletters that don't exist, addresses with a mistyped domain, sections that don't exist, deleted posts and searches without results are skipped and listed with the reason in the FAILED_INPUTS record of the run's key-value store. The run still succeeds with the posts it could read; it fails only when Substack could not be reached for any input.

Can it scrape comments or Notes? Not yet. It saves the comment count of each post, but not the comments themselves. Notes links (substack.com/@handle/note/…) are skipped and listed in FAILED_INPUTS, so they are never charged.

Is it legal to scrape Substack? The Actor collects only data Substack shows publicly to any visitor. Posts can contain personal data such as author names; make sure you have a legitimate reason to collect it and follow the GDPR and other regulations that apply to you. Respect authors' copyright when you reuse their writing. Substack's robots.txt and Terms of Service apply to your use; if you are unsure, consult your lawyer.

Something missing or not working? Please open an issue on the Issues tab. We read every report and usually fix problems quickly. Need a custom solution, such as comments, Notes or subscriber leaderboards? Get in touch through the Issues tab.