Substack Scraper - Posts, Substack Notes & Newsletter Archive

A Substack scraper for posts and Substack Notes: paste publication URLs or profile handles and get every post of a newsletter archive (title, subtitle, date, paywall flag, likes, restacks, comments, word count, tags, authors, cover image, podcast link and, for free posts, the full article text) plus the Notes of any profile (text, date, likes, restacks, replies, attached posts and links). It reads the public JSON endpoints that every Substack publication serves, so it needs no login, no API key and no proxy, and it never bypasses a paywall.

What does Substack Scraper do?

Give it newsletters (https://www.lennysnewsletter.com/, noahpinion, any post URL, or a profile URL that is resolved to the author's main publication) and it walks the archive newest first, 50 posts per request, with an optional date cut-off so scheduled runs only read what is new. Give it profile handles (lenny, @noahpinion, https://substack.com/@lenny) and it reads their Substack Notes feed with the same metrics. Both kinds land in one dataset with a kind field (post or note); the Posts and Notes views show each kind as a clean table, and CSV, Excel, JSON and the API export work as usual.

Typical runs: export a whole newsletter archive with full text for research or a RAG index, monitor competitor newsletters and their engagement, collect the Notes of the creators you follow, find a publication's most liked posts, or build a topic feed across many newsletters.

Who is it for?

• Newsletter operators and content marketers: competitor archives with likes, restacks and comments per post; top posts by engagement; which topics and tags perform.
• Researchers, journalists and analysts: searchable archives of expert newsletters as plain text, with dates, authors and paywall status.
• AI and RAG builders: the full text of free posts as clean plain text, one row per post, ready for embeddings.
• Creators and ghostwriters: the Notes of a profile with engagement, to see what works on Substack Notes.
• Growth and sales teams: a scheduled feed of new posts and Notes from a watchlist of publications and people.

What data can you get?

Posts (kind = post):

Field	Example
id, title, subtitle, slug, url	215694124, All of the Lenny & Friends Summit talks are now online!, Plus, some reflections and takeaways from the day, ..., https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit
postType, publishedAt	newsletter (or podcast, thread, video), 2026-09-29T13:15:57.512Z
audience, isPaywalled	everyone / only_paid / founding, false
likes, restacks, comments, wordCount	291, 5, 7, 1276
tags, sectionName, language	["Career"], null, en
authors	[{"name": "Lenny Rachitsky", "handle": "lenny", "url": "https://substack.com/@lenny", "isGuest": false}]
description, coverImage, podcastUrl, podcastDurationSeconds	the post description, cover image URL, audio URL and length for podcast posts
publicationId, publicationName, publicationUrl	10845, Lenny's Newsletter, https://www.lennysnewsletter.com
bodyText, bodyTruncated	the full article as plain text (free posts, with Include the full text on); paywalled posts carry the public preview and bodyTruncated = true

Notes (kind = note):

Field	Example
id, url	216329331, https://substack.com/@lenny/note/c-216329331
text, publishedAt, editedAt	the Note as plain text, 2026-02-18T18:18:50.292Z, null
likes, restacks, replies	36, 1, 6
isRestack, attachments	true when the Note only shares something; [{"type": "publication", "title": "Marketing Ideas", "url": "https://www.marketingideas.com"}] (types: post, link, image, publication, note)
authorName, authorHandle, authorUrl, authorId	Lenny Rachitsky, lenny, https://substack.com/@lenny, 1849774
publicationName, publicationUrl	Lenny's Newsletter, https://www.lennysnewsletter.com

Authors are the publishers' public bylines. Commenters, subscribers and other reader data are never collected.

How much does it cost?

You pay per item: $1.20 per 1,000 posts or Notes ($0.0012 per item) on the Free plan, with Store discounts on paid Apify plans: Bronze $1.08, Silver $1.02, Gold and above $0.96 per 1,000 items. Platform usage is included; the full text of free posts costs nothing extra.

Worked examples:

• The archive of one newsletter, 300 posts with metadata: 300 x $0.0012 = $0.36.
• The same archive with the full text of its 180 free posts (Only free posts on): 180 x $0.0012 = $0.22.
• A daily watchlist of 20 publications and 20 profiles with Published after set to yesterday, about 15 new posts and 40 new Notes a day: 55 x $0.0012 = $0.07 per day.

Publications and profiles that cannot be read (private, deleted, mistyped) are listed in the run's OUTPUT record and are never charged. Posts skipped by a filter are not charged either. Set Max items or a maximum cost per run to cap spending.

How to use it

1. Click Try for free.
2. Paste newsletters into Publications (URL, subdomain or post URL, one per line) and profile handles into Profiles (Substack Notes). Either list can stay empty.
3. Optional: turn on Include the full text of free posts, Only free posts, choose Top posts first, or set Published after for a scheduled feed.
4. Click Start and download the table as CSV, Excel, JSON or through the API. Switch to the Posts or Notes view for a table of one kind.

Input example

{
  "publications": ["https://www.lennysnewsletter.com/", "noahpinion"],
  "profiles": ["lenny"],
  "maxPostsPerPublication": 50,
  "maxNotesPerProfile": 50,
  "includeFullText": false,
  "onlyFreePosts": false,
  "sortPosts": "new",
  "maxItems": 1000
}

Output example

Two unedited rows from a live run (long strings shortened):

{
  "kind": "post",
  "id": 215694124,
  "title": "All of the Lenny & Friends Summit talks are now online!",
  "subtitle": "Plus, some reflections and takeaways from the day",
  "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit",
  "slug": "all-of-the-lenny-and-friends-summit",
  "postType": "newsletter",
  "publishedAt": "2026-09-29T13:15:57.512Z",
  "audience": "everyone",
  "isPaywalled": false,
  "description": "Plus, some reflections and takeaways from the day",
  "likes": 291,
  "restacks": 5,
  "comments": 7,
  "wordCount": 1276,
  "tags": [],
  "sectionName": null,
  "authors": [{"name": "Lenny Rachitsky", "handle": "lenny", "url": "https://substack.com/@lenny", "isGuest": false}],
  "language": "en",
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-...jpeg",
  "podcastUrl": null,
  "podcastDurationSeconds": null,
  "publicationId": 10845,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "bodyText": null,
  "bodyTruncated": null
}
{
  "kind": "note",
  "id": 216329331,
  "url": "https://substack.com/@lenny/note/c-216329331",
  "text": "I'm thrilled to welcome The Skip with @Nikhyl Singhal to Lenny's Podcast Network!\nI've been a massive fan of Nikhyl's for many years. He spoke at my inaugural summit and was an early guest on my podcast. ...",
  "publishedAt": "2026-02-18T18:18:50.292Z",
  "editedAt": null,
  "likes": 36,
  "restacks": 1,
  "replies": 6,
  "isRestack": false,
  "attachments": [{"type": "link", "title": "- YouTube", "url": "https://www.youtube.com/watch?v=38FFgtSXMew&t=1s"}],
  "authorName": "Lenny Rachitsky",
  "authorHandle": "lenny",
  "authorUrl": "https://substack.com/@lenny",
  "authorId": 1849774,
  "publicationName": "Lenny's Newsletter",
  "publicationUrl": "https://www.lennysnewsletter.com",
  "language": "en"
}

Integrations and API

Run it from your code with the Apify API or the apify-client package, schedule it in Apify (a daily run with Published after gives you only the new posts and Notes), or connect it to Make, n8n, Zapier, Google Sheets or a webhook.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("kamerozkan/substack-scraper-posts-notes").call(run_input={
    "publications": ["https://www.lennysnewsletter.com/", "noahpinion"],
    "profiles": ["lenny"],
    "includeFullText": True,
    "onlyFreePosts": True,
    "maxItems": 200,
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["kind"], item.get("title") or item.get("text"), item["publishedAt"], item["likes"])

FAQ

Does it return the text of paid posts? No. The full text is read for free posts only. Paywalled posts return their public metadata and the preview that Substack shows to everyone, with bodyTruncated = true. Use Only free posts to leave them out entirely.

How do I get only new posts on a schedule? Set Published after to the date of your last run (or use a date one day back for a daily schedule). With Newest first the scraper stops reading the archive as soon as it reaches older posts, so each run costs only the new items.

Can I scrape Notes of a profile without a publication? Yes. Notes come from the profile feed; publicationName is then empty.

What counts as a Note? Every entry of the profile's Notes feed: original Notes with text, and restacks (shares of a post or publication without text), flagged with isRestack = true and the shared item in attachments.

Which URLs are accepted? Publication home pages on substack.com subdomains or custom domains, any post URL (the publication is derived from it), bare subdomains, and profile URLs or handles. Private or deleted publications end up in the OUTPUT record as failed and are not charged.

How fast is it? About one request per 50 posts and one per page of Notes, with a short pause between requests to stay polite; a 300-post archive takes a few seconds, full text adds about half a second per free post.

Is it legal to collect this data? It collects publicly available posts, Notes and publisher bylines only, respects paywalls and never collects reader or subscriber data. You are responsible for how you use it, including copyright and privacy laws like GDPR.

Something is not working? Open an issue on the Issues tab with your run link and we will fix it fast.

Support

Questions, a publication that answers oddly, or a field you need (for example comments of a post, podcast transcripts or a publication's recommendations)? Open an issue on the Issues tab of this actor with your run link. Issues are read every working day and fixes ship within a few days; feature requests that other users share get built.