# Substack Scraper (`anyxsolutions/substack-scraper`) Actor Scrape posts from any Substack newsletter, on substack.com or a custom domain: titles, dates, authors, likes, comments, restacks, podcast episodes, and the full text of free posts, with the publication's name and subscriber count. Paste a newsletter URL, get clean JSON. No login, no API key. - **URL**: https://apify.com/anyxsolutions/substack-scraper.md - **Developed by:** [Anyx Solutions](https://apify.com/anyxsolutions) (community) - **Categories:** News, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ![banner](https://anyx.solutions/banners/substack-scraper.png) ## Substack Scraper **Every post from any Substack newsletter ā€” with likes, comments, authors, and the full text of free posts ā€” as clean JSON.** Substack hosts tens of thousands of newsletters and podcasts, from Lenny's Newsletter and Simon Willison's Newsletter to the Dwarkesh Podcast, on `substack.com` addresses and on their own custom domains. This Actor reads any of them and returns their posts, newest first: title, subtitle, publish and update dates, authors, likes, comments, restacks, word count, cover image, podcast audio, and the post body as HTML and plain text. Free posts come in full; paid posts come with the preview Substack shows non-subscribers, and are flagged as such. Every row also carries the newsletter's name, URL, author and subscriber count. Paste a newsletter URL; there is no login, no API key, and no proxy to configure. Built for media monitoring, creator and sponsorship research, content analysis, and AI and search datasets. ### āš” Quick start ```json { "startUrls": [{ "url": "https://www.lennysnewsletter.com/" }], "maxItems": 5 } ``` ### šŸ§© Input | Field | Type | Required | Default | Description | | -------------------- | ------- | -------- | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `startUrls` | array | yes | ā€” | Substack newsletter URLs (`.substack.com` or the newsletter's own domain), archive pages (`ā€¦/archive`), single post URLs (`ā€¦/p/`), or `open.substack.com/pub/ā€¦` links. | | `maxItems` | integer | no | ā€” | Maximum number of posts per start URL, newest first. The input form starts at 5; leave it empty to get the whole archive. | | `includeContent` | boolean | no | `true` | Read each post's body as HTML and plain text. Adds one fast request per post; turn it off for quick archive listings. | | `proxyConfiguration` | object | no | `{ "useApifyProxy": false }` | Proxy settings. Substack newsletters need no proxy. | **Custom domains work.** Many newsletters run on their own domain (for example `www.lennysnewsletter.com`). Paste it as it is: the Actor checks that the site is a Substack newsletter and says so in the log when it is not. ### šŸ“¤ Output Each post is one dataset row, with the newsletter it belongs to. #### Fields | Field | Type | Description | | -------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | number | Substack post ID. | | `title` | string | Post title. | | `subtitle` | string | Post subtitle. | | `slug` | string | The post's name in its URL. | | `url` | string | The post's public page. | | `postDate` | string | When the post was published (ISO 8601). | | `updatedAt` | string | When the post was last edited (ISO 8601). Filled when `includeContent` is on. | | `type` | string | `newsletter`, `podcast` or `thread` (a discussion thread). | | `audience` | string | Who can read it: `everyone` for free posts, `only_paid` for subscriber-only posts. | | `isPaid` | boolean | Whether the post is for paying subscribers. | | `wordcount` | number | Word count of the full post. | | `reactionCount` | number | Number of likes. | | `commentCount` | number | Number of comments. | | `restacks` | number | Number of restacks (Substack's reshares). | | `coverImage` | string | Cover image URL. | | `sectionName` | string | The newsletter section the post belongs to, when it uses sections. | | `tags` | array | Tag names. | | `language` | string | Language of the post. | | `authors` | array | Bylines: `id`, `name`, `handle`, `photoUrl` and `bio`. | | `podcastUrl` | string | Audio file of a podcast episode. | | `podcastDuration` | number | Episode length in seconds. | | `bodyHtml` | string | Post body as HTML. `null` when `includeContent` is off. | | `bodyText` | string | Post body as plain text. `null` when `includeContent` is off. | | `isContentTruncated` | boolean | `true` when the body is the paywall preview of a paid post, `false` when it is the full post, `null` when `includeContent` is off. | | `publicationName` | string | The newsletter's name. | | `publicationUrl` | string | The newsletter's home page. | | `publication` | object | The newsletter: `id`, `name`, `url`, `subdomain`, `customDomain`, `description`, `authorName`, `authorHandle`, `logoUrl`, `subscriberCount` (when the newsletter shows it), `paidSubscriptions`, `firstPostDate` and `language`. |
Example output ```json { "id": 215694124, "title": "All of the Lenny & Friends Summit talks are now online!", "subtitle": "Plus, some reflections and takeaways from the day", "slug": "all-of-the-lenny-and-friends-summit", "url": "https://www.lennysnewsletter.com/p/all-of-the-lenny-and-friends-summit", "postDate": "2026-09-29T13:15:57.512Z", "updatedAt": "2026-09-29T16:23:45.543Z", "type": "newsletter", "audience": "everyone", "isPaid": false, "wordcount": 1276, "reactionCount": 288, "commentCount": 7, "restacks": 5, "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/9fed347c-0e02-4190-a590-1236719b99de_9213x6142.jpeg", "sectionName": null, "tags": [], "language": "en", "authors": [ { "id": 1849774, "name": "Lenny Rachitsky", "handle": "lenny", "photoUrl": "https://bucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com/public/images/afba5161-65bb-4d99-8d6b-cce660917fa1_1540x1540.png", "bio": "Writing ā€¢ Angel investing ā€¢ Advising" } ], "podcastUrl": null, "podcastDuration": null, "bodyHtml": "

šŸ‘‹ Hey there, Iā€™m Lenny. Each week, I share deeply re ā€¦", "bodyText": "šŸ‘‹ Hey there, Iā€™m Lenny. Each week, I share deeply researched product, growth, and career ā€¦", "isContentTruncated": false, "publicationName": "Lenny's Newsletter", "publicationUrl": "https://www.lennysnewsletter.com", "publication": { "id": 10845, "name": "Lenny's Newsletter", "url": "https://www.lennysnewsletter.com", "subdomain": "lenny", "customDomain": "www.lennysnewsletter.com", "description": "Deeply researched product, growth, and career advice for product leaders, founders, and ambitious builders.", "authorName": "Lenny Rachitsky", "authorHandle": "lenny", "logoUrl": "https://substackcdn.com/image/fetch/$s_!8MSN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F441213db-4824-4e48-9d28-a3a18952cbfc_592x592.png", "subscriberCount": 1200000, "paidSubscriptions": true, "firstPostDate": "2019-06-14T15:00:48.833Z", "language": "en" } } ```

### šŸ’” Use cases - **Media monitoring** ā€” follow the newsletters in your industry and catch every new post, with its engagement. - **Creator and sponsorship research** ā€” compare newsletters by subscribers, posting cadence, likes and comments before you pitch or sponsor. - **Content analysis** ā€” collect full post text for topic tracking, sentiment, and trend research. - **AI and search datasets** ā€” build clean, attributed corpora from free newsletter posts and podcast show notes. ### ā“ FAQ - **Do I need a Substack account or an API key?** No. The Actor reads the same public pages any visitor sees. - **Can I get the full text of paid posts?** No. Paid posts come with the preview Substack shows non-subscribers; `isContentTruncated` is `true` on those rows, and `wordcount` still gives the length of the full post. - **Does it work with newsletters on their own domain?** Yes. Paste the domain as it is. If the site is not a Substack newsletter, the log says so and the URL is skipped. - **Can I scrape many newsletters at once?** Yes. Add one URL per newsletter to `startUrls`; `maxItems` applies to each one separately. - **Why is `subscriberCount` empty for some newsletters?** Authors choose whether to show their subscriber count. When they hide it, the field is `null`. - **Do I need a proxy?** No. It runs without one by default. ### šŸ”— More scrapers by Anyx - [Web Content Scraper](https://apify.com/anyxsolutions/web-content-scraper) - [Telegram Channel Scraper](https://apify.com/anyxsolutions/telegram-channel-scraper) - [Bluesky Scraper](https://apify.com/anyxsolutions/bluesky-scraper) - [Google Search Scraper](https://apify.com/anyxsolutions/google-search-scraper) ### šŸ¤ Anyx Solutions We build custom scrapers and data-extraction pipelines. - Email: tantosthor@gmail.com ### šŸ–¼ Image credit Image credit: [substack.com](https://substack.com/) # Actor input Schema ## `startUrls` (type: `array`): Substack publication URLs, such as https://simonw.substack.com or a custom domain like https://www.lennysnewsletter.com. Archive pages (ā€¦/archive), single post URLs (ā€¦/p/) and open.substack.com links work too. ## `maxItems` (type: `integer`): Maximum number of posts to return per start URL, newest first. Leave empty to get the newsletter's whole archive. ## `includeContent` (type: `boolean`): Fetch each post on its own to read its body as HTML and plain text. Free posts come in full; paid posts end at the paywall preview. Turn it off for faster archive listings. ## `proxyConfiguration` (type: `object`): Proxy settings. Substack newsletters need no proxy. ## Actor input object example ```json { "startUrls": [ { "url": "https://www.lennysnewsletter.com/" } ], "maxItems": 5, "includeContent": true, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://www.lennysnewsletter.com/" } ], "maxItems": 5 }; // Run the Actor and wait for it to finish const run = await client.actor("anyxsolutions/substack-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`šŸ’¾ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // šŸ“š Want to learn more šŸ“–? Go to ā†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://www.lennysnewsletter.com/" }], "maxItems": 5, } # Run the Actor and wait for it to finish run = client.actor("anyxsolutions/substack-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"šŸ’¾ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # šŸ“š Want to learn more šŸ“–? Go to ā†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://www.lennysnewsletter.com/" } ], "maxItems": 5 }' | apify call anyxsolutions/substack-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,anyxsolutions/substack-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/UK0EUHEq5M9904ege/builds/vHTTu3yTnnjvibx6w/openapi.json