# Substack Newsletter & Full Article Intelligence (`lepone31/substack-newsletter-intelligence`) Actor Scrape full articles, clean markdown text, voiceover audio streams & engagement metrics from any Substack newsletter for AI RAG & NotebookLM. - **URL**: https://apify.com/lepone31/substack-newsletter-intelligence.md - **Developed by:** [Cedric poma](https://apify.com/lepone31) (community) - **Categories:** AI, Business, Automation - **Security:** Limited permissions - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $2.50 / 1,000 substack article extracteds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## 📰 Substack Newsletter & Full Article Intelligence Scraper > Scrape full article text, clean Markdown, voiceover audio streams (MP3), word counts, reading times, and engagement metrics from any Substack newsletter for AI RAG pipelines, NotebookLM, Notion & Obsidian. *** ### ⚡ What Does This Actor Do? Substack hosts the world's highest-quality independent analysis across Tech, AI, Macro Finance, Strategy, and Culture (authors like Lenny Rachitsky, Packy McCormick's *Not Boring*, Mario Gabriele's *The Generalist*, Alex Xu's *ByteByteGo*, Gergely Orosz's *The Pragmatic Engineer*, etc.). This Actor provides an automated, lightweight API to extract both archive metadata and full formatted post contents: - **Any Substack Publication**: Supports publication handles (`thegeneralist`, `bytebytego`, `lenny`), direct substack URLs (`https://stratechery.substack.com`), and custom domains (`https://www.generalist.com`). - **AI-Ready Formats**: - `contentMarkdown`: Formatted Markdown with clean headers, bullet points, quotes, and links (ready for Obsidian, Notion, or NotebookLM). - `contentText`: Pure plain text stripped of HTML tags (perfect for LLM context windows and vector embeddings). - `contentHtml`: Raw structured article HTML. - **Voiceover & Podcast Audio**: Extracts direct MP3 voiceover / narration links (`audioStreamUrl`) when available for automated transcription via Whisper. - **Engagement & Social Proof**: Reactions count, comments count, estimated reading time, and word counts. - **Aggregated Publication Summary**: Includes a `publicationSummary` record with average word counts, top 3 most popular articles, and audio availability. *** ### 💡 Top Use Cases 1. **NotebookLM & AI Knowledge Base Feeding**: Ingest weekly analysis from top thought leaders into Google NotebookLM or custom RAG pipelines. 2. **Automated Newsletter Monitoring & Competitive Intelligence**: Track strategy updates, market breakdowns, and company teardowns automatically via Make/n8n. 3. **Audio Extraction & Whisper Transcriptions**: Automatically download audio voiceover streams for podcast repurposing or text summarization. 4. **Personal Knowledge Management (PKM)**: Sync full markdown posts into Obsidian vaults, Readwise, or Notion databases. *** ### 📥 Input Parameters | Parameter | Type | Default | Description | | :--- | :--- | :--- | :--- | | `publications` | Array of Strings | `["thegeneralist", "bytebytego"]` | Substack handles or URLs (e.g. `thegeneralist`, `https://www.generalist.com`). | | `maxArticlesPerPublication` | Integer | `15` | Max recent articles to retrieve per newsletter (1 to 100). | | `extractFullText` | Boolean | `true` | Fetch complete article body formatted in clean Markdown, Plain Text and HTML. | | `onlyFreeArticles` | Boolean | `false` | Exclude paid subscriber paywalled stubs. | | `includeSummary` | Boolean | `true` | Include an aggregate `publicationSummary` record. | *** ### 📤 Output Format #### 1. Full Article Record (`article`) ```json { "recordType": "article", "id": "220756708", "title": "Pete Florence: When Robots Improvise", "subtitle": "As intelligence scales, robots are learning more like humans.", "slug": "pete-florence-when-robots-improvise", "publication": "thegeneralist", "publicationUrl": "https://thegeneralist.substack.com", "articleUrl": "https://www.generalist.com/p/pete-florence-when-robots-improvise", "postDate": "2026-10-20T12:04:23.312Z", "authors": ["Mario Gabriele"], "audience": "everyone", "isPaidOnly": false, "wordCount": 4250, "readingTimeMinutes": 19.3, "reactionCount": 342, "commentCount": 28, "hasAudio": true, "audioStreamUrl": "https://api.substack.com/api/v1/audio/upload/9e69a2ac-b748-46c2-967f-16fda4d7ff21/src", "contentMarkdown": "# Pete Florence: When Robots Improvise\n\nAs intelligence scales, robots are learning...", "contentText": "Pete Florence: When Robots Improvise\n\nAs intelligence scales, robots are learning..." } ``` #### 2. Aggregated Publication Summary (`publicationSummary`) ```json { "recordType": "publicationSummary", "publication": "thegeneralist", "totalArticlesScraped": 15, "freeArticlesCount": 14, "paidArticlesCount": 1, "averageWordCount": 3850, "totalReactions": 4120, "totalComments": 315, "audioArticlesCount": 11, "topArticles": [ { "title": "Pete Florence: When Robots Improvise", "reactions": 342, "wordCount": 4250, "url": "https://www.generalist.com/p/pete-florence-when-robots-improvise" } ] } ``` *** ### 💰 Pay-Per-Event Pricing - **$0.0025 per extracted article** ($2.50 per 1,000 articles). - **Free Overview Summary**: The aggregated publication metrics record is included free of charge. - **Instant Speed**: Pure HTTP TLS emulation without heavy browser overhead. # Actor input Schema ## `publications` (type: `array`): List of Substack publications (e.g. 'thegeneralist', 'bytebytego', 'lenny', 'https://stratechery.substack.com', 'https://www.generalist.com'). ## `maxArticlesPerPublication` (type: `integer`): Maximum number of recent articles to extract per publication (1 to 100). ## `extractFullText` (type: `boolean`): Whether to fetch the complete article text formatted in clean Markdown, Plain Text and HTML (ideal for NotebookLM, Claude and RAG pipelines). ## `onlyFreeArticles` (type: `boolean`): If enabled, only fully accessible free public articles will be kept. ## `includeSummary` (type: `boolean`): Whether to output a 'publicationSummary' record with average word counts, top 3 most popular articles, and audio availability. ## `proxyConfiguration` (type: `object`): Optional proxy settings. Substack public archive endpoints do not require proxies. ## Actor input object example ```json { "publications": [ "thegeneralist", "bytebytego" ], "maxArticlesPerPublication": 15, "extractFullText": true, "onlyFreeArticles": false, "includeSummary": true } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "publications": [ "thegeneralist", "bytebytego" ] }; // Run the Actor and wait for it to finish const run = await client.actor("lepone31/substack-newsletter-intelligence").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "publications": [ "thegeneralist", "bytebytego", ] } # Run the Actor and wait for it to finish run = client.actor("lepone31/substack-newsletter-intelligence").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "publications": [ "thegeneralist", "bytebytego" ] }' | apify call lepone31/substack-newsletter-intelligence --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,lepone31/substack-newsletter-intelligence" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/p2LmDWZ6hqBlDrb5i/builds/Xn3J0hNHJu7u7QOsO/openapi.json