Substack Comments Scraper

Extract public Substack comments and nested replies into a clean, analysis-ready dataset. Add one or many post URLs and receive comment text, thread relationships, authors, timestamps, reactions, and direct source links.

Use the results for audience research, creator intelligence, sentiment pipelines, community monitoring, or newsletter feedback analysis—without manually expanding discussion threads.

What does Substack Comments Scraper do?

The Actor resolves each public Substack post URL to its internal post identifier and reads the public discussion endpoint used by the website.

It saves every publicly available comment and reply as a separate row.

Each row retains its parent comment and depth, so you can reconstruct conversations or analyze top-level feedback separately.

The scraper uses efficient HTTP requests rather than a browser, keeping runs fast and economical.

Why scrape Substack comments?

Reader discussions contain product requests, objections, questions, recommendations, and language that rarely appears in newsletter analytics.

Use structured comments to:

• 🧭 discover recurring audience needs
• 💬 compare feedback across posts
• 📈 monitor engagement over time
• 🔎 find quotes, questions, and topic ideas
• 🧪 feed sentiment and topic-classification workflows
• 🗂️ archive public discussions for research

Who is it for?

Newsletter operators

Track what readers ask for and which topics create meaningful discussion.

Audience-research teams

Combine feedback from multiple posts and classify themes in spreadsheets, notebooks, or BI tools.

Creator intelligence vendors

Collect consistent public engagement records across publications for creator and community analysis.

Researchers

Preserve public conversation structure, authorship, dates, and reaction metadata in machine-readable form.

Developers

Use the API, webhooks, or MCP to make Substack discussion data part of an automated pipeline.

Key features

• Multiple public post URLs per run
• Top-level comments and nested replies
• Parent IDs and explicit thread depth
• Plain text and structured body JSON
• Author names, handles, profiles, and photos
• Published and edited timestamps
• Reaction totals and emoji breakdowns
• Reply counts and deletion state
• Direct post and comment URLs
• Configurable limits and ordering
• JSON, CSV, Excel, XML, and RSS exports
• HTTP-first extraction with bounded retries

Input

The input form contains four fields.

Field	Type	Default	Description
startUrls	array	required	Public Substack post or /comments URLs
maxCommentsPerPost	integer	100	Maximum rows saved for each post
includeReplies	boolean	true	Include replies embedded in discussion threads
sort	string	best	Request best, newest, or oldest order

Example input

{
  "startUrls": [
    { "url": "https://www.lennysnewsletter.com/p/new-a-free-year-of-cursor-google" },
    { "url": "https://www.lennysnewsletter.com/p/a-visual-guide-to-getting-out-of" }
  ],
  "maxCommentsPerPost": 200,
  "includeReplies": true,
  "sort": "newest"
}

Regular post URLs and URLs ending in /comments are both accepted.

Output data

Every dataset row represents one comment or reply.

Field	Description
commentId	Substack comment identifier
parentCommentId	Parent ID for a reply, otherwise null
depth	Thread depth, starting at 0
postId	Internal Substack post identifier
publicationId	Internal publication identifier
postUrl	Canonical source post URL
commentUrl	Direct discussion anchor URL
text	Plain comment text
bodyJson	Structured editor document
authorName	Public display name
authorHandle	Public handle
authorProfileUrl	Public Substack profile URL
authorPhotoUrl	Public profile image URL
publishedAt	Comment creation timestamp
editedAt	Last edit timestamp when available
reactionCount	Total public reactions
reactions	Counts grouped by reaction symbol
replyCount	Number of public child replies
deleted	Whether Substack marks the record deleted
scrapedAt	Extraction timestamp

Example output

{
  "commentId": 123456789,
  "parentCommentId": null,
  "depth": 0,
  "postId": 165204731,
  "publicationId": 10845,
  "text": "This was a useful breakdown.",
  "authorName": "Example Reader",
  "authorHandle": "example-reader",
  "reactionCount": 4,
  "reactions": { "❤": 4 },
  "replyCount": 2,
  "deleted": false,
  "publishedAt": "2026-01-15T10:30:00.000Z"
}

How to scrape Substack comments

1. Open the Actor input page.
2. Paste one or more public Substack post URLs.
3. Choose the maximum comments per post.
4. Keep Include nested replies enabled for complete threads.
5. Select the preferred comment order.
6. Click Start.
7. Export the resulting dataset in your preferred format.

How much does it cost to extract Substack comments?

The Actor uses pay-per-event pricing.

A small start fee covers URL resolution and run initialization. Each saved comment or reply is then charged as one result event. Subscription tiers receive decreasing per-result prices.

The pricing table below uses the current per-comment rates. The final charge is the $0.005 start fee plus the number of comments and replies saved multiplied by your tier's item price.

Apify tier	Price per saved comment or reply	Example calculation for 100 comments
Free	$0.000047465	start fee + 100 item events ≈ 0.97465 cents
Bronze	$0.000041274	start fee + 100 item events ≈ 0.91274 cents
Silver	$0.000032194	start fee + 100 item events ≈ 0.82194 cents
Gold	$0.000024764	start fee + 100 item events ≈ 0.74764 cents
Platinum	$0.000016510	start fee + 100 item events ≈ 0.66510 cents
Diamond	$0.000011557	start fee + 100 item events ≈ 0.61557 cents

For example, a Bronze run that saves 100 comments costs about 0.91274 cents: the $0.005 start fee plus 100 item events at the Bronze rate. A 1,000-comment Bronze run costs about 4.63 cents. Limits are maximums, so a post with fewer public comments costs less. Check the Actor pricing tab before running for the authoritative live prices.

Thread reconstruction

Top-level comments have parentCommentId: null and depth: 0.

Replies contain the parent comment ID and a depth greater than zero.

To rebuild a tree, group rows by postId, index them by commentId, and attach each reply to parentCommentId.

For flat sentiment analysis, simply treat every row independently.

Comment ordering

Choose best to follow Substack's relevance ordering.

Choose newest for monitoring recent reader feedback.

Choose oldest for chronological archives.

Ordering is requested from Substack; nested replies retain their thread relationship.

Tips for reliable runs

• Use canonical public post URLs.
• Start with 20–100 comments per post.
• Split very large URL lists into scheduled batches.
• Keep replies enabled when conversation context matters.
• Deduplicate recurring runs by commentId.
• Store scrapedAt when comparing snapshots.
• Review logs for posts with restricted discussions.

Public-data scope and limitations

The Actor only extracts discussions available through public Substack pages and public web responses.

Subscriber-only, private, removed, geographically blocked, or login-restricted discussions may return no records.

Some posts show an engagement count while withholding comments from anonymous visitors. The Actor does not bypass access controls or request user credentials.

Substack can change its web response structure. The scraper uses retries and clear per-post logs, but upstream changes can temporarily affect extraction.

Integrations

Google Sheets

Send finished datasets to a spreadsheet for tagging reader questions, requests, and sentiment.

Slack

Use an Actor webhook to notify a channel when a scheduled run finds new comments.

Zapier or Make

Trigger downstream CRM, research, or content-planning workflows after each successful run.

Data warehouses

Export JSON or CSV to BigQuery, Snowflake, S3, or your internal analytics pipeline.

AI analysis

Pass text, reactionCount, and thread fields to an LLM for topic clustering, summaries, or intent classification.

Scheduling and monitoring

Create an Apify schedule to run the Actor hourly, daily, or weekly.

For incremental monitoring, retain previous commentId values and keep only unseen IDs after each run.

Use the newest sort order and a practical per-post limit to minimize repeat processing.

API usage with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/substack-comments-scraper').call({
  startUrls: [{ url: 'https://www.lennysnewsletter.com/p/new-a-free-year-of-cursor-google' }],
  maxCommentsPerPost: 100,
  includeReplies: true,
  sort: 'newest'
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API usage with Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')
run = client.actor('automation-lab/substack-comments-scraper').call(run_input={
    'startUrls': [{'url': 'https://www.lennysnewsletter.com/p/new-a-free-year-of-cursor-google'}],
    'maxCommentsPerPost': 100,
    'includeReplies': True,
    'sort': 'newest',
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

API usage with cURL

curl -X POST \
  'https://api.apify.com/v2/acts/automation-lab~substack-comments-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"startUrls":[{"url":"https://www.lennysnewsletter.com/p/new-a-free-year-of-cursor-google"}],"maxCommentsPerPost":100,"includeReplies":true}'

Use with Apify MCP and Claude

The Actor is available through Apify's hosted MCP server at:

https://mcp.apify.com?tools=automation-lab/substack-comments-scraper

Claude Code MCP setup

Run this command once, then start a new Claude Code session:

$claude mcp add --transport http apify-substack-comments 'https://mcp.apify.com?tools=automation-lab/substack-comments-scraper'

Claude Desktop, Cursor, and VS Code MCP setup

Add this server to the app's MCP JSON configuration. Claude Desktop uses claude_desktop_config.json; Cursor and VS Code accept the same mcpServers entry in their MCP settings:

{
  "mcpServers": {
    "apify-substack-comments": {
      "url": "https://mcp.apify.com?tools=automation-lab/substack-comments-scraper"
    }
  }
}

Restart the client after saving the configuration. If your client asks for Apify authorization, complete the browser sign-in flow.

Example prompts for this actor:

• “Run the Substack Comments Scraper on https://www.lennysnewsletter.com/p/new-a-free-year-of-cursor-google, return the newest 100 comments, and include nested replies.”
• “Extract public comments from these three Substack post URLs, then summarize recurring product requests and cite each comment URL.”
• “Group the scraped replies by parentCommentId and identify top-level questions that have no author response.”
• “Compare reactionCount and reply depth across these newsletter discussions and list the most engaged threads.”

Export formats

The default Apify dataset supports JSON, JSONL, CSV, Excel, XML, HTML, and RSS exports.

Use the overview view for a concise table, or request raw dataset items to retain bodyJson and the complete reaction object.

Data quality notes

Author fields can be absent when an account is removed or a comment is anonymized.

Edited timestamps are optional.

Deleted comments can have empty text.

Reaction symbols vary by publication and over time.

Reply counts can include replies that are no longer publicly accessible.

Error handling

A failed post is logged with its URL and reason while other input URLs continue processing.

Invalid non-Substack URLs are rejected per item.

HTTP errors are retried a bounded number of times.

The Actor exits cleanly after processing all posts so successful results remain available.

Legality: is scraping Substack comments legal?

This Actor collects publicly visible information. Laws and contractual obligations vary by jurisdiction and use case.

Only process data you are authorized to collect. Respect privacy rights, intellectual property, Substack's terms, publication rules, and applicable data-protection law.

Avoid using personal data for harassment, discrimination, spam, or other harmful purposes.

Troubleshooting

Why did a post return zero comments?

The discussion may be empty, subscriber-only, removed, or unavailable to anonymous visitors. Open the post in a logged-out browser to verify public access.

Why are fewer rows returned than the visible count?

Some visible counts include comments withheld by access rules or removed records. Increase maxCommentsPerPost and keep replies enabled, but the Actor only returns publicly accessible rows.

Why does a reply have depth greater than one?

Substack supports nested discussion branches. Use parentCommentId to locate the immediate parent.

How do I avoid duplicates in scheduled runs?

Use commentId as the stable deduplication key within your destination system.

Related Substack actors

• Substack Scraper — extract posts, content, and broader publication data.
• Substack People Scraper — collect public writer and profile data.
• Substack Leaderboard Scraper — analyze publication rankings and categories.

Choose this Actor when comments, replies, reactions, and discussion structure are the primary output.

FAQ

Can it scrape any Substack publication?

It supports public posts on *.substack.com domains and custom publication domains whose URLs use Substack's /p/ post structure.

Does it require my Substack account?

No. The public scope uses no user credentials.

Are nested replies included?

Yes, by default. Disable includeReplies when only top-level comments are needed.

Can I process multiple posts?

Yes. Add multiple entries to startUrls; the per-post limit applies independently.

Can I download CSV or Excel?

Yes. Choose CSV or Excel from the dataset export menu after the run.

Can I run it on a schedule?

Yes. Use Apify schedules and webhooks for recurring comment monitoring.

What identifies a unique comment?

Use commentId. Combine it with postId if your storage model requires a compound key.

Does it scrape private discussions?

No. Private and access-restricted comments are intentionally outside scope.

Support

If a public post fails unexpectedly, include the post URL, run ID, and relevant log message in your support request. This makes it easier to distinguish an upstream access restriction from a parser change.