Substack Scraper: Posts, Newsletters, Notes & Leaderboards
Pricing
from $1.00 / 1,000 results
Substack Scraper: Posts, Newsletters, Notes & Leaderboards
Scrape Substack newsletters, posts, comments, Notes and leaderboards. Find newsletters by keyword with subscriber counts, paid prices, posting cadence, sponsors and public contact emails. Track leaderboard growth. No login needed; pay only for results.
Pricing
from $1.00 / 1,000 results
Rating
0.0
(0)
Developer
Ten Lakes Software
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
26 minutes ago
Last modified
Categories
Features
Share
What is Substack Scraper?
Substack Scraper extracts public data from Substack newsletters: posts, newsletters, subscriber counts, paid plan prices, sponsors, comments, Notes, author profiles, recommendations and category leaderboards. Search Substack by keyword, enrich every newsletter with business intelligence, or pull a publication's full post archive, all without a Substack account or API key.
It's built for people who use Substack data to make money decisions: newsletter ad buyers, sponsorship and influencer agencies, marketers building outreach lists, investors tracking the creator economy, and AI teams that need clean newsletter text.
Try it now: click Try for free, keep the default input (search for "AI" newsletters) and press Start. You'll have about 100 newsletters with subscriber counts in under a minute.
- 🔎 Find newsletters by keyword with public subscriber counts, follower counts and bestseller tier
- 💰 Newsletter intelligence: monthly and yearly paid prices, posting cadence, active or dormant status, paid-post share
- 🤝 Sponsor detection: who sponsors which newsletter, with brand, domain, link and placement
- 📈 Leaderboard growth tracking: rank and subscriber changes between scheduled runs
- 📬 Public business contacts: emails a newsletter publishes itself, with the source of each one
- 📝 Posts, comments, Notes, profiles and recommendations, with only-new mode for scheduled runs
- ✅ Compliance-safe by default: no paywall bypass, no commenter identities unless you turn them on
What Substack data can you extract?
| Mode | What you get |
|---|---|
| Search | Newsletters (or posts) matching your keywords, with subscriber count, follower count, bestseller tier, author and description |
| Publication intelligence | Monthly and yearly price, currency, posts in the last 90 days, average days between posts, last post date, active flag, paid-post share, recent sponsors, "takes sponsors" signal, advertise page, X/Twitter, optional public contact emails |
| Posts | Title, subtitle, URL, date, authors, audience (free or paid), paywall flag, word count, likes, comments, restacks, cover image, podcast info, optional full text, sponsors and comments |
| Leaderboards | Rank, category, paid or overall board, subscriber band, price, logo, plus rank change, subscriber change and new entries since your last run |
| Comments | Comment text, date, likes, replies, reply depth, author-reply flag, and the commenter's own newsletter (optional) |
| Notes | Note text, date, likes, restacks, replies, and attachments such as recommended newsletters and shared links |
| Profiles | Name, handle, bio, subscriber and follower counts, bestseller tier and the newsletters they write |
| Recommendations | Which newsletters recommend which, so you can map a niche or find cross-promotion partners |
Results open in tidy tables in Apify Console, with separate views for Newsletters, Contacts, Sponsors, Posts, Leaderboard, Comments, Notes, Profiles and Recommendations. Export to JSON, CSV, Excel, XML or HTML, or fetch them through the API.
Who uses this Substack scraper?
Newsletter ad buyers and brands
See which companies sponsor which newsletters, how often, and where the ad sits in the post. Find newsletters in your niche that already take sponsors, then check their size, price and posting rhythm before you reach out.
Sponsorship and influencer agencies
Build prospect lists of newsletters by topic with subscriber counts, prices, X/Twitter handles and the contact emails they publish. Track which brands are spending in your clients' categories.
Marketers and growth teams
Find partners for cross-promotion through the recommendations graph, monitor competitors' new posts, and see what topics get the most likes and comments.
Investors and analysts
Schedule the leaderboard mode weekly and get a feed of which newsletters are climbing fastest in each category, with rank and subscriber changes calculated for you.
AI and research teams
Pull clean full text of public posts for analysis, summarization or retrieval. Turn on only new since last run so scheduled runs never send you the same post twice.
How to scrape Substack
- Click Try for free and sign in to Apify (a free account comes with monthly credit).
- Under What to scrape, pick a mode: Search, Publication intelligence, Posts, Leaderboard, Recommendations, Profiles or Notes.
- Fill in what you're looking for: keywords, newsletter URLs (for example
lennyorhttps://www.lennysnewsletter.com), leaderboard categories or author handles. - Turn on extras if you want them: full post text, sponsor detection, comments, publication intelligence or contact emails.
- Optionally set Max cost per run so the run stops at your budget.
- Press Start. When the run finishes, open the Output tab and pick the view you want, or export the data.
- To keep the data fresh, schedule the run (daily or weekly) and turn on Only new since last run or Track rank & subscriber growth.
Custom domains work too: www.lennysnewsletter.com and lenny.substack.com are treated as the same newsletter.
How much does it cost to scrape Substack?
Substack Scraper uses pay-per-event pricing. You pay only for the data you get, and platform usage is included: there's no separate compute bill.
| Event | Price |
|---|---|
| Run start | $0.005 (at the default 2 GB memory) |
| Result: post, newsletter, leaderboard row, profile, Note or recommendation | $0.001 ($1.00 per 1,000) |
| Comment | $0.0005 ($0.50 per 1,000) |
| Publication intelligence (one fully enriched newsletter) | $0.005 |
| Sponsor mention found | $0.01 |
| Public contact email found | $0.02 |
| Leaderboard growth change detected | $0.01 |
Real-world cost examples
| Job | Roughly costs |
|---|---|
| Last 100 posts from one newsletter | about $0.11 |
| Top 100 of one leaderboard category | about $0.11 |
| 500 comments from a popular post | about $0.26 |
| 100 "AI" newsletters with full intelligence | about $0.51, plus $0.01 per sponsor and $0.02 per email found |
| Weekly growth report for 3 categories | about $0.30, plus $0.01 per newsletter that moved |
Sponsor and email charges apply only when something is actually found. A lookup that finds nothing is free. Set Max cost per run and the scraper stops cleanly at your limit, so you never pay more than you planned.
Input example
Find AI newsletters that take sponsors, with prices and public contacts:
{"mode": "search","keywords": ["AI", "machine learning"],"searchType": "publications","enrichPublications": true,"includeContactEmails": true,"maxItemsPerSource": 50}
Weekly growth tracker for two leaderboard categories (schedule it):
{"mode": "leaderboard","categories": ["Technology", "Business"],"leaderboardType": "paid","trackGrowth": true}
New posts with sponsors and comments from newsletters you follow:
{"mode": "posts","publications": ["https://newsletter.pragmaticengineer.com", "lenny"],"detectSponsors": true,"includeComments": true,"onlyNew": true}
Output example
Publication intelligence for one newsletter (real output, shortened):
{"type": "publication","name": "The Pragmatic Engineer","url": "https://newsletter.pragmaticengineer.com","subscriberCountLabel": "1.1M+","monthlyPrice": 15,"yearlyPrice": 150,"priceCurrency": "USD","postsLast90Days": 12,"avgDaysBetweenPosts": 2.5,"isActive": true,"paidPostShare": 0.58,"acceptsSponsors": true,"recentSponsors": ["turbopuffer.com", "workos.com", "linear.app"],"twitter": "https://x.com/GergelyOrosz"}
A detected sponsor:
{"type": "sponsor","sponsorName": "WorkOS","sponsorDomain": "workos.com","placement": "top","confidence": "high","cue": "Brought to You by","postUrl": "https://newsletter.pragmaticengineer.com/p/..."}
A leaderboard row with growth tracking (illustrative values):
{"type": "leaderboard","rank": 4,"previousRank": 9,"rankChange": 5,"name": "Example Newsletter","categoryName": "Technology","subscriberCount": 120000,"subscriberChange": 3000,"isNewEntry": false}
Every row has a type field, so you can filter one export into posts, newsletters, sponsors and comments in a spreadsheet.
How does sponsor detection work?
The scraper reads each post and looks for sponsorship cues such as "Brought to you by", "Sponsored by", "Today's sponsor" and "In partnership with". It then takes the brand and link that follow the cue. Each sponsor becomes its own row with the brand name, website domain, link, placement (top, middle or bottom of the post), a confidence level and the surrounding text, so you can check any match in seconds.
In publication intelligence mode, it scans a newsletter's most recent posts (6 by default) and summarizes them as recentSponsors and an acceptsSponsors flag. That's the fastest way to shortlist newsletters that already sell ads.
Track Substack leaderboard growth
Substack's category leaderboards show who's on top today, but not who's rising. Turn on Track rank & subscriber growth and schedule the run. On every run after the first, each row includes previousRank, rankChange, subscriberChange and isNewEntry, and each real change is reported as a growth event. You get a ready-made "fastest growing newsletters" feed for any category, paid or overall.
Use Substack Scraper with the API
Run it from your own code with the Apify API client.
JavaScript / Node.js
import { ApifyClient } from 'apify-client';const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });const run = await client.actor('tenlakessoftware/substack-scraper').call({mode: 'search',keywords: ['personal finance'],enrichPublications: true,});const { items } = await client.dataset(run.defaultDatasetId).listItems();console.log(items);
Python
from apify_client import ApifyClientclient = ApifyClient("YOUR_APIFY_TOKEN")run = client.actor("tenlakessoftware/substack-scraper").call(run_input={"mode": "posts","publications": ["lenny"],"maxItemsPerSource": 50,})for item in client.dataset(run["defaultDatasetId"]).iterate_items():print(item["title"], item["url"])
You can also call it over plain HTTP with the run-sync-get-dataset-items endpoint and get the results back in a single request. See the API tab on this page for every option.
Use with AI agents (MCP)
AI assistants such as Claude, ChatGPT and Cursor can run Substack Scraper through the Apify MCP server. Connect your assistant to Apify, then ask in plain language, for example:
- "Find 30 newsletters about climate tech that take sponsors and list their prices."
- "Which Technology newsletters gained the most rank this week?"
- "Pull the last 20 posts from Lenny's Newsletter and summarize the main topics."
The assistant calls tenlakessoftware/substack-scraper with the right input and reads the results for you.
Integrations
Send Substack data wherever you work. Connect runs to Google Sheets, Slack, Zapier, Make, n8n, Airtable or any webhook, and use Apify schedules to run it daily or weekly. A common setup: a weekly leaderboard run with growth tracking that posts the biggest movers to Slack.
Tips for best results
- Substack search returns about 100 results per keyword. Use several related keywords ("AI", "machine learning", "LLM") for wider coverage. A newsletter found under more than one keyword is returned, and billed, only once.
- For lead lists, combine Search with Add publication intelligence and Include publicly listed contact emails.
- For monitoring, schedule Posts mode with Only new since last run turned on.
- Use the Overview view for a quick look and the specific views (Sponsors, Contacts, Leaderboard) for exports.
- If a newsletter can't be reached, the run skips it, keeps going, and lists the reason in the Run summary.
Is it legal to scrape Substack?
Substack Scraper collects only public, logged-out data, the same information anyone can see in a browser. It never bypasses paywalls, never logs in, and doesn't collect commenter names or handles unless you switch that on. Contact emails are only ones a newsletter publishes itself (for example on its About page), and each comes with its source.
Scraping public data is generally lawful, but your use of it must follow the laws that apply to you, such as GDPR for personal data and CAN-SPAM for email outreach. If you're unsure, ask a lawyer. Read more in Apify's guide to the legality of web scraping.
FAQ
Do I need a Substack account or API key?
No. Substack Scraper works with public pages and needs no login, cookies or API key.
How accurate are the subscriber counts?
They're the public figures Substack shows on each newsletter, such as "1.1M+ subscribers" or "Thousands of paid subscribers". Large newsletters are shown rounded by Substack itself. The raw label is always included next to the number.
Can it scrape paid or paywalled posts?
It returns metadata for every post, including paid ones, and flags them as paywalled. It never accesses paid content, so full text is only available for free posts.
Does it work with custom domains?
Yes. Both name.substack.com and custom domains such as www.lennysnewsletter.com work.
How do I get only new posts?
Turn on Only new since last run and schedule the run. The scraper remembers what it already delivered to you and only returns, and only charges for, new posts.
Why did my run stop early?
Either it reached Max total results or your Max cost per run budget. The run's status message tells you which. Raise the limit to get more.
What happens if Substack changes its site?
We run automated health checks against live Substack every few hours. If Substack changes something, the run fails loudly with a clear message instead of returning empty data, and we're alerted to fix it.
Can I get the data in Excel or Google Sheets?
Yes. Export any run as Excel or CSV from the Output tab, or connect the Google Sheets integration to update a sheet automatically.
I need a field or mode that isn't here.
Open an issue on the Issues tab with an example newsletter URL. We actively maintain this Actor and add requested fields regularly.
Support
Found a bug or have an idea? Use the Issues tab and we'll reply quickly. Need a custom Substack or newsletter data solution for your team? Reach us through our Apify profile.
If Substack Scraper saves you time, a rating on this page helps other people find it. Thank you!
Substack Scraper is built and maintained by Ten Lakes Software, Minnesota. It isn't affiliated with or endorsed by Substack Inc.