Substack Leaderboard Scraper
Pricing
from $1.99 / 1,000 results
Substack Leaderboard Scraper
Substack Leaderboard Scraper Substack Leaderboard scraper extracts publicly available ranking data, newsletter names, authors, categories, subscriber indicators, and leaderboard metrics 🏆📰 Perfect for market research, creator discovery, trend analysis, and competitive intelligence.
Pricing
from $1.99 / 1,000 results
Rating
0.0
(0)
Developer
Scrapers Hub
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
2 days ago
Last modified
Categories
Share
Scrapes the full publication object for every newsletter on a Substack category leaderboard (e.g. Technology → paid), in leaderboard rank order.
How it works
- Each leaderboard URL is parsed into a category slug and a ranking
type —
https://substack.com/leaderboard/technology/paid→technology,paid. - The slug is resolved to a numeric category id via
GET https://substack.com/api/v1/categories(with a bundled fallback map). - Publications are read from the public leaderboard API, paginated 25 at a time:
GET https://substack.com/api/v1/category/public/{categoryId}/{type}?page=N&limit=25. This endpoint already returns the complete publication object (plans,theme,appTheme,contributors,logoPalette, subscriber ranking details, …) — no per-publication request is needed. - Every publication is pushed to the dataset, enriched with
leaderboardUrl,leaderboardRank, andscrapedAt.
Networking
Built on the requested stack:
requests— primary HTTP client.curl_cffi— automatic fallback with a real Chrome TLS fingerprint, used wheneverrequestsreturns a block/challenge (401/403/407/429/503 or a challenge page) or raises.regex— leaderboard-URL parsing and embedded-JSON (window._preloads) extraction.parsel— lightweight HTML parsing (page title / validation).
Apify Proxy is used by default; a pool of pre-rotated proxy URLs is cycled across requests.
Input
| Field | Type | Description |
|---|---|---|
leaderboardUrls | string[] | One or more https://substack.com/leaderboard/<category>/<paid|free> URLs. Required. |
maxItems | integer | Max publications to scrape in total. 0 = unlimited. Default 50. |
proxyConfiguration | object | Proxy settings. Default: Apify Proxy. |
{"leaderboardUrls": ["https://substack.com/leaderboard/technology/paid"],"maxItems": 50,"proxyConfiguration": { "useApifyProxy": true }}
Output
One dataset item per publication — the full Substack publication object plus:
leaderboardUrl— the source leaderboard URL.leaderboardRank— 1-based rank on that leaderboard.scrapedAt— ISO-8601 timestamp (...Z).
Run locally
pip install -r requirements.txtapify run # requires the Apify CLI