# Substack Leaderboard Scraper (`deriverge/substack-newsletter-directory`) Actor Top Substack newsletters in each of the 33 categories as Substack ranks them, with author, paid subscriber band, displayed subscriber count and launch date. Read the overall or the paid list to any depth Substack shows, track rank moves daily and add each newsletter's latest posts on request. - **URL**: https://apify.com/deriverge/substack-newsletter-directory.md - **Developed by:** [deriverge s.r.o.](https://apify.com/deriverge) (community) - **Categories:** News, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.50 / 1,000 leaderboard entries This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Substack Leaderboard Scraper Substack Leaderboard Scraper returns the newsletters listed on Substack's category leaderboards, one row per listing: rank, category, newsletter name and link, author, tagline, paid subscriber tier, the subscriber count Substack displays, launch date, language, and whether it sells subscriptions or runs a podcast. Pick categories such as `Technology` or `Finance`, choose how deep to go and click **Start**. Substack publishes paid subscribers only as a band such as `Thousands of paid subscribers`, never as an exact number. It costs $1.00 per 1,000 leaderboard entries on the Free plan ($0.50 on Business), and the $5 of monthly credit in Apify's Free plan covers about 5,000 leaderboard entries. ### What data does it return? | Field | Description | |---|---| | `rank`, `rankingList` | Position on the leaderboard, `1` at the top, and the list it comes from: `all` for the overall list, `paid` for the list Substack ranks by paid subscriptions. | | `category`, `categoryId` | The leaderboard's category, such as `Technology`, and Substack's ID for it, such as `4`. | | `name`, `url` | Newsletter name and its home page, on its own domain when it has one. | | `subdomain`, `customDomain` | The part before `.substack.com`, and the newsletter's own domain such as `www.lennysnewsletter.com`, `null` when it has none. | | `author`, `authorHandle` | Public name and Substack handle of the publication's author. Some newsletters show the publication name here instead of a person. | | `description` | Tagline from the newsletter's home page. | | `subscriberTier` | Paid audience as Substack words it, from `Hundreds of paid subscribers` up to `Hundreds of thousands of paid subscribers`. `null` when Substack shows no paid band for the newsletter. | | `freeSubscribers` | Subscriber count shown on the listing, rounded by Substack, such as `271000`. `null` when the newsletter hides it. | | `paidEnabled` | `true` when the newsletter sells paid subscriptions. | | `launchedAt`, `firstPostAt` | When the publication was created on Substack and when its first post is dated, ISO 8601 in UTC. A newsletter that imported an older archive has a first post before its launch. | | `language`, `hasPodcast`, `explicit` | Language code such as `en`, whether the newsletter publishes a podcast, and whether it is marked as explicit. | | `logo`, `coverImage` | Links to the newsletter logo and cover image, `null` when missing. | | `id`, `key` | Substack's publication ID, and the row key `dir:::` that new-only mode compares between runs. | | `checkedAt` | When the leaderboard page was read, ISO 8601 in UTC. | With **Include latest posts** on, rows of `type` `post` follow the newsletter rows. They carry `title`, `subtitle`, `url`, `publishedAt`, `audience` (`everyone`, `only_paid` or `founding`), `postType`, `authors`, `wordCount`, `likes`, `comments`, `restacks` and the `previewText` Substack shows in listings, but not the full text of the post. ### How to scrape Substack leaderboards 1. In **Categories**, enter category names as Substack writes them, such as `Technology`, `U.S. Politics` or `Health & Wellness`. Upper and lower case and the `&` sign do not matter, and Substack's numeric category IDs work too. Leave the field empty for all 33 categories. 2. Pick the **Ranking list**: **All newsletters** or **Paid newsletters (by paid subscribers)**. A newsletter usually holds a different rank on each list. 3. Set **Newsletters per category**. Substack serves 25 per page, so the top 100 takes four pages per category. 4. For recent posts as well, turn on **Include latest posts** and set **Maximum posts per newsletter** (3 by default). **Published after**, **Audience** and **Keywords (any of)** narrow those posts and never remove a newsletter row. 5. Click **Start**. The rows appear in the **Output** tab and can be exported as CSV, Excel or JSON. The `SUMMARY` record in the key-value store counts the rows by type, names any category it did not recognize and lists categories that were not read in full under `categoriesIncomplete`. An API input for the paid lists of three categories with their latest October posts: ```json { "categories": ["Technology", "Finance", "Science"], "ranking": "paid", "maxPerCategory": 100, "includePosts": true, "maxPostsPerNewsletter": 3, "publishedAfter": "2026-10-01" } ``` ### Example output The third newsletter on the overall Technology leaderboard in a run on 11 October 2026. Its first post is older than its launch on Substack because the archive was imported. ```json { "type": "newsletter", "key": "dir:4:all:piratewires", "id": 5931581, "name": "Pirate Wires", "subdomain": "piratewires", "customDomain": null, "url": "https://piratewires.substack.com", "author": "Pirate Wires", "authorHandle": "piratewires", "description": "An American media company reporting at the intersection of technology, politics, and culture. Subscribe, or die.", "logo": "https://substackcdn.com/image/fetch/$s_!QQiX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Feed454fe-ae6d-484b-bd7a-026c6ccba7b8_250x250.png", "coverImage": null, "language": "en", "launchedAt": "2025-08-08T17:42:37.955Z", "firstPostAt": "2019-01-29T21:39:00.000Z", "paidEnabled": true, "subscriberTier": "Thousands of paid subscribers", "freeSubscribers": 271000, "hasPodcast": true, "explicit": false, "category": "Technology", "categoryId": "4", "rank": 3, "rankingList": "all", "checkedAt": "2026-10-11T15:51:20.212Z" } ``` ### How much does it cost to scrape Substack leaderboards? | | Free plan | Starter | Scale | Business | |---|---|---|---|---| | 1,000 leaderboard entries | $1.00 | $0.80 | $0.65 | $0.50 | | Post, per 1,000 | $0.80 | $0.64 | $0.52 | $0.40 | You pay only for the events in the table. There is no start fee, and compute time and proxies are included. The top 100 of 10 categories is 1,000 rows and costs $1.00 on the Free plan or $0.50 on Business. Three latest posts for each of those newsletters add up to 3,000 post rows, another $2.40 on the Free plan. Posts that **Published after**, **Audience** or **Keywords (any of)** remove, and rows already returned to a watch in new-only mode, are not charged. ### Limits - Leaderboard depth differs by category and list. On 11 October 2026 the overall Comics list ended after 248 newsletters, and the paid Technology list after 525, where Substack answers the next page with HTTP 400 and the actor takes that as the end of the list. - Substack limits fast reading from one address with HTTP 429. The actor reads at most one leaderboard page a second, and after a 429 it waits as long as Substack asks, or 15 seconds, and goes on more slowly. A run of all 33 categories at 100 each on 11 October 2026 returned all 3,300 rows in about 3 minutes, with two such pauses. If Substack keeps refusing, the actor stops asking and returns what it has, and the status message and `categoriesIncomplete` in `SUMMARY` name the categories that are missing or partial. - Newsletter archives fall under the same limit and pace: with **Include latest posts** on for the paid Finance and Science lists (50 each), 99 of the 100 archives were read in under 2 minutes. The one newsletter whose site refused its archive with HTTP 403 came without posts, and the status message said so. - `freeSubscribers` was filled for 1,782 of 1,975 listings in the top 100 run, between 1,000 and 3,000,000 with a median of 40,000. Substack rounds it. - Each category and list makes its own row, so a newsletter listed in two categories appears twice: 42 newsletters did so in the top 100 of all categories. - A run returns at most 10,000 rows unless you raise **Maximum rows**. ### Tracking new entrants and rank moves Save the input as a task, or give it a **Watch name**, and schedule it daily. Every such run writes a `CHANGES` record that compares the listings with the previous run: `added` and `removed` count newsletters that entered or left the leaderboards you read, and each item in `changes` gives a row `key`, its `kind` (`added`, `removed` or `changed`) and, for rank moves, the old and new `rank`. With **Return only rows new since the last run** on, the dataset holds only the newsletters that entered since then. Rows cut off by **Maximum rows** are kept for the next run. ### Which categories are covered Substack has 33 leaderboard categories: Technology, Business, Culture, Finance, U.S. Politics, World Politics, Health Politics, News, International, Food & Drink, Podcasts, Sports, Art & Illustration, Fashion & Beauty, Music, Faith & Spirituality, Climate & Environment, Science, Literature, Fiction, Health & Wellness, Design, Travel, Parenting, Philosophy, Comics, Crypto, Humor, History, Education, Film & TV, Games and Home & Garden. The actor reads the current list from Substack at the start of each run, so a category Substack adds later is picked up without a change. ### FAQ #### Is it legal to scrape Substack leaderboards? The leaderboards are public pages that Substack shows to anyone without an account. The actor stores what a listing shows about a newsletter: its name, rank, tagline, the public name of its author and the subscriber figures Substack chooses to display. It collects no subscriber lists, email addresses or reader accounts, and you are responsible for how you use the data. #### What is the difference between the two lists? **All newsletters** is Substack's overall ranking of a category. **Paid newsletters** ranks only newsletters with paid plans, by paid subscriptions, and almost every row on it has a `subscriberTier`. Run the actor once per list to compare them, and give each list its own **Watch name** when you track changes. #### How do I get the full text of the posts? This actor returns post metadata and the preview. Pass the `url` values of the newsletters to Substack Scraper, which reads whole archives with the text of each post and its comments. ### Related scrapers - [Substack Scraper](https://apify.com/deriverge/substack-scraper) - [Substack Comments Scraper](https://apify.com/deriverge/substack-comments-scraper) - [Apple Podcasts Scraper](https://apify.com/deriverge/apple-podcasts-scraper) ### Support This actor is built and maintained by deriverge s.r.o., a software company based in the Czech Republic. If a run fails or a field you need is missing, please open an issue in the **Issues** tab or write to us at info@deriverge.com. We respond in English and Czech. Runs can be scheduled in Apify Console or started from the **API** tab, which has examples for Python, JavaScript and cURL and works with Make, Zapier, n8n and the Apify MCP server. If the actor saves you time, a short review helps other people find it. # Changelog This Actor's version history is a separate document: https://apify.com/deriverge/substack-newsletter-directory/changelog.md # Actor input Schema ## `categories` (type: `array`): Substack category names, for example Technology, Business, Finance, Culture, U.S. Politics, Science, Health & Wellness, Crypto. Leave empty for all 33. ## `ranking` (type: `string`): Substack keeps two leaderboards per category. ## `maxPerCategory` (type: `integer`): From the top of the leaderboard down, 25 per page. ## `includePosts` (type: `boolean`): Also return the latest posts of every listed newsletter, with likes, comments and word count. Charged per post. ## `maxPostsPerNewsletter` (type: `integer`): Newest first. Archives go back years, so this is what keeps a broad run affordable. ## `publishedAfter` (type: `string`): Drop posts older than this date (YYYY-MM-DD). Dropped posts are never charged. ## `audience` (type: `string`): Substack marks each post as free for everyone or for paying subscribers. ## `keywords` (type: `array`): Keep only posts whose title, subtitle, description or preview contain at least one of these words, case-insensitive. Dropped posts are never charged. ## `newOnly` (type: `boolean`): Keeps a snapshot per watch name (or per saved task) and returns only newsletters that entered the leaderboard since the last run; the CHANGES record lists rank moves and drop-outs for free. ## `watchName` (type: `string`): Name of the snapshot used by the new-only mode, for example "my-newsletters". Runs from a saved task get a snapshot automatically even without a name. ## `maxItems` (type: `integer`): Hard cap on returned rows in the run. ## Actor input object example ```json { "categories": [ "Technology", "Business" ], "ranking": "all", "maxPerCategory": 25, "includePosts": false, "maxPostsPerNewsletter": 3, "audience": "all", "newOnly": false, "maxItems": 10000 } ``` # Actor output Schema ## `rows` (type: `string`): One row per newsletter and category with rank, author, subscriber tier, launch date and paid plans; latest posts on request. ## `changes` (type: `string`): Newsletters that entered or left the leaderboard and every rank move, compared with the previous snapshot. ## `summary` (type: `string`): Counts per row type, what each filter dropped, what was charged and notes. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "categories": [ "Technology", "Business" ], "maxPerCategory": 25 }; // Run the Actor and wait for it to finish const run = await client.actor("deriverge/substack-newsletter-directory").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "categories": [ "Technology", "Business", ], "maxPerCategory": 25, } # Run the Actor and wait for it to finish run = client.actor("deriverge/substack-newsletter-directory").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "categories": [ "Technology", "Business" ], "maxPerCategory": 25 }' | apify call deriverge/substack-newsletter-directory --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,deriverge/substack-newsletter-directory" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/rKwfK1wqleozInnbp/builds/u04BATGzHSIkoB9nr/openapi.json