Medium Scraper: articles, claps, responses, authors and full text

Scrape Medium articles by tag, author, publication or direct URL. For every article you get the title, author, publication, publish date, tags and the public text, plus clap count, number of responses, reading time and word count whenever Medium serves the article page. No login and no API key.

Medium often refuses article pages to automated visitors. Those articles are still saved from Medium's RSS feed, with title, author, date, tags and text, but without claps and responses (fromFeed: true). You pay the same per article either way.

What data you get

• Title, subtitle, URL and article ID
• Author name, username, profile link and follower count
• Publication name and slug
• Published and last-updated dates (ISO 8601)
• Claps, responses, reading time (minutes) and word count
• Tags, language and cover image
• Full article text as plain text. Member-only articles give the public preview, flagged with textIsPreviewOnly
• Which input (tag, author, publication or URL) found the article
• fromFeed: true when the article page could not be loaded and the item was built from the RSS feed instead. These items have the title, author, date, tags and text (full text when the feed carries it, otherwise a snippet, flagged with textIsPreviewOnly), but claps and responses are null

Sample result

{
  "id": "a64152b37c35",
  "url": "https://karpathy.medium.com/software-2-0-a64152b37c35",
  "title": "Software 2.0",
  "authorName": "Andrej Karpathy",
  "authorUsername": "karpathy",
  "publishedAt": "2017-11-11T22:18:53.751Z",
  "claps": 61030,
  "responses": 194,
  "readingTimeMinutes": 8.8,
  "tags": ["machine-learning", "artificial-intelligence", "programming", "software-development", "future"],
  "isMemberOnly": false,
  "text": "Software 2.0\n\nI sometimes see people refer to neural networks as just “another tool in your machine learning toolbox”…",
  "source": "author:karpathy"
}

How to use

Field	What it does
tags	Latest articles for each tag, e.g. artificial-intelligence, startup, javascript
authors	Latest articles by each author, e.g. karpathy or @karpathy
publications	Latest articles in each publication, e.g. towards-data-science, better-programming
articleUrls	Specific articles you already have links for
maxItems	Stop after this many articles (default 50)
includeFullText	Turn off to get metadata and stats only

Example inputs:

{ "tags": ["artificial-intelligence", "startup"], "maxItems": 40 }

{ "authors": ["karpathy"], "publications": ["towards-data-science"], "includeFullText": false }

{ "articleUrls": [{ "url": "https://karpathy.medium.com/software-2-0-a64152b37c35" }] }

Medium's tag, author and publication feeds list each source's most recent articles (usually the latest 10 to 25). For larger sets, add more tags, authors or publications, or pass article URLs directly.

Pricing

$3 per 1,000 articles, charged only for articles saved to your dataset. 200 articles cost about $0.60.

Use cases

• Track what is being written about your brand, product or market
• Collect articles for AI training sets, RAG pipelines and summarisation
• Find popular writers in a niche by claps and follower counts
• Monitor publications and competitors for new posts
• Content research: which topics and headlines get the most claps and responses

FAQ

Is it legal? The scraper only reads publicly available pages, the same ones any visitor sees without logging in. It does not get around Medium's paywall: member-only articles return the public preview only. You are responsible for how you use the data, including personal data and copyright.

Why did I get fewer articles than maxItems? Feeds only list recent articles. Add more sources or pass URLs.

Why do some articles have fromFeed: true and no claps? Medium sometimes refuses to serve an article page (HTTP 403), especially on author subdomains like karpathy.medium.com. When the article came from a tag, author or publication feed, the scraper still saves it using the feed data, so you do not lose it. An article you passed directly in articleUrls has no feed data to fall back on, so if its page is refused it is skipped with a warning. Using Apify Proxy usually reduces these refusals.

Can I run it on a schedule? Yes. Use Apify Schedules to run it daily and get new articles from your tags or authors, then send them to Google Sheets, Slack, Make, Zapier or a webhook with Apify integrations.

API

Call it from any language with the Apify API or client libraries (apify-client for JavaScript and Python), or connect it to an AI agent through the Apify MCP server.